castanet.net
robots.txt
Robots Exclusion Standard data for castanet.net
Resource Scan
Scan Details
| Site Domain | castanet.net |
| Base Domain | castanet.net |
| Scan Status | Ok |
| Last Scan | 2024-05-30T21:47:14+00:00 |
| Next Scan | 2024-06-06T21:47:14+00:00 |
Last Scan
| Scanned | 2024-05-30T21:47:14+00:00 |
| URL | https://castanet.net/robots.txt |
| Redirect | https://www.castanet.net/robots.txt |
| Redirect Domain | www.castanet.net |
| Redirect Base | castanet.net |
| Domain IPs | 104.26.12.15, 104.26.13.15, 172.67.74.243, 2606:4700:20::681a:c0f, 2606:4700:20::681a:d0f, 2606:4700:20::ac43:4af3 |
| Redirect IPs | 104.26.12.15, 104.26.13.15, 172.67.74.243, 2606:4700:20::681a:c0f, 2606:4700:20::681a:d0f, 2606:4700:20::ac43:4af3 |
| Response IP | 104.26.13.15 |
| Found | Yes |
| Hash | 16bcc9661968ce89ff378050799ad875172c81e5a0600e249e23169e6a121f7d |
| SimHash | c1110742b4b0 |
Groups
yahoo-mmaudvid
| Rule | Path |
|---|---|
| Disallow | /clients/ |
| Disallow | /admin/ |
| Disallow | /sales/ |
| Disallow | /cgi-bin/ |
| Disallow | /cgi-bin2/ |
| Disallow | /ccs/ |
| Disallow | /lists/ |
| Disallow | /logs/ |
| Disallow | /menus/ |
| Disallow | /quote/ |
| Disallow | /survey/ |
| Disallow | /traffic/ |
| Disallow | /webmail/ |
yahoo-mmcrawler
| Rule | Path |
|---|---|
| Disallow | /clients/ |
| Disallow | /admin/ |
| Disallow | /cgi-bin/ |
| Disallow | /cgi-bin2/ |
| Disallow | /ccs/ |
| Disallow | /lists/ |
| Disallow | /logs/ |
| Disallow | /menus/ |
| Disallow | /quote/ |
| Disallow | /survey/ |
| Disallow | /traffic/ |
| Disallow | /webmail/ |
vegi bot (we follow your robots.txt settings before crawling, you can slow down the bot by change the crawl-delay parameter in the settings.if you have an enquiry, please email to: abuse-report@terrykyleseoagency.com)
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /sales/ |
| Disallow | /postcomment.php |
| Allow | /cgi-bin/kelownaevents/ |
| Disallow | /cgi-bin/ |
| Allow | /templates/weather/images/ |
| Disallow | /templates/ |
| Disallow | /ccs/ |
| Disallow | /lists/ |
| Disallow | /logs/ |
| Disallow | /menus/ |
| Disallow | /quote/ |
| Disallow | /survey/ |
| Disallow | /traffic/ |
| Disallow | /webmail/ |
| Disallow | /events_dev/ |
| Disallow | /demo.htm |
| Disallow | /search.htm |
| Disallow | /crash.htm |
| Disallow | /index-jc.htm |
| Disallow | /chances.htm |
| Disallow | /chances1.htm |
| Disallow | /boats.htm |
| Disallow | /v.htm |
| Disallow | /index-irish.htm |
| Disallow | /index-noirish.htm |
| Disallow | /index-troy.htm |
| Disallow | /index-troy2.htm |
| Disallow | /troy/ |
| Disallow | /homepage/ |
| Disallow | /fred/ |
| Disallow | /real-v2/ |
| Disallow | /refresh/ |
| Disallow | /widgets/ |
| Disallow | /wine/example/ |
| Disallow | /realestatenew/ |
| Disallow | /cat/marketplace/home_leisure/store_-_hot_tub_essentials/ |
| Disallow | /clients/img/ |
| Disallow | /realestate-old/ |
| Disallow | /ryan/ |
| Disallow | /h/ |
| Disallow | /1/ |
| Disallow | /feeds/ |
| Disallow | /cdn-cgi/ |
| Disallow | /TCadmin/ |
Warnings
- 2 invalid lines.