tf-robots.nl
robots.txt
Robots Exclusion Standard data for tf-robots.nl
Resource Scan
Scan Details
| Site Domain | tf-robots.nl |
| Base Domain | tf-robots.nl |
| Scan Status | Ok |
| Last Scan | 2024-09-10T20:34:15+00:00 |
| Next Scan | 2024-10-10T20:34:15+00:00 |
Last Scan
| Scanned | 2024-09-10T20:34:15+00:00 |
| URL | https://tf-robots.nl/robots.txt |
| Redirect | https://www.tf-robots.nl/robots.txt |
| Redirect Domain | www.tf-robots.nl |
| Redirect Base | tf-robots.nl |
| Domain IPs | 104.24.242.6, 104.24.243.6, 172.67.65.163, 2606:4700:20::6818:f206, 2606:4700:20::6818:f306, 2606:4700:20::ac43:41a3 |
| Redirect IPs | 104.24.242.6, 104.24.243.6, 172.67.65.163, 2606:4700:20::6818:f206, 2606:4700:20::6818:f306, 2606:4700:20::ac43:41a3 |
| Response IP | 104.24.242.6 |
| Found | Yes |
| Hash | 5d04df742f5990a055af590921c81d17a7ab66428cd4e1f8bacd82a26c5274bb |
| SimHash | 461fd470dc91 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /cart/ |
| Disallow | /checkout/ |
| Disallow | /customer/* |
| Disallow | /search/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.tf-robots.nl/rss/ |