voetbalheadlines.nl
robots.txt
Robots Exclusion Standard data for voetbalheadlines.nl
Resource Scan
Scan Details
Site Domain | voetbalheadlines.nl |
Base Domain | voetbalheadlines.nl |
Scan Status | Ok |
Last Scan | 2024-07-02T11:17:58+00:00 |
Next Scan | 2024-07-09T11:17:58+00:00 |
Last Scan
Scanned | 2024-07-02T11:17:58+00:00 |
URL | https://voetbalheadlines.nl/robots.txt |
Domain IPs | 185.104.28.21 |
Response IP | 185.104.28.21 |
Found | Yes |
Hash | 3e5f34f25b44d8b6eb816a4d52bc64ee9233b88601a8d059144186f70afa6ecb |
SimHash | 7a747dd0d8f1 |
Groups
*
No rules defined. All paths allowed.
Other Records
Field | Value |
---|---|
crawl-delay | 5 |
Other Records
Field | Value |
---|---|
sitemap | http://www.voetbalheadlines.nl/sitemap.xml |
sitemap | http://www.voetbalheadlines.nl/articles.xml |
Warnings
- 2 invalid lines.