nieuwsdump.nl
robots.txt
Robots Exclusion Standard data for nieuwsdump.nl
Resource Scan
Scan Details
Site Domain | nieuwsdump.nl |
Base Domain | nieuwsdump.nl |
Scan Status | Ok |
Last Scan | 2024-06-27T01:38:35+00:00 |
Next Scan | 2024-07-04T01:38:35+00:00 |
Last Scan
Scanned | 2024-06-27T01:38:35+00:00 |
URL | https://nieuwsdump.nl/robots.txt |
Domain IPs | 185.104.28.21 |
Response IP | 185.104.28.21 |
Found | Yes |
Hash | 0557256886f1c5b0e009c6c693ab5db9d972ff08d575261ae78082545b02d187 |
SimHash | 6af76d90d5e0 |
Groups
*
No rules defined. All paths allowed.
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
Warnings
- 2 invalid lines.