hungryfoody.com
robots.txt
Robots Exclusion Standard data for hungryfoody.com
Resource Scan
Scan Details
Site Domain | hungryfoody.com |
Base Domain | hungryfoody.com |
Scan Status | Ok |
Last Scan | 2024-09-23T04:07:01+00:00 |
Next Scan | 2024-09-30T04:07:01+00:00 |
Last Scan
Scanned | 2024-09-23T04:07:01+00:00 |
URL | https://hungryfoody.com/robots.txt |
Domain IPs | 104.21.1.44, 172.67.128.132, 2606:4700:3030::ac43:8084, 2606:4700:3037::6815:12c |
Response IP | 104.21.1.44 |
Found | Yes |
Hash | 2ec10d7d5c8281941012633969b8ef6f2786406c7976b404c2c84cb855d9ab92 |
SimHash | 23d259b3ee81 |
Groups
slurp
Rule | Path |
---|---|
Disallow | /book-now/ |
Disallow | /go/ |
Disallow | /ajax/ |
Disallow | /member* |
Disallow | /report/ |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
bingbot
Rule | Path |
---|---|
Disallow | /book-now/ |
Disallow | /go/ |
Disallow | /ajax/ |
Disallow | /member* |
Disallow | /report/ |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
msnbot
Rule | Path |
---|---|
Disallow | /book-now/ |
Disallow | /go/ |
Disallow | /ajax/ |
Disallow | /member* |
Disallow | /report/ |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
*
Rule | Path |
---|---|
Disallow | /go/ |
Disallow | /ajax/ |
Disallow | /member* |
Disallow | /cdn-cgi/ |
Warnings
- 2 invalid lines.