foodhub.com
robots.txt
Robots Exclusion Standard data for foodhub.com
Resource Scan
Scan Details
Site Domain | foodhub.com |
Base Domain | foodhub.com |
Scan Status | Ok |
Last Scan | 2024-09-25T07:35:59+00:00 |
Next Scan | 2024-10-09T07:35:59+00:00 |
Last Scan
Scanned | 2024-09-25T07:35:59+00:00 |
URL | https://foodhub.com/robots.txt |
Domain IPs | 2600:9000:2721:200:6:59fa:a40:93a1, 2600:9000:2721:2200:6:59fa:a40:93a1, 2600:9000:2721:3600:6:59fa:a40:93a1, 2600:9000:2721:600:6:59fa:a40:93a1, 2600:9000:2721:9a00:6:59fa:a40:93a1, 2600:9000:2721:a800:6:59fa:a40:93a1, 2600:9000:2721:b600:6:59fa:a40:93a1, 2600:9000:2721:fc00:6:59fa:a40:93a1, 3.165.102.52, 3.165.102.54, 3.165.102.79, 3.165.102.98 |
Response IP | 18.161.49.19 |
Found | Yes |
Hash | 5596f4374b144805ba67eb522230577efaa62aaaaccdd2f04e2792cf1ac5480b |
SimHash | 0514ba72cdd0 |
Groups
*
Rule | Path |
---|---|
Allow | / |
Disallow | /selby-/the-pizza-grill/info |
Disallow | /profile |
Disallow | /account |
Disallow | /basket |
Disallow | /checkout |
Disallow | /orderHistory |
Disallow | /list/* |
Disallow | /*%5B |
Disallow | /*%5D |
Disallow | /*undefined |
Disallow | /*null |
Disallow | /area/ |
Disallow | /ordernow/ |
Disallow | /ordernow/* |
Disallow | /takeaway/ |
Disallow | /ordernow/*/*/* |
Disallow | /review/ |
Disallow | /*.php |
Disallow | /*.aspx |
Disallow | /*1000$ |
Other Records
Field | Value |
---|---|
sitemap | https://foodhub.com/sitemap.xml |