hungryweb.net
robots.txt
Robots Exclusion Standard data for hungryweb.net
Resource Scan
Scan Details
Site Domain | hungryweb.net |
Base Domain | hungryweb.net |
Scan Status | Ok |
Last Scan | 2024-09-27T18:42:24+00:00 |
Next Scan | 2024-10-27T18:42:24+00:00 |
Last Scan
Scanned | 2024-09-27T18:42:24+00:00 |
URL | https://hungryweb.net/robots.txt |
Redirect | https://www.hungryweb.net/robots.txt |
Redirect Domain | www.hungryweb.net |
Redirect Base | hungryweb.net |
Domain IPs | 104.21.44.7, 172.67.192.142, 2606:4700:3031::6815:2c07, 2606:4700:3035::ac43:c08e |
Redirect IPs | 104.21.44.7, 172.67.192.142, 2606:4700:3031::6815:2c07, 2606:4700:3035::ac43:c08e |
Response IP | 172.67.192.142 |
Found | Yes |
Hash | 351626278f3140d7d85cae2c96dc62e29b33a112f3e3efdaa06f06a426e8665f |
SimHash | 5058047085fa |
Groups
*
Rule | Path |
---|---|
Disallow | /app/ |
Disallow | /var/ |
Disallow | /store_closed.html |
Disallow | /demo/ |
Disallow | /demo/* |
Disallow | /downloads/* |
Disallow | /cs-cart-custom-development* |
Disallow | /newsletter* |
Disallow | /development/ |
Disallow | /hungryweb.net.api/ |
Disallow | /hungryweb.net.clients/ |
Disallow | /hungryweb.net.demo/ |
Disallow | /hw_ajax_search.php* |
Allow | *cloudfront.net* |
Other Records
Field | Value |
---|---|
sitemap | https://www.hungryweb.net/sitemap.xml |
Warnings
- 2 invalid lines.
Comments