search.engadget.com
robots.txt
Robots Exclusion Standard data for search.engadget.com
Resource Scan
Scan Details
Site Domain | search.engadget.com |
Base Domain | engadget.com |
Scan Status | Ok |
Last Scan | 2025-07-16T18:26:17+00:00 |
Next Scan | 2025-07-30T18:26:17+00:00 |
Last Scan
Scanned | 2025-07-16T18:26:17+00:00 |
URL | https://search.engadget.com/robots.txt |
Domain IPs | 106.10.218.137, 2406:2000:e4:1404::3000 |
Response IP | 106.10.218.137 |
Found | Yes |
Hash | 2af926cd14e8fd20cfa781cbb1dca94c470fdf5ce71ce7772bbe4019c2e06a1b |
SimHash | 74057b52c280 |
Groups
*
Rule | Path |
---|---|
Disallow | /search |
Disallow | /bin |
Disallow | /language |
Disallow | /yhs |
Disallow | /aol |
Disallow | /reviews |
Disallow | /click |
Disallow | /local |