neopets.com
robots.txt
Robots Exclusion Standard data for neopets.com
Resource Scan
Scan Details
| Site Domain | neopets.com |
| Base Domain | neopets.com |
| Scan Status | Ok |
| Last Scan | 2024-10-03T00:27:17+00:00 |
| Next Scan | 2024-10-10T00:27:17+00:00 |
Last Scan
| Scanned | 2024-10-03T00:27:17+00:00 |
| URL | https://neopets.com/robots.txt |
| Redirect | https://www.neopets.com/robots.txt |
| Redirect Domain | www.neopets.com |
| Redirect Base | neopets.com |
| Domain IPs | 23.96.35.235 |
| Redirect IPs | 184.87.193.134, 184.87.193.136 |
| Response IP | 42.99.140.202 |
| Found | Yes |
| Hash | c37ad24735fed21ce0b1c09d71e74f33047d0110016057450a6269fe334b6641 |
| SimHash | f55ed0c04771 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /objects.phtml |
| Disallow | /browseshop.phtml |
| Disallow | /login.phtml |
| Disallow | /login/* |
| Disallow | /signup/* |
| Disallow | /*process_* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |