archive.rpgamer.com
robots.txt
Robots Exclusion Standard data for archive.rpgamer.com
Resource Scan
Scan Details
| Site Domain | archive.rpgamer.com |
| Base Domain | rpgamer.com |
| Scan Status | Ok |
| Last Scan | 2025-12-09T21:59:39+00:00 |
| Next Scan | 2026-01-08T21:59:39+00:00 |
Last Scan
| Scanned | 2025-12-09T21:59:39+00:00 |
| URL | https://archive.rpgamer.com/robots.txt |
| Domain IPs | 104.26.12.20, 104.26.13.20, 172.67.75.137, 2606:4700:20::681a:c14, 2606:4700:20::681a:d14, 2606:4700:20::ac43:4b89 |
| Response IP | 104.26.12.20 |
| Found | Yes |
| Hash | 55e73c3a816cb86b31b3114a54aad8fd06aa0408a53d2c39dd59ec1b3720a320 |
| SimHash | 424a481ce7e8 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | newbie.html | Newbie staff training guide shouldn't be parsed |
| Disallow | /board/ | Message boards shouldn't be indexed |
| Disallow | /bios/ | Bios contain emails and shouldn't be parsed |
| Disallow | /staff/ | - |