aws.pro-football-reference.com
robots.txt
Robots Exclusion Standard data for aws.pro-football-reference.com
Resource Scan
Scan Details
| Site Domain | aws.pro-football-reference.com |
| Base Domain | pro-football-reference.com |
| Scan Status | Ok |
| Last Scan | 2024-10-23T09:00:02+00:00 |
| Next Scan | 2024-11-06T09:00:02+00:00 |
Last Scan
| Scanned | 2024-10-23T09:00:02+00:00 |
| URL | https://aws.pro-football-reference.com/robots.txt |
| Domain IPs | 34.230.173.210, 54.208.81.163, 54.88.103.86 |
| Response IP | 34.230.173.210 |
| Found | Yes |
| Hash | ed66b0b4b4ab5a4abb4348b90c88e1899d38613d1e39d91eebf1d1cb89a6c80b |
| SimHash | 4865d726c010 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /football/ |
| Disallow | /cgi-bin/search.cgi |
| Disallow | /player_search.fcgi |
| Disallow | /player_search.fcgi* |
| Disallow | /search/player_search.fcgi |
| Disallow | /search/player_search.fcgi* |
| Disallow | /my/ |
| Disallow | /players/*/*/splits |
| Disallow | /players/*/*/gamelog |
| Disallow | /players/*/*/big-games |
| Disallow | /players/*/*/field-goals |
| Disallow | /players/*/*/super-bowl |
| Disallow | /players/*/*/touchdowns |
| Disallow | /players/*/*/pick-sixes |
| Disallow | /players/*/*/penalties |
| Disallow | /players/*/*/fantasy |
| Disallow | /players/*/*/receiving-plays |
| Disallow | /players/*/*/red-zone-receiving-plays |
| Disallow | /players/*/*/passing-plays |
| Disallow | /players/*/*/red-zone-passing-plays |
| Disallow | /players/*/*/rushing-plays |
| Disallow | /players/*/*/red-zone-rushing-plays |
| Disallow | /req/ |
| Disallow | /short/ |
| Disallow | /nocdn/ |