sports-reference.com
robots.txt
Robots Exclusion Standard data for sports-reference.com
Resource Scan
Scan Details
| Site Domain | sports-reference.com |
| Base Domain | sports-reference.com |
| Scan Status | Ok |
| Last Scan | 2024-04-28T04:16:21+00:00 |
| Next Scan | 2024-05-05T04:16:21+00:00 |
Last Scan
| Scanned | 2024-04-28T04:16:21+00:00 |
| URL | https://sports-reference.com/robots.txt |
| Redirect | https://www.sports-reference.com/robots.txt |
| Redirect Domain | www.sports-reference.com |
| Redirect Base | sports-reference.com |
| Domain IPs | 104.18.40.39, 172.64.147.217, 2606:4700:4400::6812:2827, 2606:4700:4400::ac40:93d9 |
| Redirect IPs | 104.18.40.39, 172.64.147.217, 2606:4700:4400::6812:2827, 2606:4700:4400::ac40:93d9 |
| Response IP | 172.64.147.217 |
| Found | Yes |
| Hash | f281b66a4d963fac44c2f807726194b81d08ea735faa5302630da7c3e9a855c1 |
| SimHash | 783240738b13 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /bb/ |
| Disallow | /olympics/athlete_search.cgi |
| Disallow | /cfb/search.cgi |
| Disallow | /cbb/search.cgi |
| Disallow | /cbb/boxscores/index.cgi?* |
| Disallow | /cbb/players/*/splits/*/?* |
| Disallow | /cbb/players/*/splits/*?* |
| Disallow | /cbb/players/*/splits/* |
| Disallow | /cbb/players/*/splits* |
| Disallow | /cfb/boxscores/index.cgi?* |
| Disallow | /cfb/players/*/splits/*/?* |
| Disallow | /cfb/players/*/splits/*?* |
| Disallow | /cfb/players/*/splits/* |
| Disallow | /cfb/players/*/splits* |
| Disallow | /cfb/schools/*/*/splits/*/?* |
| Disallow | /cfb/schools/*/*/splits/*?* |
| Disallow | /cfb/schools/*/*/splits/* |
| Disallow | /cfb/schools/*/*/splits* |
| Disallow | /cfb/req/ |
| Disallow | /cfb/short/ |
| Disallow | /cfb/nocdn/ |
| Disallow | /cbb/req/ |
| Disallow | /cbb/short/ |
| Disallow | /cbb/nocdn/ |