asu.edu
robots.txt
Robots Exclusion Standard data for asu.edu
Resource Scan
Scan Details
| Site Domain | asu.edu |
| Base Domain | asu.edu |
| Scan Status | Ok |
| Last Scan | 2024-11-12T22:38:18+00:00 |
| Next Scan | 2024-11-26T22:38:18+00:00 |
Last Scan
| Scanned | 2024-11-12T22:38:18+00:00 |
| URL | https://asu.edu/robots.txt |
| Redirect | https://www.asu.edu/robots.txt |
| Redirect Domain | www.asu.edu |
| Redirect Base | asu.edu |
| Domain IPs | 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133 |
| Redirect IPs | 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133, 2a04:4e42:200::645, 2a04:4e42:400::645, 2a04:4e42:600::645, 2a04:4e42::645 |
| Response IP | 199.232.46.133 |
| Found | Yes |
| Hash | efdca2101cceee171513efb59d0663137835f6e6db00e54044366e9783ca59ee |
| SimHash | 45fc4e856cf0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /courses/oasis/BabbScholarship/ |
| Disallow | /wwwstat/ |
| Disallow | /webtrends/ |
| Disallow | /stats/ |
| Disallow | /it/fyiada/ |
| Disallow | /it/fyitest/ |
| Disallow | /asu.sph |
| Disallow | /it/iris/dist/ |
| Disallow | /data_admin/WUG/ |
| Disallow | /alumni/programs/images/*.pdf$ |
| Disallow | /parking/ |
| Disallow | /news/_mm/ |
| Disallow | /news/_notes/ |
| Disallow | /news/_baks/ |
| Disallow | /news/MMWIP/ |
| Disallow | /news/archives/ |
| Disallow | /news/arts/ |
| Disallow | /news/asuinthenews/ |
| Disallow | /news/Backup/ |
| Disallow | /news/business/ |
| Disallow | /news/campus/ |
| Disallow | /news/community/ |
| Disallow | /news/faculty_students/ |
| Disallow | /news/features/ |
| Disallow | /news/forthemedia/ |
| Disallow | /news/images/ |
| Disallow | /news/includes/ |
| Disallow | /news/Insight/ |
| Disallow | /news/invest/ |
| Disallow | /news/investment/ |
| Disallow | /news/Library/ |
| Disallow | /news/mediacoverage/ |
| Disallow | /news/mediadinner/ |
| Disallow | /news/mediadinner06/ |
| Disallow | /news/printedition/ |
| Disallow | /news/publications/ |
| Disallow | /news/research/ |
| Disallow | /news/sc/ |
| Disallow | /news/mediadinner/ |
| Disallow | /news/stories/ |
| Disallow | /news/story_uploads/ |
| Disallow | /news/wiki_cicp/ |
| Disallow | /ui/entrepreneurship/ |
| Disallow | /aad/catalogs/ |
| Disallow | /aad/manuals/policyarchives/ |
| Disallow | /provost/articulation/ |
| Disallow | /counsel/forms/ |
| Disallow | /front/ |
Comments