macalester.edu
robots.txt
Robots Exclusion Standard data for macalester.edu
Resource Scan
Scan Details
| Site Domain | macalester.edu |
| Base Domain | macalester.edu |
| Scan Status | Ok |
| Last Scan | 2024-10-26T18:24:57+00:00 |
| Next Scan | 2024-11-25T18:24:57+00:00 |
Last Scan
| Scanned | 2024-10-26T18:24:57+00:00 |
| URL | https://macalester.edu/robots.txt |
| Redirect | https://www.macalester.edu:443/robots.txt |
| Redirect Domain | www.macalester.edu |
| Redirect Base | macalester.edu |
| Domain IPs | 152.70.201.227 |
| Redirect IPs | 104.22.12.64, 104.22.13.64, 172.67.23.204, 2606:4700:10::6816:c40, 2606:4700:10::6816:d40, 2606:4700:10::ac43:17cc |
| Response IP | 172.67.23.204 |
| Found | Yes |
| Hash | 161060a7c4d4a378893cdd8457fdadd50a01dfdfbc43e404f0a46901d17f032b |
| SimHash | 6129cb0081b0 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /wp-admin/admin-ajax.php |
| Disallow | /wp-admin/ |
| Disallow | /160-* |
| Disallow | /*/outreach/ |
| Disallow | /brian-rosenberg/ |
| Disallow | /ga-test/ |
| Disallow | /wordpress-sandbox/ |
| Disallow | /drc-sandbox/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.macalester.edu/sitemap.xml |
| sitemap | https://www.macalester.edu/sitemap.rss |