roberts.edu
robots.txt
Robots Exclusion Standard data for roberts.edu
Resource Scan
Scan Details
| Site Domain | roberts.edu |
| Base Domain | roberts.edu |
| Scan Status | Ok |
| Last Scan | 2024-09-19T13:53:45+00:00 |
| Next Scan | 2024-10-19T13:53:45+00:00 |
Last Scan
| Scanned | 2024-09-19T13:53:45+00:00 |
| URL | https://roberts.edu/robots.txt |
| Redirect | https://www.roberts.edu/robots.txt |
| Redirect Domain | www.roberts.edu |
| Redirect Base | roberts.edu |
| Domain IPs | 63.151.148.64 |
| Redirect IPs | 63.151.148.64 |
| Response IP | 63.151.148.64 |
| Found | Yes |
| Hash | 3e38c7e4e63ecb6391149d7d888f23f89ab01ff38b146d76eec33290b7242482 |
| SimHash | 4b005800c502 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /app_browsers/ |
| Disallow | /app_code/ |
| Disallow | /app_data/ |
| Disallow | /app_plugins/ |
| Disallow | /aspnet_client/ |
| Disallow | /bin/ |
| Disallow | /bandn/ |
| Disallow | /changepw/ |
| Disallow | /changeemployeepw/ |
| Disallow | /config/ |
| Disallow | /fonts/ |
| Disallow | /honorsfiles/ |
| Disallow | /macroscripts/ |
| Disallow | /masterpages/ |
| Disallow | /metadata/ |
| Disallow | /nelnet/ |
| Disallow | /nfl-app/ |
| Disallow | /processrobertscart/ |
| Disallow | /requiressl/ |
| Disallow | /services/ |
| Disallow | /webservice/ |
| Disallow | /student_photos/ |
| Disallow | /home.aspx |
| Disallow | /age-open-house-thank-you/ |
| Disallow | /umbraco/ |
| Disallow | /registration/important-forms/registration-form/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.roberts.edu/xmlsitemap |