ltu.edu
robots.txt
Robots Exclusion Standard data for ltu.edu
Resource Scan
Scan Details
| Site Domain | ltu.edu |
| Base Domain | ltu.edu |
| Scan Status | Ok |
| Last Scan | 2024-11-16T08:40:14+00:00 |
| Next Scan | 2024-12-16T08:40:14+00:00 |
Last Scan
| Scanned | 2024-11-16T08:40:14+00:00 |
| URL | https://www.ltu.edu/robots.txt |
| Domain IPs | 74.235.42.172 |
| Response IP | 74.235.42.172 |
| Found | Yes |
| Hash | 3957a5c0fa3f3a65fbe714c4a41a1f30639d035349a5f9bf1c1067700e091c8d |
| SimHash | 283577128681 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | |
| Disallow | /cgi-bin/ |
| Disallow | /cm/* |
| Disallow | /cm_content_/ |
| Disallow | /external_attach/* |
| Disallow | /library/ |
| Disallow | /webroot/ |
| Disallow | /demo/ |
| Disallow | /admin/ |
| Disallow | /js/ |
| Disallow | /login/ |
| Disallow | /xml/ |
| Disallow | /blogs/ |
| Disallow | /uploads/* |
| Disallow | /data/courses/* |
| Disallow | /*.pdf$ |
| Disallow | /*.doc$ |
| Disallow | /*.docx$ |
| Disallow | /*.xls$ |
| Disallow | /*.xlsx$ |
| Disallow | /*.ppt$ |
| Disallow | /*.pptx$ |
| Disallow | /*.csv$ |
| Disallow | /*.rtf$ |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.ltu.edu/sitemap.xml |
Warnings
- 2 invalid lines.