thomas.edu
robots.txt
Robots Exclusion Standard data for thomas.edu
Resource Scan
Scan Details
| Site Domain | thomas.edu |
| Base Domain | thomas.edu |
| Scan Status | Ok |
| Last Scan | 2024-11-11T00:06:26+00:00 |
| Next Scan | 2024-12-11T00:06:26+00:00 |
Last Scan
| Scanned | 2024-11-11T00:06:26+00:00 |
| URL | https://thomas.edu/robots.txt |
| Redirect | http://www.thomas.edu/robots.txt |
| Redirect Domain | www.thomas.edu |
| Redirect Base | thomas.edu |
| Domain IPs | 141.193.213.20, 141.193.213.21 |
| Redirect IPs | 141.193.213.20, 141.193.213.21 |
| Response IP | 141.193.213.21 |
| Found | Yes |
| Hash | 061bbfb0d6df4aee4ee503bbb8418320e3c2d759b4f32d251a3c3223eb96271f |
| SimHash | 1920d8468513 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /?s= |
| Disallow | /page/*/?s= |
| Disallow | /search/ |
| Disallow | /wp-json/ |
| Disallow | /?rest_route= |
| Disallow | /wp-admin/ |
| Disallow | /wp-content/uploads/pres-search-mats/ |
| Disallow | /photo/ |
| Disallow | /list/ |
| Disallow | /author/ |
| Disallow | /*.gif$ |
| Disallow | /*.jpg$ |
| Disallow | /*.jpeg$ |
| Disallow | /*.png$ |
| Disallow | /*.svg$ |
| Disallow | /*.bmp$ |
| Disallow | /*.ico$ |
| Disallow | /TC/email-signature/linkedin.jpg |
| Disallow | /TC/email-signature/you-tube.jpg |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.thomas.edu/sitemap_index.xml |
Comments