southeasttexas.com
robots.txt
Robots Exclusion Standard data for southeasttexas.com
Resource Scan
Scan Details
| Site Domain | southeasttexas.com |
| Base Domain | southeasttexas.com |
| Scan Status | Ok |
| Last Scan | 2024-06-11T09:45:36+00:00 |
| Next Scan | 2024-07-11T09:45:36+00:00 |
Last Scan
| Scanned | 2024-06-11T09:45:36+00:00 |
| URL | https://www.southeasttexas.com/robots.txt |
| Domain IPs | 3.230.109.55, 34.224.116.99 |
| Response IP | 3.230.109.55 |
| Found | Yes |
| Hash | f649d8f8be4db2a5f6837ac05827cf0ef5a58f2606a753bb4f0711b559ea715b |
| SimHash | 9b1c4d054281 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /beaumont-marketplace/login/ |
| Disallow | /beaumont-marketplace/registration/ |
| Disallow | /beaumont-marketplace/search/ |
| Disallow | /beaumont-marketplace/account/ |
| Disallow | /beaumont-marketplace/basket/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.southeasttexas.com/beaumont-marketplace/sitemap.xml |
Comments