autos.timesunion.com
robots.txt
Robots Exclusion Standard data for autos.timesunion.com
Resource Scan
Scan Details
| Site Domain | autos.timesunion.com |
| Base Domain | timesunion.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-05-20T06:55:27+00:00 |
| Next Scan | 2024-05-27T06:55:27+00:00 |
Last Successful Scan
| Scanned | 2024-04-19T05:21:15+00:00 |
| URL | https://autos.timesunion.com/robots.txt |
| Redirect | https://cars.timesunion.com/robots.txt |
| Redirect Domain | cars.timesunion.com |
| Redirect Base | timesunion.com |
| Domain IPs | 98.129.228.69 |
| Redirect IPs | 40.124.28.75 |
| Response IP | 40.124.28.75 |
| Found | Yes |
| Hash | dcb1b989c1848ac917ca70b05e0b6b8605b7eafc1c8d340079cf74c295089ea1 |
| SimHash | 80068a500313 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /Dealer/VehicleDetailsPrintOut |
| Disallow | /cars-for-sale*? |
| Allow | /cars-for-sale*?state=*&city=* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://autopublishers.com/sitemap.xml |