htauto.com
robots.txt
Robots Exclusion Standard data for htauto.com
Resource Scan
Scan Details
| Site Domain | htauto.com |
| Base Domain | htauto.com |
| Scan Status | Ok |
| Last Scan | 2024-11-12T06:35:15+00:00 |
| Next Scan | 2024-11-19T06:35:15+00:00 |
Last Scan
| Scanned | 2024-11-12T06:35:15+00:00 |
| URL | https://htauto.com/robots.txt |
| Redirect | https://auto.hindustantimes.com/robots.txt |
| Redirect Domain | auto.hindustantimes.com |
| Redirect Base | hindustantimes.com |
| Domain IPs | 23.36.49.254, 2600:1413:b000:396::1833, 2600:1413:b000:398::1833 |
| Redirect IPs | 23.46.230.154, 23.46.230.155, 2600:1413:b000:13::b857:c192, 2600:1413:b000:13::b857:c19b |
| Response IP | 23.45.207.170 |
| Found | Yes |
| Hash | f9d5f2975a9124c57ee77aae5b5ed7e466067a21aa7b1fee107332f07307179a |
| SimHash | e2053c530933 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /*/on-road-price-*/images/* |
| Disallow | /*/on-road-price-*/images |
| Disallow | /*/images/on-road-price-*/* |
| Disallow | /*/on-road-price-*/exterior-* |
| Disallow | *?req* |
| Disallow | *?v1* |
| Disallow | *?vid* |
| Disallow | *?page* |
| Disallow | *?src* |
| Disallow | *?city* |
| Disallow | *?popup* |
| Disallow | *?priceWidgetCity* |
| Disallow | *?showdifference* |
| Disallow | *?topautomobile* |
| Disallow | *?mobMenu* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://auto.hindustantimes.com/google-news-sitemap.xml |
| sitemap | https://auto.hindustantimes.com/sitemap/index.xml |
| sitemap | https://auto.hindustantimes.com/section/sitemap.xml |