auto.hindustantimes.com
robots.txt
Robots Exclusion Standard data for auto.hindustantimes.com
Resource Scan
Scan Details
| Site Domain | auto.hindustantimes.com |
| Base Domain | hindustantimes.com |
| Scan Status | Ok |
| Last Scan | 2024-05-04T19:41:09+00:00 |
| Next Scan | 2024-05-18T19:41:09+00:00 |
Last Scan
| Scanned | 2024-05-04T19:41:09+00:00 |
| URL | https://auto.hindustantimes.com/robots.txt |
| Domain IPs | 13.225.4.2, 13.225.4.5, 13.225.4.88, 13.225.4.9, 2600:9000:21b4:3600:8:f240:3380:93a1, 2600:9000:21b4:3c00:8:f240:3380:93a1, 2600:9000:21b4:5400:8:f240:3380:93a1, 2600:9000:21b4:600:8:f240:3380:93a1, 2600:9000:21b4:6200:8:f240:3380:93a1, 2600:9000:21b4:8000:8:f240:3380:93a1, 2600:9000:21b4:9e00:8:f240:3380:93a1, 2600:9000:21b4:ba00:8:f240:3380:93a1 |
| Response IP | 13.225.4.88 |
| Found | Yes |
| Hash | 873f5e1250af2b508c0888b8ebefe9b1c9e07ebef4f185f89d9600d31b6da7c7 |
| SimHash | ea10ac522b11 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /*/on-road-price-*/images/* |
| Disallow | /*/on-road-price-*/images |
| Disallow | /*/images/on-road-price-*/* |
| Disallow | /*/on-road-price-*/exterior-* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://auto.hindustantimes.com/google-news-sitemap.xml |
| sitemap | https://auto.hindustantimes.com/sitemap/index.xml |
| sitemap | https://auto.hindustantimes.com/section/sitemap.xml |