urdu.news18.com
robots.txt
Robots Exclusion Standard data for urdu.news18.com
Resource Scan
Scan Details
| Site Domain | urdu.news18.com |
| Base Domain | news18.com |
| Scan Status | Ok |
| Last Scan | 2024-10-22T18:06:15+00:00 |
| Next Scan | 2024-11-05T18:06:15+00:00 |
Last Scan
| Scanned | 2024-10-22T18:06:15+00:00 |
| URL | https://urdu.news18.com/robots.txt |
| Domain IPs | 23.36.49.159 |
| Response IP | 23.54.57.187 |
| Found | Yes |
| Hash | 6eb91280f1eb77e67b3a36cdad8ce51283b1cd4cab3e6b0165f0f49c4826c541 |
| SimHash | d821c3307d17 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /api/ |
| Disallow | /includes/ |
| Disallow | /uc/ |
| Disallow | /SolrPhpClient/ |
| Disallow | /1039154/ |
| Disallow | /tag/*/page-*/ |
| Disallow | /tag/*/news/page-*/ |
| Disallow | /tag/*/videos/page-*/ |
| Disallow | /*/error_log* |
| Disallow | /*/access_log* |
| Disallow | /tag/*/photogallery/logs/access_log* |
| Disallow | /trending/page-*/logs/access_log* |
| Disallow | /trending/page-*/page-*/logs/error_log* |
| Disallow | /tag/*/logs/access_log* |
| Disallow | /tag/*/logs/error_log* |
| Disallow | /*/page-*/page-* |
| Disallow | /ghmc-election-result-*/* |
| Disallow | /scorecard/ |
| Disallow | /amp/tag/*/page-*/ |
| Disallow | /*/page-*/$ |
| Disallow | /image-sitemap |
| Disallow | /videos-sitemap |
| Disallow | /articles-sitemap |
| Disallow | /amp/tag/*/photogallery/ |
| Disallow | /amp/tag/*/videos/ |
| Disallow | /amp/tag/*/news/ |
| Disallow | /news/- |
| Disallow | /tag/*/photogallery/ |
| Disallow | /tag/*/videos/ |
| Disallow | /tag/*/news/ |
| Disallow | /videos/- |
| Disallow | /photogallery/- |
| Disallow | /bypoll-2021/ |
| Disallow | /*/undefined/ |
| Disallow | /amp/tag/*/videos/page-*/ |
| Disallow | /amp/tag/*/news/page-*/ |
| Disallow | /amp/tag/*/photogallery/page-*/ |
| Disallow | /byline/*/photogallery/ |
| Disallow | /assembly-elections-2022/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://urdu.news18.com/commonfeeds/v1/urd/sitemap/webstories-sitemap-index.xml |
| sitemap | https://urdu.news18.com/commonfeeds/v1/urd/sitemap-video-index.xml |
| sitemap | https://urdu.news18.com/commonfeeds/v1/urd/sitemap-image-index.xml |
| sitemap | https://urdu.news18.com/commonfeeds/v1/urd/sitemap/google-news.xml |
| sitemap | https://urdu.news18.com/commonfeeds/v1/urd/sitemap-index.xml |
| sitemap | https://urdu.news18.com/commonfeeds/v1/urd/sitemap/today |