punjab.news18.com
robots.txt
Robots Exclusion Standard data for punjab.news18.com
Resource Scan
Scan Details
| Site Domain | punjab.news18.com |
| Base Domain | news18.com |
| Scan Status | Ok |
| Last Scan | 2024-09-25T18:05:26+00:00 |
| Next Scan | 2024-10-09T18:05:26+00:00 |
Last Scan
| Scanned | 2024-09-25T18:05:26+00:00 |
| URL | https://punjab.news18.com/robots.txt |
| Domain IPs | 23.36.50.99, 2600:1413:b000:38c::3379, 2600:1413:b000:390::3379 |
| Response IP | 96.16.118.125 |
| Found | Yes |
| Hash | 2fe0bf37426c7364b6df153e05ecbaf5063e0ed9b830e026c27759406935c31f |
| SimHash | 4c594958c523 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /marketing/ |
| Disallow | /uc/ |
| Disallow | /amp/tag/*/videos/ |
| Disallow | /amp/tag/*/photogallery/ |
| Disallow | /amp/tag/*/news/ |
| Disallow | /assembly-elections-2022/ |
| Disallow | /photogallery/*/page-* |
| Disallow | /page-*/ |
| Disallow | /explainedpage-*/page-* |
| Disallow | /news/page-*/ |
| Disallow | /tag/*/photogallery |
| Disallow | /tag/*/videos |
| Disallow | /tag/*/news |
| Disallow | /fatehgarh-sahib/page-* |
| Disallow | /byline/*/news/page-*/ |
| Disallow | /assembly-elections/%3Astate_slug%3A/*/ |
| Disallow | /photogallery/*/privacy_policy.html |
| Disallow | /cartoon/page-*/privacy_policy.html |
| Disallow | /national/page-*/privacy_policy.html |
| Disallow | /videos/page-*/privacy_policy.htmlpage-*/ |
| Disallow | /south-bengal/page-*/privacy_policy.html |
| Disallow | /scorecard/ |
| Disallow | /election_framework_next/ |
| Disallow | /vod/ |
| Disallow | /amp/tag/*/page-*/ |
| Disallow | /byline/page-*/videos/page-*/ |
| Disallow | /videos/page-*/ |
| Disallow | /assembly-elections/ |
| Disallow | /videos/*.compage-*/page-*/ |
| Disallow | /tag/*/page-*/ |
| Disallow | /punjab-local-body-elections-2021/ |
| Disallow | /amp/*/page-*/ |
| Disallow | /videos/*/page-*/ |
| Disallow | /byline/page-*/page-*/ |
| Disallow | /byline/*.html/*/page-*/ |
| Disallow | /uncategorizedpage-*page-*/page-*/ |
| Disallow | /amp/assembly-elections-2022/ |
| Disallow | /1 |
| Disallow | /2 |
| Disallow | /3 |
| Disallow | /4 |
| Disallow | /5 |
| Disallow | /6 |
| Disallow | /7 |
| Disallow | /8 |
| Disallow | /9 |
| Disallow | /0 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://punjab.news18.com/commonfeeds/v1/pan/sitemap/today |
| sitemap | https://punjab.news18.com/commonfeeds/v1/pan/sitemap-index.xml |
| sitemap | https://punjab.news18.com/commonfeeds/v1/pan/sitemap/google-news.xml |
| sitemap | https://punjab.news18.com/commonfeeds/v1/pan/sitemap-image-index.xml |
| sitemap | https://punjab.news18.com/commonfeeds/v1/pan/sitemap-video-index.xml |
| sitemap | https://punjab.news18.com/commonfeeds/v1/pan/sitemap/webstories-sitemap-index.xml |