indianexpress.com
robots.txt
Robots Exclusion Standard data for indianexpress.com
Resource Scan
Scan Details
| Site Domain | indianexpress.com |
| Base Domain | indianexpress.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-10-17T12:03:34+00:00 |
| Next Scan | 2025-01-15T12:03:34+00:00 |
Last Successful Scan
| Scanned | 2023-03-29T21:50:35+00:00 |
| URL | https://indianexpress.com/robots.txt |
| Domain IPs | 23.39.4.128 |
| Response IP | 23.39.4.128 |
| Found | Yes |
| Hash | 61a2daf252fb74bf258696cb944f43023026bdde385f78af323f515c09986175 |
| SimHash | b3118940a60c |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /wp-admin/ |
| Allow | /wp-admin/admin-ajax.php |
| Disallow | /1050432/ |
| Disallow | /dummy/ |
| Disallow | /homepage/ |
| Disallow | /prime-suspect/ |
| Disallow | /ie/daily/ |
| Disallow | /static-home-ad-template/ |
| Disallow | /static-article-ad-template/ |
| Disallow | /entertainment/wallpapers/ |
| Disallow | /entertainment/section/Television/617/ |
| Disallow | /entertainment/old/screenawards08/best-editing.html |
| Disallow | /entertainment/fullstory.php |
| Disallow | /citiesfullstory.php |
| Disallow | /citiesarticle/ |
| Disallow | /interstitial/ |
| Disallow | /Blog-IE/ |
| Disallow | /ajax/ |
| Disallow | /ramnath-goenka-foundation/dump |
| Disallow | /entertainment/old/ |
| Disallow | /citiesfullstory.php?newsid=* |
| Disallow | /inner_video.php?videoId=* |
| Disallow | /storyOld.php?storyId=* |
| Disallow | /entertainment/old/fullstory.php?content_id=* |
| Disallow | /entertainment/old/* |
| Disallow | /comments_print.php?content_id=* |
| Disallow | /search/* |
| Disallow | /comments/* |
| Disallow | /*?s |
| Disallow | /*?newsid |
| Disallow | /quote/* |
| Disallow | /openinappbtn/* |
| Disallow | /apple-app-site-association |
| Disallow | /ie_byline_authors |
| Disallow | /ie_byline_authors/* |
| Disallow | /login/* |
| Disallow | /registration/* |
| Disallow | /my-account/ |
| Disallow | /reset-password/ |
| Disallow | /forgot-password/ |
| Disallow | /change-password/ |
| Disallow | /subscriptions/ |
| Disallow | /my-subscriptions/ |
| Disallow | /purchase/ |
| Disallow | /thank-you/ |
| Disallow | /my-premium/ |
| Disallow | /campaign/ |
| Disallow | /campaign-premium/ |
| Disallow | /campaign-test/ |
| Disallow | /international-offer/ |
| Disallow | /epaper-upsc/ |
| Disallow | /special-offer/ |
| Disallow | /brandlover/ |
| Disallow | /limited-offer/ |
| Disallow | /all-access-offer/ |
| Disallow | /campaign-2/ |
| Disallow | /campaign-3/ |
| Disallow | /campaign-4/ |
| Disallow | /campaign-premium-2/ |
| Disallow | /campaign-premium-3/ |
| Disallow | /campaign-premium-4/ |
| Disallow | /campaign-epaper/ |
| Disallow | /campaign-epaper-2/ |
| Disallow | /campaign-epaper-3/ |
| Disallow | /campaign-epaper-4/ |
| Disallow | /campaign-all-access/ |
| Disallow | /campaign-all-access-2/ |
| Disallow | /campaign-all-access-3/ |
| Disallow | /campaign-all-access-4/ |
| Disallow | /diwali-offer/ |
| Disallow | /product-page-testing/ |
| Disallow | /offers/ |
| Disallow | /exclusive-benefits-subscription/ |
| Disallow | /digital_1/ |
| Disallow | /digital_2/ |
| Disallow | /digital_3/ |
| Disallow | /allaccess_1/ |
| Disallow | /allaccess_2/ |
| Disallow | /allaccess_3/ |
| Disallow | /paytm/ |
| Disallow | /partner/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://indianexpress.com/news-sitemap.xml |
| sitemap | https://indianexpress.com/image-sitemap.xml |
| sitemap | https://indianexpress.com/video-sitemap.xml |
| sitemap | https://indianexpress.com/category-sitemap.xml |
| sitemap | https://indianexpress.com/allsitemap.xml |
| sitemap | https://indianexpress.com/aboutsitemapindex.xml |
| sitemap | https://indianexpress.com/webstories-sitemap.xml |
| sitemap | https://indianexpress.com/sitemap.xml |
Comments