andindia.com
robots.txt
Robots Exclusion Standard data for andindia.com
Resource Scan
Scan Details
| Site Domain | andindia.com |
| Base Domain | andindia.com |
| Scan Status | Ok |
| Last Scan | 2026-01-06T13:00:37+00:00 |
| Next Scan | 2026-02-05T13:00:37+00:00 |
Last Scan
| Scanned | 2026-01-06T13:00:37+00:00 |
| URL | https://andindia.com/robots.txt |
| Redirect | https://www.andindia.com/robots.txt |
| Redirect Domain | www.andindia.com |
| Redirect Base | andindia.com |
| Domain IPs | 10.10.0.1, 10.10.0.2, 104.18.47.31, 172.64.156.225 |
| Redirect IPs | 104.18.37.63, 172.64.150.193 |
| Response IP | 172.64.150.193 |
| Found | Yes |
| Hash | b4497e4027699817e742592a13816c7adb970f957e7d0b18b6edd0a27479543c |
| SimHash | 523ed342efe3 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /*?q=* |
| Disallow | /on/demandware.store/* |
| Disallow | /default/ |
| Disallow | /Search-ShowAjax |
| Disallow | /login/ |
| Disallow | /dashboard/ |
| Disallow | /?q=search* |
| Disallow | /Search-Show |
| Disallow | Baiduspider |
| Disallow | %2BBaiduspider |
| Disallow | Sosospider%2B |
| Disallow | ezooms |
| Disallow | 008 |
| Disallow | Wotbox/2.01 |
| Disallow | dwbot |
| Disallow | asterias |
| Disallow | BackDoorBot/1.0 |
| Disallow | BlowFish/1.0 |
| Disallow | BotALot |
| Disallow | BuiltBotTough |
| Disallow | Bullseye/1.0 |
| Disallow | BunnySlippers |
| Disallow | Cegbfeieh |
| Disallow | CheeseBot |
| Disallow | CherryPicker |
| Disallow | CopyRightCheck |
| Disallow | cosmos |
| Disallow | Crescent |
| Disallow | DittoSpyder |
| Disallow | EroCrawler |
| Disallow | ExtractorPro |
| Disallow | Foobot |
| Disallow | Harvest/1.5 |
| Disallow | hloader |
| Disallow | httplib |
| Disallow | humanlinks |
| Disallow | ia_archiver |
| Disallow | InfoNaviRobot |
| Disallow | JennyBot |
| Disallow | LexiBot |
| Disallow | libWeb/clsHTTP |
| Disallow | MIIxpc |
| Disallow | moget |
| Disallow | NetAnts |
| Disallow | NICErsPRO |
| Disallow | Openfind |
| Disallow | ProPowerBot/2.14 |
| Disallow | ProWebWalker |
| Disallow | RepoMonkey |
| Disallow | RMA |
| Disallow | SiteSnagger |
| Disallow | SpankBot |
| Disallow | suzuran |
| Disallow | Szukacz/1.4 |
| Disallow | Teleport |
| Disallow | TeleportPro |
| Disallow | Telesoft |
| Disallow | TheNomad |
| Disallow | TightTwatBot |
| Disallow | Titan |
| Disallow | toCrawl/UrlDispatcher |
| Disallow | True_Robot |
| Disallow | True_Robot/1.0 |
| Disallow | turingos |
| Disallow | VCI |
| Disallow | WebStripper |
| Disallow | WebZip |
| Disallow | WebZip/4.0 |
| Disallow | Wget |
| Disallow | Wget/1.5.3 |
| Disallow | Wget/1.6 |
| Disallow | WWW-Collector-E |
| Disallow | Xenu%27s |
| Disallow | Zeus |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.andindia.com/sitemap_index.xml |
| sitemap | https://www.andindia.com/sitemap_0.xml |
Warnings
- 15 invalid lines.
Comments