Log In Sign Up

chennaiiq.com
robots.txt

Robots Exclusion Standard data for chennaiiq.com

Resource Scan

Scan Details

Site Domain chennaiiq.com
Base Domain chennaiiq.com
Scan Status Ok
Last Scan2026-02-07T06:53:53+00:00
Next Scan 2026-02-14T06:53:53+00:00

Last Scan

Scanned2026-02-07T06:53:53+00:00
URL https://chennaiiq.com/robots.txt
Domain IPs 104.21.7.172, 172.67.187.243, 2606:4700:3031::ac43:bbf3, 2606:4700:3035::6815:7ac
Response IP 172.67.187.243
Found Yes
Hash 24b056fdd28011b20f580d7e8cb45715f556af8304561b2f1804402c37c7804e
SimHash 4540821040a3

Groups

*

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /*?sort=
Disallow /*?filter=
Back to top

Other Records

Field Value
sitemap https://chennaiiq.com/sitemap-index.xml
sitemap https://chennaiiq.com/sitemap-static.xml
sitemap https://chennaiiq.com/sitemap-guides.xml
Back to top

Comments

  • ChennaiIQ Robots.txt
  • https://chennaiiq.com - India's Complete Information Portal
  • Canonical host
  • Default rule: Allow all crawlers
  • Block API endpoints (not meant for indexing)
  • Block admin section
  • Block query parameters that create duplicate content
  • Note: ?page= is allowed for crawling paginated content (stations, trains)
  • Pages use rel="prev/next" and self-referencing canonicals for SEO
  • Main Sitemap Index (references all other sitemaps)
  • Individual Sitemaps (for faster discovery)
  • TODO: Uncomment when location sitemaps are generated
  • Sitemap: https://chennaiiq.com/sitemap-locations-1.xml
  • Sitemap: https://chennaiiq.com/sitemap-locations-2.xml
  • ... (up to 12 files for 557K+ locations)
  • Sitemap: https://chennaiiq.com/sitemap-banking-1.xml
  • Sitemap: https://chennaiiq.com/sitemap-postal-1.xml
  • Sitemap: https://chennaiiq.com/sitemap-railway.xml
  • Crawl-delay: Removed - let Google decide optimal crawl rate
  • Google ignores Crawl-delay anyway, and we want fast indexing
Back to top

Warnings

  • `host` is not a known field.
Back to top