Log In Sign Up

isrgrajan.com
robots.txt

Robots Exclusion Standard data for isrgrajan.com

Resource Scan

Scan Details

Site Domain isrgrajan.com
Base Domain isrgrajan.com
Scan Status Ok
Last Scan2026-02-14T09:43:14+00:00
Next Scan 2026-02-21T09:43:14+00:00

Last Scan

Scanned2026-02-14T09:43:14+00:00
URL https://isrgrajan.com/robots.txt
Domain IPs 104.21.36.102, 172.67.192.82, 2606:4700:3031::6815:2466, 2606:4700:3035::ac43:c052
Response IP 172.67.192.82
Found Yes
Hash 00e9f30eef8a7c1020b94ac1763b79ec693e5a837c7c08af9cbe4f4573c92a8b
SimHash 4184c8d16ba5

Groups

googlebot-image

Rule Path
Allow /

mediapartners-google*

Rule Path
Allow /

*

Rule Path
Allow /assets/
Allow /wp-includes/js/
Allow /wp-admin/admin-ajax.php
Allow /cdn-cgi/*
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /xmlrpc.php
Disallow /api/
Disallow /docs/
Disallow /search
Disallow /comments/
Disallow /url/
Disallow /dereferer/*
Disallow /*/trackback/
Disallow /*/comments/
Disallow /*?s=
Disallow /*?replytocom=
Disallow /*?share=
Disallow /*utm_%3D

Other Records

Field Value
crawl-delay 3
Back to top

Other Records

Field Value
sitemap https://www.isrgrajan.com/sitemap_index.xml
Back to top

Comments

  • Sitemap for better indexing
  • Allow Googlebot for images
  • Allow Google AdSense bots for ads
  • General bots (applies to all crawlers including GPTBot, Bingbot, etc.)
  • Disallow admin, backend, and low-value areas
  • Balanced crawl pacing for all bots
Back to top