Log In Sign Up

extrabuzz.in
robots.txt

Robots Exclusion Standard data for extrabuzz.in

Resource Scan

Scan Details

Site Domain extrabuzz.in
Base Domain extrabuzz.in
Scan Status Ok
Last Scan2026-02-26T00:20:20+00:00
Next Scan 2026-03-05T00:20:20+00:00

Last Scan

Scanned2026-02-26T00:20:20+00:00
URL https://extrabuzz.in/robots.txt
Domain IPs 208.91.199.170
Response IP 208.91.199.170
Found Yes
Hash 1905e74240b002d0720c290d2818bc6f49912cc5ef77e927e96afcb9392e5dcd
SimHash 295ee1c5c5f3

Groups

blexbot

Rule Path
Disallow /

serankingbacklinksbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /nogooglebot/

googlebot-image
googlebot-news
mediapartners-google

Rule Path
Allow /

*

Rule Path
Allow /
Back to top

Comments

  • Block BLEXBot to prevent excessive, non-human traffic that may trigger ad limits
  • Block SE Ranking Bot
  • Block Anthropic's ClaudeBot
  • Google's main crawler
  • You can add other specific exclusions here if needed.
  • Google Image Crawler
  • Allow all images by default
  • Google News Crawler
  • Allow all news content by default
  • Explicitly allow AdSense crawler to ensure ads can be verified
  • General rule for ALL other bots and crawlers (including Bing, Yandex, etc.)
  • By default, allow all content for other search engines
  • Optional: Add the path to your main sitemap file
  • Replace example.com with your actual domain
  • Sitemap: https://www.example.com/sitemap.xml
Back to top