Log In Sign Up

cluj.info
robots.txt

Robots Exclusion Standard data for cluj.info

Resource Scan

Scan Details

Site Domain cluj.info
Base Domain cluj.info
Scan Status Ok
Last Scan2026-01-21T18:23:05+00:00
Next Scan 2026-01-28T18:23:05+00:00

Last Scan

Scanned2026-01-21T18:23:05+00:00
URL https://cluj.info/robots.txt
Domain IPs 95.217.153.169
Response IP 95.217.153.169
Found Yes
Hash 597ea3710230915d2b4188be6174f821a48e2aed90c85495fbb202c7ba7c437b
SimHash 144f9632fff0

Groups

*

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /atbdp_orders
Back to top

Comments

  • Block CazoodleBot as it does not present correct accept content headers
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot
  • Block Bingbot
  • User-agent: bingbot
  • Disallow: /
Back to top