Log In Sign Up

blog.frankel.ch
robots.txt

Robots Exclusion Standard data for blog.frankel.ch

Resource Scan

Scan Details

Site Domain blog.frankel.ch
Base Domain frankel.ch
Scan Status Ok
Last Scan2025-10-15T22:53:47+00:00
Next Scan 2025-11-14T22:53:47+00:00

Last Scan

Scanned2025-10-15T22:53:47+00:00
URL https://blog.frankel.ch/robots.txt
Domain IPs 104.21.31.45, 172.67.174.246, 2606:4700:3032::ac43:aef6, 2606:4700:3035::6815:1f2d
Response IP 104.21.31.45
Found Yes
Hash ee8084ba879b018743acd60bd4a5e33aefa654a76c395f51a0222e1556e13120
SimHash bc46f736edf0

Groups

*

Rule Path
Disallow

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /
Back to top

Comments

  • For all robots
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot
Back to top