zenbu.org
robots.txt

Robots Exclusion Standard data for zenbu.org

Resource Scan

Scan Details

Site Domain zenbu.org
Base Domain zenbu.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-17T21:53:02+00:00
Next Scan 2026-03-17T21:53:02+00:00

Last Successful Scan

Scanned2025-05-21T21:38:33+00:00
URL https://www.zenbu.org/robots.txt
Domain IPs 104.21.7.36, 172.67.135.178, 2606:4700:3032::ac43:87b2, 2606:4700:3033::6815:724
Response IP 172.67.135.178
Found Yes
Hash 3808ba05b65368095406df33e1dec95211fec4379e1024276ac6f4019e20e218
SimHash c626d0e0fa50

Groups

*

Rule Path
Disallow /harming/humans
Disallow /ignoring/human/orders
Disallow /harm/to/self
Disallow /account/
Disallow /account
Disallow /export/
Disallow /export
Disallow /entries/rss/
Disallow /entries/*/edit
Disallow /entries/report/
Disallow /entries/new
Disallow /entries/newcopy/
Disallow /entries/newneighbour/
Disallow /entries/version/
Disallow /users/all_updates_by/
Disallow /sessions/nonmobile
Disallow /sessions/mobile
Disallow /findnearest

mediapartners-google

Rule Path
Allow /

linguee

Rule Path
Disallow /

seoengbot

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

mozilla/5.0 (x11; compatible; semantic-visions.com crawler; httpclient 3.1)

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5