webtoki.org
robots.txt

Robots Exclusion Standard data for webtoki.org

Resource Scan

Scan Details

Site Domain webtoki.org
Base Domain webtoki.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-08T18:48:14+00:00
Next Scan 2025-10-08T18:48:14+00:00

Last Successful Scan

Scanned2025-08-10T17:58:39+00:00
URL https://webtoki.org/robots.txt
Domain IPs 104.21.28.124, 172.67.146.44, 2606:4700:3031::ac43:922c, 2606:4700:3034::6815:1c7c
Response IP 104.21.28.124
Found Yes
Hash feda73edb49a4e47760670cad448497e9c9b74b3526c83c7c4b98ae61f46ebd5
SimHash 507cd162ec1a

Groups

gptbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

iboubot

Rule Path
Disallow /