cicada2024.com
robots.txt

Robots Exclusion Standard data for cicada2024.com

Resource Scan

Scan Details

Site Domain cicada2024.com
Base Domain cicada2024.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-20T09:28:21+00:00
Next Scan 2025-11-19T09:28:21+00:00

Last Successful Scan

Scanned2025-06-30T00:33:45+00:00
URL https://cicada2024.com/robots.txt
Redirect https://www.wtransporter.com/robots.txt
Redirect Domain www.wtransporter.com
Redirect Base wtransporter.com
Domain IPs 104.26.0.43, 104.26.1.43, 172.67.69.50, 2606:4700:20::681a:12b, 2606:4700:20::681a:2b, 2606:4700:20::ac43:4532
Redirect IPs 104.21.7.145, 172.67.187.154, 2606:4700:3030::ac43:bb9a, 2606:4700:3032::6815:791
Response IP 104.21.7.145
Found Yes
Hash 386f0612df671ed653f06a47bd0d8f00b00c5174e0e743c1eba366599d4acc8c
SimHash 629a88c2e4e6

Groups

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Comments

  • Cloudflare Managed Robots.txt to block AI related bots - START
  • Cloudflare Managed Robots.txt to block AI related bots - END