simlea.com
robots.txt

Robots Exclusion Standard data for simlea.com

Resource Scan

Scan Details

Site Domain simlea.com
Base Domain simlea.com
Scan Status Ok
Last Scan2025-09-27T08:19:09+00:00
Next Scan 2025-10-27T08:19:09+00:00

Last Scan

Scanned2025-09-27T08:19:09+00:00
URL https://simlea.com/robots.txt
Domain IPs 104.21.59.74, 172.67.218.106, 2606:4700:3032::ac43:da6a, 2606:4700:3037::6815:3b4a
Response IP 172.67.218.106
Found Yes
Hash 25186b04d22186468c50ba2dfa3b730d54302284864945a0a380861a01e54f73
SimHash 11149f0069e3

Groups

*

Rule Path
Disallow /var/assets/
Disallow /uploads/

gptbot

Rule Path
Disallow /

ccbot

Product Comment
ccbot Common Crawl feeds many AIs
Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

Comments

  • Common AI/LLM crawlers