crctlessons.com
robots.txt

Robots Exclusion Standard data for crctlessons.com

Resource Scan

Scan Details

Site Domain crctlessons.com
Base Domain crctlessons.com
Scan Status Ok
Last Scan2026-01-10T07:05:59+00:00
Next Scan 2026-01-17T07:05:59+00:00

Last Scan

Scanned2026-01-10T07:05:59+00:00
URL https://crctlessons.com/robots.txt
Redirect http://www.crctlessons.com/robots.txt
Redirect Domain www.crctlessons.com
Redirect Base crctlessons.com
Domain IPs 104.21.86.25, 172.67.214.59, 2606:4700:3034::6815:5619, 2606:4700:3034::ac43:d63b
Redirect IPs 104.21.86.25, 172.67.214.59, 2606:4700:3034::6815:5619, 2606:4700:3034::ac43:d63b
Response IP 104.21.86.25
Found Yes
Hash f20812c2a21c95d439f53cf58e60cc134eda5dbbdc9bd3876c7aa4d3a6c7f6e0
SimHash 4d54d9752753

Groups

bingbot

Rule Path
Allow /

yisouspider

Rule Path
Allow /

sogou web spider

Rule Path
Allow /

baiduspider

Rule Path
Allow /

googlebot

Rule Path
Allow /

yandexbot

Rule Path
Allow /

*

Rule Path
Disallow /

Warnings

  • 2 invalid lines.