crock.recipes
robots.txt

Robots Exclusion Standard data for crock.recipes

Resource Scan

Scan Details

Site Domain crock.recipes
Base Domain crock.recipes
Scan Status Ok
Last Scan2026-03-15T03:26:34+00:00
Next Scan 2026-03-22T03:26:34+00:00

Last Scan

Scanned2026-03-15T03:26:34+00:00
URL https://crock.recipes/robots.txt
Domain IPs 104.21.76.51, 172.67.188.171, 2606:4700:3030::6815:4c33, 2606:4700:3032::ac43:bcab
Response IP 104.21.76.51
Found Yes
Hash 18dc377d7bb9b0a8cdb39630581f22297440930b948badc02fd0949be4f9deb4
SimHash 49188852c591

Groups

*

Rule Path
Allow /

gptbot

Rule Path
Allow /

claudebot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

googleother

Rule Path
Allow /

Other Records

Field Value
sitemap https://crock.recipes/sitemap_index.xml