food-pi.com
robots.txt

Robots Exclusion Standard data for food-pi.com

Resource Scan

Scan Details

Site Domain food-pi.com
Base Domain food-pi.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2026-02-24T15:04:53+00:00
Next Scan 2026-04-25T15:04:53+00:00

Last Successful Scan

Scanned2025-12-27T14:19:18+00:00
URL https://food-pi.com/robots.txt
Domain IPs 164.90.148.208
Response IP 164.90.148.208
Found Yes
Hash abd54fa3c966b5eceac87d6ce81044cfc82015d6f2e2d306a43c441674f0a63a
SimHash 581c8900d850

Groups

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

*

Rule Path
Allow /