linsfood.com
robots.txt

Robots Exclusion Standard data for linsfood.com

Resource Scan

Scan Details

Site Domain linsfood.com
Base Domain linsfood.com
Scan Status Ok
Last Scan2024-10-05T01:05:18+00:00
Next Scan 2024-10-12T01:05:18+00:00

Last Scan

Scanned2024-10-05T01:05:18+00:00
URL https://linsfood.com/robots.txt
Domain IPs 104.21.90.165, 172.67.158.50, 2606:4700:3032::ac43:9e32, 2606:4700:3037::6815:5aa5
Response IP 104.21.90.165
Found Yes
Hash b677ae25880de3a9f217117cbf51757784d69664bf3997f3c37900d4738a8273
SimHash 2c7ed8408b93

Groups

*

Rule Path
Disallow *wprm-print*
Disallow *my-linsfood-recipe-collection*
Disallow */?s=*
Disallow */search*

Other Records

Field Value
crawl-delay 300

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.linsfood.com/sitemap.xml