turkeysforlife.com
robots.txt

Robots Exclusion Standard data for turkeysforlife.com

Resource Scan

Scan Details

Site Domain turkeysforlife.com
Base Domain turkeysforlife.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-06-23T21:24:09+00:00
Next Scan 2025-07-07T21:24:09+00:00

Last Successful Scan

Scanned2025-06-08T21:23:06+00:00
URL https://turkeysforlife.com/robots.txt
Redirect https://www.turkeysforlife.com/robots.txt
Redirect Domain www.turkeysforlife.com
Redirect Base turkeysforlife.com
Domain IPs 104.21.68.136, 172.67.195.235, 2606:4700:3031::ac43:c3eb, 2606:4700:3032::6815:4488
Redirect IPs 104.18.37.69, 172.64.150.187, 2606:4700:4400::6812:2545, 2606:4700:4400::ac40:96bb
Response IP 104.18.37.69
Found Yes
Hash 78034f864a6c7b0c1be253e69e18619efc81e0e3047a507952fc3147eb2ac17e
SimHash a4c16b00e610

Groups

*

Rule Path
Disallow /cdn-cgi/
Disallow /*add-to-cart%3D*

*

Rule Path
Disallow /wp-admin/
Disallow /product/

Comments

  • Prevent Crawling Unnecessary Endpoints - Dynamically added by BigScoots