freshlabels.com
robots.txt

Robots Exclusion Standard data for freshlabels.com

Resource Scan

Scan Details

Site Domain freshlabels.com
Base Domain freshlabels.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-04T18:26:09+00:00
Next Scan 2026-02-02T18:26:09+00:00

Last Successful Scan

Scanned2025-03-16T16:40:31+00:00
URL https://freshlabels.com/robots.txt
Redirect https://www.freshlabels.com/robots.txt
Redirect Domain www.freshlabels.com
Redirect Base freshlabels.com
Domain IPs 104.21.84.190, 172.67.196.23, 2606:4700:3030::6815:54be, 2606:4700:3034::ac43:c417
Redirect IPs 104.21.84.190, 172.67.196.23, 2606:4700:3030::6815:54be, 2606:4700:3034::ac43:c417
Response IP 172.67.196.23
Found Yes
Hash e948b660a91da41793ddc3c825f49276c6883f225335d658fcb6494091aa2f49
SimHash 5076dc50e69b

Groups

*

Rule Path
Disallow

Other Records

Field Value
crawl-delay 2

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

awariorssbot
awariosmartbot
dotbot
dataforseobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

ahrefsbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.freshlabels.de/sitemaps/sitemap-com.xml