tastybits.de
robots.txt

Robots Exclusion Standard data for tastybits.de

Resource Scan

Scan Details

Site Domain tastybits.de
Base Domain tastybits.de
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-20T13:27:45+00:00
Next Scan 2025-09-03T13:27:45+00:00

Last Successful Scan

Scanned2025-08-05T12:13:15+00:00
URL https://tastybits.de/robots.txt
Redirect https://www.tastybits.de/robots.txt
Redirect Domain www.tastybits.de
Redirect Base tastybits.de
Domain IPs 185.125.174.22
Redirect IPs 185.125.174.22
Response IP 185.125.174.22
Found Yes
Hash 768e164bf0a476469b5587225a683ef52da0c8bea488e029f3c66079b9ec5756
SimHash 69101b42a017

Groups

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /
Disallow /

claudebot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

*

Rule Path
Disallow /wprm_print

Other Records

Field Value
sitemap https://www.tastybits.de/sitemap.xml

Warnings

  • `‍user-agent` is not a known field.