cross-stitch-corner.de
robots.txt

Robots Exclusion Standard data for cross-stitch-corner.de

Resource Scan

Scan Details

Site Domain cross-stitch-corner.de
Base Domain cross-stitch-corner.de
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan3/30/2025, 5:13:47 AM
Next Scan 6/28/2025, 5:13:47 AM

Last Successful Scan

Scanned11/8/2024, 5:03:16 AM
URL https://cross-stitch-corner.de/robots.txt
Redirect https://www.cross-stitch-corner.de/robots.txt
Redirect Domain www.cross-stitch-corner.de
Redirect Base cross-stitch-corner.de
Domain IPs 217.160.0.163
Redirect IPs 2001:8d8:100f:f000::290, 217.160.0.163
Response IP 217.160.0.163
Found Yes
Hash 1938426c0935a4408a4572214a85aa54f238ee2816d8c529de3a1b11feb461c6
SimHash a81a8940c0fd

Groups

*

Rule Path
Disallow /admin/
Disallow /Core/
Disallow /tmp/
Disallow /views/
Disallow /Setup/
Disallow /log/
Disallow /newsletter/
Disallow /en/newsletter/
Disallow /index.php?cl=newsletter
Disallow /AGB/
Disallow /en/Terms-and-Conditions/
Disallow /warenkorb/
Disallow /en/cart/
Disallow /index.php?cl=basket
Disallow /mein-konto/
Disallow /en/my-account/
Disallow /index.php?cl=account
Disallow /mein-merkzettel/
Disallow /en/my-wishlist/
Disallow /index.php?cl=account_noticelist
Disallow /mein-wunschzettel/
Disallow /en/my-gift-registry/
Disallow /index.php?cl=account_wishlist
Disallow /konto-eroeffnen/
Disallow /en/open-account/
Disallow /index.php?cl=register
Disallow /passwort-vergessen/
Disallow /en/forgot-password/
Disallow /index.php?cl=forgotpwd
Disallow /index.php?cl=moredetails
Disallow /index.php?cl=review
Disallow /index.php?cl=search
Disallow /EXCEPTION_LOG.txt
Disallow /*?cl=newsletter
Disallow /*%26cl%3Dnewsletter
Disallow /*?cl=basket
Disallow /*%26cl%3Dbasket
Disallow /*?cl=account
Disallow /*%26cl%3Daccount
Disallow /*?cl=account_noticelist
Disallow /*%26cl%3Daccount_noticelist
Disallow /*?cl=account_wishlist
Disallow /*%26cl%3Daccount_wishlist
Disallow /*?cl=register
Disallow /*%26cl%3Dregister
Disallow /*?cl=forgotpwd
Disallow /*%26cl%3Dforgotpwd
Disallow /*?cl=moredetails
Disallow /*%26cl%3Dmoredetails
Disallow /*?cl=review
Disallow /*%26cl%3Dreview
Disallow /*?cl=search
Disallow /*%26cl%3Dsearch
Disallow /*%26fnc%3Dtobasket
Disallow /*%26fnc%3Dtocomparelist
Disallow /*%26addcompare%3D
Disallow /*/sid/
Disallow /*?sid=
Disallow /*%26sid%3D
Disallow /*?cur=
Disallow /*%26cur

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

Comments

  • wildcards at the end, because of some crawlers see it as errors
  • User-agent: Google-Extended
  • Disallow: /