knuspr.de
robots.txt

Robots Exclusion Standard data for knuspr.de

Resource Scan

Scan Details

Site Domain knuspr.de
Base Domain knuspr.de
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-10T20:06:44+00:00
Next Scan 2026-01-09T20:06:44+00:00

Last Successful Scan

Scanned2025-09-04T20:55:50+00:00
URL https://knuspr.de/robots.txt
Redirect https://www.knuspr.de/robots.txt
Redirect Domain www.knuspr.de
Redirect Base knuspr.de
Domain IPs 104.18.24.201, 104.18.25.201, 2606:4700::6812:18c9, 2606:4700::6812:19c9
Redirect IPs 104.18.24.201, 104.18.25.201, 2606:4700::6812:18c9, 2606:4700::6812:19c9
Response IP 104.18.25.201
Found Yes
Hash 8b240db8364bd6d1d143cce9fc1774238f50c0ebb3f941b8bff96b52e6461d20
SimHash 473a88b3eb91

Groups

*

Rule Path
Disallow /regal/*

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow *?products=*
Disallow */sort/price-asc
Disallow */sort/price-desc
Disallow */sort/unit-price-asc

Other Records

Field Value
sitemap https://www.knuspr.de/sitemap_products.xml
sitemap https://www.knuspr.de/sitemap_base.xml