kaufland.de
robots.txt

Robots Exclusion Standard data for kaufland.de

Resource Scan

Scan Details

Site Domain kaufland.de
Base Domain kaufland.de
Scan Status Ok
Last Scan2024-09-14T12:34:49+00:00
Next Scan 2024-09-21T12:34:49+00:00

Last Scan

Scanned2024-09-14T12:34:49+00:00
URL https://kaufland.de/robots.txt
Redirect https://www.kaufland.de/robots.txt
Redirect Domain www.kaufland.de
Redirect Base kaufland.de
Domain IPs 34.149.249.52
Redirect IPs 34.149.249.52
Response IP 34.149.249.52
Found Yes
Hash 1c9fc87e8c3598f18573ab56746c80a8996b3da3eced3b090ca2c0615e27b598
SimHash 40c680e8a730

Groups

*

Rule Path
Disallow /cart/*
Disallow /cart-new/*
Disallow /checkout/*
Disallow /item/search/*page
Disallow /s/
Allow /shops/*/$
Disallow /shops/*/*
Disallow /account/
Disallow *autosuggest*
Disallow /backend/account/v1/widget
Disallow /optmzly/df-json/
Disallow /backend/iam/
Disallow /backend/search/
Disallow /backend/navigation/
Disallow /backend/tracking/
Disallow /backend/home-page/
Disallow /backend/recommendations/
Disallow /backend/product-detail-page/
Disallow *%26st%3D*
Disallow *%26ac%3D*
Disallow *id_unit%3D*

mediapartners-google

Rule Path
Allow /item/search/
Allow /s/

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

petalbot

Rule Path
Disallow /

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1