caso-design.de
robots.txt

Robots Exclusion Standard data for caso-design.de

Resource Scan

Scan Details

Site Domain caso-design.de
Base Domain caso-design.de
Scan Status Ok
Last Scan2024-05-26T09:30:40+00:00
Next Scan 2024-06-25T09:30:40+00:00

Last Scan

Scanned2024-05-26T09:30:40+00:00
URL https://caso-design.de/robots.txt
Domain IPs 49.12.117.48
Response IP 49.12.117.48
Found Yes
Hash 60fa9577a8ab5d286d8cac7ddfe757cc19de5ad21f4279448b2f2f630373c8a5
SimHash 6f11532c9808

Groups

*

Rule Path
Disallow /api/
Disallow /djcms/
Disallow /cart/
Disallow /basket/
Disallow /comparison/
Disallow /compare/
Disallow /checkout/
Disallow /payment/
Disallow /contact/
Disallow /accounts/
Disallow /favourites/
Disallow /history/
Disallow /filter/
Disallow /newsletter/
Disallow /productrating/
Disallow /setplanner/
Disallow /*designerlist
Disallow /*ajax
Disallow /restocknotifier/
Disallow /generate-sitemap/*
Disallow /*suggestsearch/*
Disallow /*search/*
Disallow /addressbook/
Disallow /accounts/address/create*
Disallow /three60-ajax/
Disallow /storelocator/retailerproductstock/
Disallow /public-api/*
Disallow /tracking/*
Disallow /discount/
Disallow */popup/*
Disallow */webservices/*
Allow /static/*

Other Records

Field Value
crawl-delay 5

crazywebcrawler-spider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bspider

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

iframely

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

googlebot

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

Other Records

Field Value
sitemap https://caso-design.de/media/sitemap/sitemap-caso-design.de.xml