schaefer-shop.lu
robots.txt

Robots Exclusion Standard data for schaefer-shop.lu

Resource Scan

Scan Details

Site Domain schaefer-shop.lu
Base Domain schaefer-shop.lu
Scan Status Ok
Last Scan2024-11-11T03:27:04+00:00
Next Scan 2024-11-25T03:27:04+00:00

Last Scan

Scanned2024-11-11T03:27:04+00:00
URL https://schaefer-shop.lu/robots.txt
Redirect https://www.schaefer-shop.lu/robots.txt
Redirect Domain www.schaefer-shop.lu
Redirect Base schaefer-shop.lu
Domain IPs 167.235.217.66
Redirect IPs 184.87.193.69, 184.87.193.79, 2600:1413:b000:1e::17d1:2e4c, 2600:1413:b000:1e::17d1:2e61
Response IP 23.45.207.166
Found Yes
Hash 8872ddcf58bf684191d29c5b5f1908f6fe05a43d3c8bc52bb22a025b8d74762e
SimHash c92c9857a201

Groups

*

Rule Path
Disallow /shop/
Disallow /h/
Disallow /search/
Disallow /print/
Disallow /.well-known/
Disallow /g/
Disallow /stores/
Disallow /cms/blog/search
Disallow /pricing/
Disallow /mapping/
Disallow /fr/mapping/
Disallow /user/profile/
Disallow /produkt/
Disallow /order/
Disallow /rating/
Disallow /en/
Disallow /product/comparison/
Disallow /product/reminder/article/
Disallow /user/lang/
Disallow /p/*https%3A//media
Disallow /fr/p/*https%3A//media

gptbot

Rule Path
Allow /ueberuns
Allow /fr/a-propos-de-nous
Allow /p/
Allow /fr/p/
Disallow /

chatgpt-user

Rule Path
Allow /p/
Allow /fr/p/
Disallow /

ccbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /order/cart

Other Records

Field Value
crawl-delay 10

siteauditbot

Rule Path
Disallow /order/cart

Other Records

Field Value
crawl-delay 10

semrushbot

Rule Path
Disallow /order/cart

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.schaefer-shop.lu/product/sitemap/sitemap.xml
sitemap https://www.schaefer-shop.lu/product/sitemap/sitemap-image.xml
sitemap https://www.schaefer-shop.lu/fr/product/sitemap/sitemap.xml

Comments

  • Sitemaps
  • Other Bots:

Warnings

  • 2 invalid lines.