ottos.ch
robots.txt

Robots Exclusion Standard data for ottos.ch

Resource Scan

Scan Details

Site Domain ottos.ch
Base Domain ottos.ch
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-02-15T13:07:18+00:00
Next Scan 2026-04-16T13:07:18+00:00

Last Successful Scan

Scanned2025-11-25T11:08:11+00:00
URL https://ottos.ch/robots.txt
Redirect https://www.ottos.ch/robots.txt
Redirect Domain www.ottos.ch
Redirect Base ottos.ch
Domain IPs 104.18.24.59, 104.18.25.59, 2606:4700::6812:183b, 2606:4700::6812:193b
Redirect IPs 104.18.24.59, 104.18.25.59, 2606:4700::6812:183b, 2606:4700::6812:193b
Response IP 104.18.24.59
Found Yes
Hash c59cc91dee72ca988d88507c13f8ee477a95cf93a97a14b91bdd256ec27da52d
SimHash 245019deccf4

Groups

*

Rule Path
Disallow */checkout/*
Disallow */order-confirmation*
Disallow */my-account*
Disallow */login
Disallow */multiuser-login
Disallow /api/*
Disallow */search*
Disallow /*?viewtype
Disallow /*?sort
Disallow /*%26viewtype
Disallow /*%26sort

Other Records

Field Value Comment
crawl-delay 10 10 seconds between page requests

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

facebookexternalhit/1.1

Rule Path
Disallow */checkout/*
Disallow */order-confirmation*
Disallow */my-account*
Disallow */login
Disallow */multiuser-login
Disallow /api/*
Disallow */search*
Disallow /*?viewtype
Disallow /*?sort
Disallow /*%26viewtype
Disallow /*%26sort

Other Records

Field Value Comment
crawl-delay 10 10 seconds between page requests

Other Records

Field Value
sitemap https://www.ottos.ch/sitemap.xml

Comments

  • For all robots
  • Block access to specific groups of pages
  • Allow search crawlers to discover the sitemap
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot
  • Definition for FacebookBot because it doesnt respect settings for Useragent *
  • Block access to specific groups of pages