swisscave.co.uk
robots.txt

Robots Exclusion Standard data for swisscave.co.uk

Resource Scan

Scan Details

Site Domain swisscave.co.uk
Base Domain swisscave.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-29T02:16:16+00:00
Next Scan 2025-12-13T02:16:16+00:00

Last Successful Scan

Scanned2025-10-22T00:30:27+00:00
URL https://swisscave.co.uk/robots.txt
Domain IPs 104.21.41.129, 172.67.165.1, 2606:4700:3032::6815:2981, 2606:4700:3035::ac43:a501
Response IP 104.21.41.129
Found Yes
Hash 0c93b46d7ee7fdbe3ea62c523c5a612a3f028acaf5dd98a3113f73ea1a9e054c
SimHash 2994b0ec6cb1

Groups

petalbot

Rule Path
Disallow /

*
geedoproductsearch

Rule Path
Disallow */admin/
Disallow */app/
Disallow */bin/
Disallow */dev/
Disallow */lib/
Disallow */phpserver/
Disallow */pkginfo/
Disallow */report/
Disallow */setup/
Disallow */update/
Disallow */var/
Disallow */vendor/
Disallow */checkout/
Disallow */customer/
Disallow */catalogsearch/
Disallow */sales/
Disallow */vault/
Disallow */swiss-engineering/customer-reviews/
Disallow /*?cat=*
Disallow /*?sort=*
Disallow /*?filter_demo=*

Other Records

Field Value
crawl-delay 100

Other Records

Field Value
sitemap https://swisscave.it/pub/media/sitemap/sitemap_it_it.xml
sitemap https://swisscave.ch/pub/media/sitemap/sitemap_it_ch.xml
sitemap https://swisscave.ch/pub/media/sitemap/sitemap_fr_ch.xml
sitemap https://swisscave.fr/pub/media/sitemap/sitemap_fr_fr.xml
sitemap https://swisscave.co.uk/pub/media/sitemap/sitemap_en_gb.xml
sitemap https://swisscave.ch/pub/media/sitemap/sitemap_de_ch.xml
sitemap https://swisscave.at/pub/media/sitemap/sitemap_de_at.xml
sitemap https://swisscave.de/pub/media/sitemap/sitemap_de_de.xml
sitemap https://swisscave.com/pub/media/sitemap/sitemap_en.xml
sitemap https://swisscave.com/pub/media/sitemap/sitemap_it.xml
sitemap https://swisscave.com/pub/media/sitemap/sitemap_fr.xml
sitemap https://swisscave.com/pub/media/sitemap/sitemap_de.xml

Comments

  • Crawlers Setup
  • Directories
  • Paths
  • XML Sitemaps