castorama.pl
robots.txt

Robots Exclusion Standard data for castorama.pl

Resource Scan

Scan Details

Site Domain castorama.pl
Base Domain castorama.pl
Scan Status Ok
Last Scan2024-11-07T12:15:59+00:00
Next Scan 2024-11-21T12:15:59+00:00

Last Scan

Scanned2024-11-07T12:15:59+00:00
URL https://castorama.pl/robots.txt
Redirect https://www.castorama.pl/robots.txt
Redirect Domain www.castorama.pl
Redirect Base castorama.pl
Domain IPs 65.9.112.109, 65.9.112.26, 65.9.112.46, 65.9.112.81
Redirect IPs 108.138.199.113, 108.138.199.48, 108.138.199.76, 108.138.199.9, 2600:9000:248d:1200:1d:7a2f:6c0:93a1, 2600:9000:248d:2400:1d:7a2f:6c0:93a1, 2600:9000:248d:5400:1d:7a2f:6c0:93a1, 2600:9000:248d:9400:1d:7a2f:6c0:93a1, 2600:9000:248d:ac00:1d:7a2f:6c0:93a1, 2600:9000:248d:f200:1d:7a2f:6c0:93a1, 2600:9000:248d:fa00:1d:7a2f:6c0:93a1, 2600:9000:248d:fc00:1d:7a2f:6c0:93a1
Response IP 108.156.22.61
Found Yes
Hash 1f156d5ee17b4da0f2a0fd1e76cf1fe39496b09724232cc3eac7c4bf0d0222df
SimHash 28179342fec3

Groups

*

Rule Path
Disallow /basket/
Disallow /checkout/
Disallow /customer/
Disallow /basket?*
Disallow *tab%3Dreviews
Disallow *_DARGS
Disallow *_requestid
Disallow *bvstate
Disallow *skuId
Disallow /basket
Disallow /facets/
Disallow /filters/
Disallow */reviews
Disallow /search/
Disallow /search?*
Disallow /*?*&*&*
Disallow *Price%3D*
Disallow *?Rating=*
Disallow *%26Rating%3D*
Disallow *Availability%3D*
Disallow *?*page=1$
Disallow */dynaTraceMonitor
Disallow *jsessionid
Disallow /assets/js/healthcheck.js
Disallow /*/ratings/*
Disallow /*/price-*
Disallow /product/productcomparison*
Disallow /stock-checker/*
Disallow navCount%3D
Disallow navAction%3D
Disallow /*Cena%3D*
Disallow *?rrec=

adsbot-google-mobile

Rule Path
Allow /*?utm_*
Allow /*%26utm_*

adsbot-google

Rule Path
Allow /*?utm_*
Allow /*%26utm_*

mediapartners-google
gsa-crawler

Rule Path
Disallow /

adequat

Rule Path
Disallow /

admantx bot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

corporama

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotmic dotbot

Rule Path
Disallow /

ellisphere

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

fetch

Rule Path
Disallow /

grapeshot bot

Rule Path
Disallow /

houzzbot

Rule Path
Disallow /

infoseek

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

leadbox

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

linko

Rule Path
Disallow /

mail.ru bot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

opinion-tracker

Rule Path
Disallow /

proxem

Rule Path
Disallow /

proximic bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

spotter

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

teleport

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

verticalsearch

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

wget

Rule Path
Disallow /

winello

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

archive.org_bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

soso spider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

sogou spider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

screaming frog seo spider

Rule Path
Allow /

net veille

Rule Path
Disallow /

paarly

Rule Path
Disallow /

spycommerce

Rule Path
Disallow /

pricing assistant

Rule Path
Disallow /

price observatory

Rule Path
Disallow /

workit

Rule Path
Disallow /

keox

Rule Path
Disallow /

pricejet

Rule Path
Disallow /

prixan

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.castorama.pl/static/sitemap.xml

Warnings

  • 1 invalid line.