citizenwatch.eu
robots.txt

Robots Exclusion Standard data for citizenwatch.eu

Resource Scan

Scan Details

Site Domain citizenwatch.eu
Base Domain citizenwatch.eu
Scan Status Ok
Last Scan2024-05-31T17:38:14+00:00
Next Scan 2024-06-30T17:38:14+00:00

Last Scan

Scanned2024-05-31T17:38:14+00:00
URL https://citizenwatch.eu/robots.txt
Domain IPs 88.99.126.235
Response IP 88.99.126.235
Found Yes
Hash 4153b147ac28036a377bbf8c957befc44bac0839bf6dd9acf89f8d8dde3c0057
SimHash 6f11532e9808

Groups

*

Rule Path
Disallow /api/
Disallow /djcms/
Disallow /cart/
Disallow /basket/
Disallow /comparison/
Disallow /compare/
Disallow /checkout/
Disallow /payment/
Disallow /contact/
Disallow /accounts/
Disallow /favourites/
Disallow /history/
Disallow /filter/
Disallow /newsletter/
Disallow /productrating/
Disallow /setplanner/
Disallow /*designerlist
Disallow /*ajax
Disallow /restocknotifier/
Disallow /generate-sitemap/*
Disallow /*suggestsearch/*
Disallow /*search/*
Disallow /addressbook/
Disallow /accounts/address/create*
Disallow /three60-ajax/
Disallow /storelocator/retailerproductstock/
Disallow /public-api/*
Disallow /tracking/*
Disallow /discount/
Disallow */popup/*
Disallow */webservices/*
Allow /static/*

Other Records

Field Value
crawl-delay 5

crazywebcrawler-spider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bspider

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

iframely

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

googlebot

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

Other Records

Field Value
sitemap https://citizenwatch.eu/media/sitemap/sitemap-citizenwatch.eu.xml