danubehome.com
robots.txt

Robots Exclusion Standard data for danubehome.com

Resource Scan

Scan Details

Site Domain danubehome.com
Base Domain danubehome.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-14T07:16:05+00:00
Next Scan 2025-01-12T07:16:05+00:00

Last Successful Scan

Scanned2024-03-12T06:25:24+00:00
URL https://danubehome.com/robots.txt
Redirect https://www.danubehome.com/robots.txt
Redirect Domain www.danubehome.com
Redirect Base danubehome.com
Domain IPs 104.22.10.248, 104.22.11.248, 172.67.6.91, 2606:4700:10::6816:af8, 2606:4700:10::6816:bf8, 2606:4700:10::ac43:65b
Redirect IPs 104.22.10.248, 104.22.11.248, 172.67.6.91, 2606:4700:10::6816:af8, 2606:4700:10::6816:bf8, 2606:4700:10::ac43:65b
Response IP 104.22.10.248
Found Yes
Hash 38457af10d15dff6142936b3848728a3c623084b792488c1527f6acb11eb3c2e
SimHash 305a4982c4ba

Groups

*

Rule Path
Disallow */danube-deals/*
Disallow /my-account/*
Disallow /*query%3D
Disallow /cart/*
Disallow /checkout/*
Disallow /*amp%3D
Disallow /*?*bedroom_style=
Disallow /*?*cat=
Disallow /*?*chair_type=
Disallow /*?*cid=
Disallow /*?*color=
Disallow /*?*customFilters=
Disallow /*?*dir=
Disallow /*?*discount_range=
Disallow /*?*gclid=
Disallow /*?*material_type=
Disallow /*?*mattress_feel=
Disallow /*?*mattress_type=
Disallow /*?*p=
Disallow /*?*product_list_order=
Disallow /*?*SID=
Disallow /*?*style=
Disallow /*?*utm_source=
Disallow /*?*order=
Disallow /*?*priceRange=
Disallow /*?*sortKey=
Disallow /*?*sortDirection=
Disallow /dhub?*sku=
Disallow /*?*category_id
Disallow /*?*discount_percentage
Disallow /*?*product_list_dir=
Disallow /*?*discount_percent
Disallow /*tests
Disallow /*checkout/
Disallow /om/ar/c/search/
Disallow /om/en/c/search/
Disallow /ae/ar/c/search/
Disallow /ae/en/c/search/
Disallow /bh/ar/c/search/
Disallow /bh/en/c/search/
Disallow /qa/ar/c/search/
Disallow /qa/en/c/search/
Disallow /ae/en/c/furniture/sofas/filter
Disallow /om/en/c/furniture/sofas/filter
Disallow /bh/en/c/furniture/sofas/filter
Disallow /qa/en/c/furniture/sofas/filter
Disallow /ae/ar/c/furniture/sofas/filter
Disallow /om/ar/c/furniture/sofas/filter
Disallow /bh/ar/c/furniture/sofas/filter
Disallow /qa/ar/c/furniture/sofas/filter

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.danubehome.com/ae/en/sitemap.xml
sitemap https://www.danubehome.com/ae/ar/sitemap.xml
sitemap https://www.danubehome.com/om/en/sitemap.xml
sitemap https://www.danubehome.com/om/ar/sitemap.xml
sitemap https://www.danubehome.com/bh/en/sitemap.xml
sitemap https://www.danubehome.com/bh/ar/sitemap.xml
sitemap https://www.danubehome.com/qa/en/sitemap.xml
sitemap https://www.danubehome.com/qa/ar/sitemap.xml

Comments

  • URL parametersASSEM
  • URL ParametersWISITE
  • Block crawling software