Log In Sign Up

debenhams.ae
robots.txt

Robots Exclusion Standard data for debenhams.ae

Resource Scan

Scan Details

Site Domain debenhams.ae
Base Domain debenhams.ae
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2026-02-05T03:05:07+00:00
Next Scan 2026-05-06T03:05:07+00:00

Last Successful Scan

Scanned2025-06-16T10:44:16+00:00
URL https://debenhams.ae/robots.txt
Redirect https://www.debenhams.ae/robots.txt
Redirect Domain www.debenhams.ae
Redirect Base debenhams.ae
Domain IPs 104.18.78.44, 104.18.79.44, 2606:4700::6812:4e2c, 2606:4700::6812:4f2c
Redirect IPs 104.18.78.44, 104.18.79.44, 2606:4700::6812:4e2c, 2606:4700::6812:4f2c
Response IP 104.18.79.44
Found Yes
Hash 303a94907d210977761fed4535d4ac04e7446ae4bbc553afe5f208b460e43e52
SimHash be9e795acbf5

Groups

ahrefsbot

Rule Path
Disallow /

*

Rule Path
Allow /*?p=
Allow /catalog/seo_sitemap/category/
Disallow /catalogsearch/result/
Disallow /ru/catalogsearch/result
Disallow /en/catalogsearch/result
Disallow /search*
Disallow /ru/search*
Disallow /en/search*

*

Rule Path
Disallow /404/

*

Rule Path
Disallow */index.php/*
Disallow */catalog/product_compare/*
Disallow */catalog/category/view/*
Disallow */catalog/product/view/*
Disallow */catalogsearch/*
Disallow */checkout/*
Disallow */control/*
Disallow */contacts/*
Disallow */customer/*
Disallow */customize/*
Disallow */newsletter/*
Disallow */poll/*
Disallow */review/*
Disallow */sendfriend/*
Disallow */tag/*
Disallow */wishlist/*
Disallow */shopping/*
Disallow */storefinder/*
Disallow */ajax/*
Disallow /en/?___from_store=ru

*
*

Rule Path
Disallow /*.php$
Disallow /*?p=*&
Disallow /*?SID=
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/wc/robots.html
  • For syntax checking, see:
  • http://www.sxw.org.uk/computing/robots/check.html
  • Crawlers Setup
  • Disallow: /
  • Allowable Index
  • Directories
  • Paths (clean URLs)
  • Files
  • Paths (no clean URLs)
Back to top