homecentre.com
robots.txt

Robots Exclusion Standard data for homecentre.com

Resource Scan

Scan Details

Site Domain homecentre.com
Base Domain homecentre.com
Scan Status Ok
Last Scan2024-10-04T10:23:31+00:00
Next Scan 2024-10-18T10:23:31+00:00

Last Scan

Scanned2024-10-04T10:23:31+00:00
URL https://homecentre.com/robots.txt
Redirect https://www.homecentre.com/robots.txt
Redirect Domain www.homecentre.com
Redirect Base homecentre.com
Domain IPs 104.18.235.231, 104.18.236.231, 2606:4700::6812:ebe7, 2606:4700::6812:ece7
Redirect IPs 104.18.235.231, 104.18.236.231, 2606:4700::6812:ebe7, 2606:4700::6812:ece7
Response IP 104.18.235.231
Found Yes
Hash e0e3a31b6c3435e7d740307f95bf339727234c8022b38f4d8ba2cfe17b0a725e
SimHash b1583b88ccf2

Groups

*

Rule Path
Allow /

*
sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

*

Rule Path
Disallow /*cart
Disallow /*checkout
Disallow /*my-account
Disallow /*fastpay
Disallow /*checkout/
Disallow /*aff
Disallow /*uat1*
Disallow /*?country
Disallow /*preBasket
Disallow /*%26discount%3D
Disallow /*%26sizeRank
Disallow /*%26price%3D
Disallow /*%26range%3D
Disallow /*manufacturerName
Disallow /*search
Disallow /*branchify?
Disallow /*login
Disallow /*register
Disallow /*helpae
Disallow /*helpsa
Disallow /*helpbh
Disallow /*helpkw
Disallow /*helpom
Disallow /*helpeg
Disallow /*helpqa

*

Rule Path
Disallow /*?q=
Disallow /*?lang=
Disallow /*?pl=
Disallow /*?request
Disallow /*?sc_wbt=1
Allow *

Other Records

Field Value
sitemap https://www.homecentre.com/ae/ar/sitemapindex.xml
sitemap https://www.homecentre.com/ae/en/sitemapindex.xml
sitemap https://www.homecentre.com/bh/ar/sitemapindex.xml
sitemap https://www.homecentre.com/bh/en/sitemapindex.xml
sitemap https://www.homecentre.com/sa/ar/sitemapindex.xml
sitemap https://www.homecentre.com/sa/en/sitemapindex.xml
sitemap https://www.homecentre.com/kw/ar/sitemapindex.xml
sitemap https://www.homecentre.com/kw/en/sitemapindex.xml
sitemap https://www.homecentre.com/qa/en/sitemapindex.xml
sitemap https://www.homecentre.com/qa/ar/sitemapindex.xml
sitemap https://www.homecentre.com/om/en/sitemapindex.xml
sitemap https://www.homecentre.com/om/ar/sitemapindex.xml
sitemap https://www.homecentre.com/eg/en/sitemapindex.xml
sitemap https://www.homecentre.com/eg/ar/sitemapindex.xml

Comments

  • Block crawling software
  • Block pages
  • URL parameters blocking for SEO
  • Sitemap

Warnings

  • `host` is not a known field.