canadadecor.ca
robots.txt

Robots Exclusion Standard data for canadadecor.ca

Resource Scan

Scan Details

Site Domain canadadecor.ca
Base Domain canadadecor.ca
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-09T14:40:12+00:00
Next Scan 2024-12-08T14:40:12+00:00

Last Successful Scan

Scanned2023-10-23T11:16:40+00:00
URL https://canadadecor.ca/robots.txt
Redirect https://www.canadadecor.ca:443/robots.txt
Redirect Domain www.canadadecor.ca
Redirect Base canadadecor.ca
Domain IPs 72.32.177.16
Redirect IPs 72.32.177.16
Response IP 72.32.177.16
Found Yes
Hash 172c8e0bcccf9624d7dd19069d193f5153c127042433145d224ac8f2e2a59a21
SimHash 6256f450d802

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1.0

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10.0

megaindex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5.0

redcarpet
addthis.com
ahrefsbot
baiduspider
becomebot
blexbot
camontspider
converacrawler
dealozbot
discobot
dotbot
exabot
fatbot
mj12bot
naverbot
ocelli
psbot
scoutjet
semanticbot
semanticdiscovery
shopwiki
sogou spider
sosospider
speedy
turnitinbot
twengabot
twiceler
ubicrawler
voyager
yandex
yanga

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.canadadecor.ca/sLuWgQr9/SiteMapIndex.xml

Comments

  • Sitemaps

Warnings

  • 1 invalid line.