kent.ca
robots.txt

Robots Exclusion Standard data for kent.ca

Resource Scan

Scan Details

Site Domain kent.ca
Base Domain kent.ca
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-16T10:45:31+00:00
Next Scan 2025-09-17T10:45:31+00:00

Last Successful Scan

Scanned2025-08-17T08:38:32+00:00
URL https://kent.ca/robots.txt
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 151.101.129.124
Found Yes
Hash d52ffa6ccd1fc44ec42dc6e64d1226c38110f7dc90343ed4416715607a048d0d
SimHash 4b3cbc57e332

Groups

yandexbot

Rule Path
Disallow /

siteauditbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

semrushbot-si

Rule Path
Allow /

*

Rule Path
Disallow /lib/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /catalog/
Disallow /customer/
Disallow /sendfriend/
Disallow /review/
Disallow /*SID%3D
Disallow /search/?q=*
Disallow /redirect/*
Disallow /en/search/?q=*
Disallow /en/catalogsearch/result/?q=*
Disallow /fr/search/?q=*
Disallow /fr/catalogsearch/result/?q=*
Disallow /search/?q=*
Disallow /catalogsearch/result/?q=*

Other Records

Field Value
sitemap https://kent.ca/media/sitemap/kent_en.xml
sitemap https://kent.ca/media/sitemap/kent_fr.xml