trader.ca
robots.txt

Robots Exclusion Standard data for trader.ca

Resource Scan

Scan Details

Site Domain trader.ca
Base Domain trader.ca
Scan Status Ok
Last Scan2024-06-04T01:59:50+00:00
Next Scan 2024-06-11T01:59:50+00:00

Last Scan

Scanned2024-06-04T01:59:50+00:00
URL http://trader.ca/robots.txt
Redirect https://www.autotrader.ca/robots.txt
Redirect Domain www.autotrader.ca
Redirect Base autotrader.ca
Domain IPs 52.186.125.163
Redirect IPs 107.154.247.249
Response IP 107.154.247.249
Found Yes
Hash debc343180867f14201d9a7c522e92b46a72f6ff95d7729315c5c44f604ed824
SimHash 1a76d94b8ac2

Groups

crystalsemanticsbot
crystalsemanticsbot
dotbot
grapeshotcrawler
maxpointcrawler
netseer crawler
oodlebot
ooyyo
scrapy
sightupbot
sogou
sogou spider
sogou web spider/4.0
trovitbot
wijubot
yandexbot
zumbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.autotrader.ca/autotrader_sitemap_index.xml

Comments

  • AutoTrader.ca
  • 2023-12-07