buysell.com
robots.txt

Robots Exclusion Standard data for buysell.com

Resource Scan

Scan Details

Site Domain buysell.com
Base Domain buysell.com
Scan Status Ok
Last Scan2024-05-22T03:40:51+00:00
Next Scan 2024-05-29T03:40:51+00:00

Last Scan

Scanned2024-05-22T03:40:51+00:00
URL http://www.buysell.com/robots.txt
Redirect https://www.autotrader.ca/robots.txt?bsat=buysell
Redirect Domain www.autotrader.ca
Redirect Base autotrader.ca
Domain IPs 52.186.125.163
Redirect IPs 107.154.247.249
Response IP 107.154.247.249
Found Yes
Hash debc343180867f14201d9a7c522e92b46a72f6ff95d7729315c5c44f604ed824
SimHash 1a76d94b8ac2

Groups

crystalsemanticsbot
crystalsemanticsbot
dotbot
grapeshotcrawler
maxpointcrawler
netseer crawler
oodlebot
ooyyo
scrapy
sightupbot
sogou
sogou spider
sogou web spider/4.0
trovitbot
wijubot
yandexbot
zumbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.autotrader.ca/autotrader_sitemap_index.xml

Comments

  • AutoTrader.ca
  • 2023-12-07