buyautoparts.com
robots.txt
Robots Exclusion Standard data for buyautoparts.com
Resource Scan
Scan Details
| Site Domain | buyautoparts.com |
| Base Domain | buyautoparts.com |
| Scan Status | Ok |
| Last Scan | 2024-06-18T15:27:25+00:00 |
| Next Scan | 2024-06-25T15:27:25+00:00 |
Last Scan
| Scanned | 2024-06-18T15:27:25+00:00 |
| URL | https://buyautoparts.com/robots.txt |
| Redirect | https://www.buyautoparts.com/robots.txt |
| Redirect Domain | www.buyautoparts.com |
| Redirect Base | buyautoparts.com |
| Domain IPs | 104.22.60.216, 104.22.61.216, 172.67.29.232, 2606:4700:10::6816:3cd8, 2606:4700:10::6816:3dd8, 2606:4700:10::ac43:1de8 |
| Redirect IPs | 104.22.60.216, 104.22.61.216, 172.67.29.232, 2606:4700:10::6816:3cd8, 2606:4700:10::6816:3dd8, 2606:4700:10::ac43:1de8 |
| Response IP | 172.67.29.232 |
| Found | Yes |
| Hash | a136e4fa15fa2711fd6ca60cb4a72828ff8432f592f87c2712252b229afafdd1 |
| SimHash | 3210cb51c7d0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /webapp/ |
| Disallow | /test* |
| Disallow | /ShopByVehicle.epc |
| Disallow | /*.epc |
| Disallow | /*.epc* |
| Disallow | /*.wws |
| Disallow | /*.wws* |
| Disallow | /Ebay/ |
| Disallow | /Ebay* |
| Disallow | /plugins/* |
| Disallow | /blog/category/* |
| Disallow | /blog/tag/* |
| Disallow | /proredirect_nc.asp |
| Disallow | /couponcart.asp |
| Disallow | /bannerjson/bannerjson/bapbanner.json |
| Disallow | /dynamic_carparts_year.asp |
| Disallow | /sendoutofstockmail.asp |
| Disallow | /ppcbannerajax.asp |
| Disallow | /buynowajax_nc.asp |
| Disallow | /qacaptchacheck.asp |
| Disallow | /search-result.asp* |
| Disallow | /chat |
| Disallow | /chat.asp |
adbeat_bot
addthis.com
adxpsffetcher-google
ahrefsbot/5.0
aihitbot
blexbot
cb
ccbot
check_http
cloudinary
coccoc
converacrawler
crawler@wishpond.com
crazywebcrawler+0.9.8
crazywebcrawler-spider
curl
domain+re-animator+bot
domainsigmacrawler
dotbot
etaospider
exabot
ezooms
fatbot
gigablastopensource
grapeshotcrawler
gsitecrawler
gumgum bot
gwpimages
halebot
ia_archiver
icc-crawler
internetseer.com
konqueror
kw-lp-suggest
kw-lp-suggest count
linkapedia
linkapediabot
linkdex
linkdexbot
linkdexbot/2.0
linkdexbot/2.1
linkdexbot/2.2
linkdexbot-mobile/2.1
lipperhey spider
maxpointcrawler
meanpathbot
mj12bot
newrelicpinger
nerdybot
nutch
orangebot
panopta
panopta+v1.1
riddler
ruby
scoutjet
scrapy
scrapybot
semrushbot-sa
seobility
showyoubot
smtbot
spbot
uber
uber spider
uptime files - website monitoring service
voltron
wishpond
xenu
xenu link sleuth
nagios
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.buyautoparts.com/sitemap.xml |
| sitemap | https://www.buyautoparts.com/buynow-sitemap.xml |
| sitemap | https://www.buyautoparts.com/buynow-images-sitemap.xml |
Warnings
- 2 invalid lines.
Comments