aboutautomobile.com
robots.txt

Robots Exclusion Standard data for aboutautomobile.com

Resource Scan

Scan Details

Site Domain aboutautomobile.com
Base Domain aboutautomobile.com
Scan Status Ok
Last Scan2024-09-23T11:07:20+00:00
Next Scan 2024-09-30T11:07:20+00:00

Last Scan

Scanned2024-09-23T11:07:20+00:00
URL https://aboutautomobile.com/robots.txt
Domain IPs 73.140.193.35
Response IP 73.140.193.35
Found Yes
Hash a0120a605ad6fd6de9ea56ff70d0f39af5d4e234ea8283a3e10bd7d91a8d7de2
SimHash 0c4799025390

Groups

*

Rule Path
Disallow /

googlebot
mediapartners-google
adsbot-google
bingbot
facebot
slurp
teoma
applebot
duckduckbot

Rule Path
Disallow

dotbot
rogerbot
grapeshotcrawler
ahrefsbot
mj12bot
alphaseobot
alphaseobot-sa
domaincrawler
megaindex
cliqzbot
sogou
blexbot
linkpadbot
semrushbot
scrapy
searchatlas
awariosmartbot
adsbot
neevabot
infotigerbot
admantx
ias_crawler
gptbot
imagesiftbot
friendlycrawler
bytespider

Rule Path
Disallow /

Comments

  • Any bot that does NOT respect this robots.txt will be banned automatically.