artabus.com
robots.txt
Robots Exclusion Standard data for artabus.com
Resource Scan
Scan Details
| Site Domain | artabus.com |
| Base Domain | artabus.com |
| Scan Status | Ok |
| Last Scan | 2024-09-28T03:06:57+00:00 |
| Next Scan | 2024-10-05T03:06:57+00:00 |
Last Scan
| Scanned | 2024-09-28T03:06:57+00:00 |
| URL | https://artabus.com/robots.txt |
| Redirect | https://www.artabus.com/robots.txt |
| Redirect Domain | www.artabus.com |
| Redirect Base | artabus.com |
| Domain IPs | 104.21.64.195, 172.67.154.218, 2606:4700:3034::ac43:9ada, 2606:4700:3036::6815:40c3 |
| Redirect IPs | 104.21.64.195, 172.67.154.218, 2606:4700:3034::ac43:9ada, 2606:4700:3036::6815:40c3 |
| Response IP | 104.21.64.195 |
| Found | Yes |
| Hash | e39a90ac3badff3d97505103401d2069d68ffa770b710c55a300d4bea5cc738b |
| SimHash | e2da7f6058c5 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /sendmail.php |
| Disallow | /pic/small/ |
| Disallow | /english/sendmail.php |
| Disallow | /french/sendmail.php |
| Disallow | /english/blog.php |
| Disallow | /french/blog.php |
| Disallow | /links.php |
| Disallow | /english/links.php |
| Disallow | /french/links.php |
| Disallow | /render.php |
| Disallow | /pricelist.php |
| Disallow | /chat.php |
| Disallow | /english/chat.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 4 |
adsbot/3.1
aipbot
bot/1.0
http://www.almaden.ibm.com/cs/crawler
ahrefsbot
anthill
antibot
amfibibot
awariosmartbot
awcheck
barkrowler
biglotron
blexbot
bruinbot
bubing
catchbot
ccbot
ccubee
ccubee/3.5
checkmarknetwork/1.0 (+https://www.checkmarknetwork.com/spider.html)
combine
converacrawler
converamultimediacrawler
coolbot
digitalshadowsbot
dimensionet
discobot
dotbot
drecombot
dtaagent
e-societyrobot
envolk
everbeecrawler
fdse
g2crawler
geniebot
gsa-crawler
hoowwwer
ioncrawl
ip-web-crawler.com
ipselonbot
irlbot
jyxobot
kavamringcrawler
larbin
linksmanager
linkwalker
lmspider
ltx71 - (http://ltx71.com/)
mauibot
mediapartners-google
mj12bot
msiecrawler
mtrobot
myfamilybot
netresearchserver
nextgensearchbot
noxtrumbot
npbot
nutch
nutchcvs
obot
omniexplorer_bot
openintelligencedata
panscient.com
phpdig
pompos
proximic
psbot
radian6
r6_feedfetcher
r6_commentreader
riddler
rufusbot
safednsbot
schibstedsokbot
sbider
scspider
searchmetricsbot
semanticdiscovery
semrushbot
shim-crawler
sistrix
sitebot
shopwiki
silk
sitecheck.internetseer.com
sproose
steeler
surdotlybot
tarantula
the knowledge ai
theophrastus
trendictionbot
tridentspider
turnitinbot
twiceler
ultraseek
vagabondo
verticrawlbot
voyager
voyager/1.0
wget
webindexer
xirq
yak
yak/1.0
zebot_www.ze.bz
zebot
zeus
zoombot
zoominfobot
| Rule | Path |
|---|---|
| Disallow | / |
Comments