santamco.com
robots.txt
Robots Exclusion Standard data for santamco.com
Resource Scan
Scan Details
| Site Domain | santamco.com |
| Base Domain | santamco.com |
| Scan Status | Ok |
| Last Scan | 2024-09-15T12:34:06+00:00 |
| Next Scan | 2024-10-15T12:34:06+00:00 |
Last Scan
| Scanned | 2024-09-15T12:34:06+00:00 |
| URL | https://santamco.com/robots.txt |
| Domain IPs | 65.21.88.135 |
| Response IP | 65.21.88.135 |
| Found | Yes |
| Hash | c8ddbd389b6709b4a0dd1660bcf288c9ae5f27afe4da4ee7e22bc1f80b50a0bb |
| SimHash | 50465de24c4b |
Groups
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 70 |
extlinksbot
crawler4j
dataprovider
daum
mauibot
panscient.com
vscooter
psbot
ia_archiver
mj12bot
twiceler
yandex
taptubot
googlebot-image
twengabot
sitebot
baiduspider
ahrefsbot
ezooms
sistrix
aihitbot
infopath
infopath.2
swebot
ec2linkfinder
turnitinbot
the knowledge ai
| Rule | Path |
|---|---|
| Disallow | / |
searchmetericsbot
wbsearchbot
exabot
sosospider
ip-web-crawler.com
netestate ne crawler
aboundexbot
aboundex
meanpathbot
mail.ru
spbot
archive.org_bot
linkpadbot
easouspider
seznambot
wotbox
blexbot
xovibot
semrushbot
a6-indexer
riddler
loadtimebot
obot
mojeekbot
memorybot
ltx71
| Rule | Path |
|---|---|
| Disallow | / |
advbot
smtbot
yisouspider
lssrocketcrawler
gsa-crawler
nutch
tbot-nutch
thunderstone
yacybot
ranksonicbot
betabot
parsijoo-bot
nextgensearchbot
gocrawl
plukkie
applebot
lipperhey
safednsbot
rankactivelinkbot
sogou blog
sogou inst spider
sogou news spider
sogou orion spider
sogou spider2
sogou web spider
uptimebot
seeker
cliqzbot
domaincrawler
yoozbot
coccocbot-web
qwantify
siteexplorer
findxbot
garlikcrawler
zoominfobot
bubing
barkrowler
rogerbot
dotbot
jamesbot
contacts-crawler
ccbot
idbot
dnyzbot
piplbot
alphabot
alphaseobot
alphaseobot-sa
seokicks-robot
| Rule | Path |
|---|---|
| Disallow | / |
slurp
| Rule | Path |
|---|---|
| Disallow | /*.jpg |
| Disallow | /*.JPG |
| Disallow | /*.png |
| Disallow | |
| Disallow | |
| Disallow | /*.mp3 |
| Disallow | /*.MOV |
| Disallow | /*.mov |
| Disallow | /*.AVI |
| Disallow | /*.avi |
| Disallow | /*.svg |
| Disallow | /*.csv |
| Disallow | /*.data |
| Disallow | /disclaimer.html |
| Disallow | /security.html |
| Disallow | /poweredby.html |
| Disallow | /about_smythies.html |
| Disallow | /unused_link.html |
| Disallow | /old_pages.html |
| Disallow | /index_0* |
| Disallow | /*index_0*$ |
| Disallow | /digital_camera/ |
| Disallow | /lab/ |
| Disallow | /xmas_* |
| Disallow | /~doug/archives/ |
| Disallow | /~doug/linux/ubuntu-docs/help.ubuntu.com/ |
| Disallow | /bot_trap.html |
Warnings
- 1 invalid line.