upcindex.com
robots.txt
Robots Exclusion Standard data for upcindex.com
Resource Scan
Scan Details
| Site Domain | upcindex.com |
| Base Domain | upcindex.com |
| Scan Status | Ok |
| Last Scan | 2024-10-26T18:16:44+00:00 |
| Next Scan | 2024-11-25T18:16:44+00:00 |
Last Scan
| Scanned | 2024-10-26T18:16:44+00:00 |
| URL | https://upcindex.com/robots.txt |
| Redirect | https://www.upcindex.com/robots.txt |
| Redirect Domain | www.upcindex.com |
| Redirect Base | upcindex.com |
| Domain IPs | 104.26.6.92, 104.26.7.92, 172.67.70.173 |
| Redirect IPs | 104.26.6.92, 104.26.7.92, 172.67.70.173 |
| Response IP | 104.26.6.92 |
| Found | Yes |
| Hash | 0cfef4f8cfa7a2da3e9054791d5f85b6234ea48e992124a3625d80555376639b |
| SimHash | 11c349de5de8 |
Groups
bingpreview
coccocbot-web
coccocbot-image
pandalytics
mixnodecache
cliqzbot
zoominfobot
xml sitemaps generator
df bot
velenpublicwebcrawler
voluumdsp-content-bot
mauibot
the knowledge ai
hypestat
lightspeedsystemscrawler
alphaseobot
alphaseobot-sa
seokicks
bidswitchbot
femtosearchbot
gowikibot
semanticscholarbot
abontool
barkrowler
beambot
demandbase-bot
tweetmemebot
elefent
ntentbot
ntentbot-fetch
ntentbot-news
eracrawler
converacrawler
djecobot
garlikcrawler
getintent crawler
uptimebot
prlog
blackboard safeassign
admantx-euastn
weborama-fetcher
rankingbot2
netestate ne crawler
laserlikebot
indeedbot
diffbot
storygizebot
yesupbot
teeraidbot
linexbot
obot
metacommentbot
seoscanners.net
plukkie
scoutjet
dataprovider
spendabot
webcorplsebot
searchie
uipbot
econtext
safednsbot
twmbot
mfibot
mindupbot
moget
ichiro
naverbot
nutch
nextgensearchbot
deepcrawl
kraken
lipperhey
lipperhey spider
lipperhey-kaus-australis
cloudservermarketspider
heritrix
yeti
wesee
wesee_bot
wbsearchbot
bdcbot
meanpathbot
surdotlybot
baiduspider
baiduspider-video
baiduspider-image
sogou spider
sogou web spider
youdaobot
ahrefsbot
ltx71
addthis.com robot
addthis.com robot tech.support@clearspring.com
mj12bot
genieo
blexbot
easouspider
schenkerianbot
grapeshotcrawler
grapeshot
grapefx
voltron
lexxebot
dotbot
terrawizbot
mail.ru_bot
xovibot
scrapy
siteexplorer
nerdybot
adbeat_bot
coccoc
dalvik
seznambot
bubing
webindex
proximic
sistrix crawler
contextad bot
crazywebcrawler
lssrocketcrawler
crawler4j
spbot
smtbot
appengine-google
crazywebcrawler-spider
exabot
mixrankbot
ia_archiver
yisouspider
linkdexbot
gigablastopensource
ccbot
sistrix
netseer
proximic
linkapediabot
yoozbot
findxbot
domainappender
archive.org_bot
skimbot
maxpointcrawler
wotbox
ia_archiver
linkwalker
livelapbot
openhosebot
acapbot
riddler
musobot
semrushbot
semrushbot-sa
tineye-bot
fatbot
rogerbot
deusu
umbot
megaindex
abonti
advbot
mediavbot
getintentcrawler
turnitinbot
leikibot
yandexdirect
yandexdirectdyn
yandexmedia
yandeximages
yadirectfetcher
yandexblogs
yandexnews
yandexpagechecker
yandexmetrika
yandexcalendar
yandex
yandexbot
yandexwebmaster
yandexmobilebot
istellabot
googleother
google-extended
| Rule | Path |
|---|---|
| Disallow | / |
| Disallow | /* |
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /e/ |
| Disallow | /r/ |
| Disallow | /t/ |
| Disallow | /terms |
| Disallow | /doubleclick/ |
| Disallow | /eyeblaster/ |
| Disallow | /translate_c/ |
Warnings
- 1 invalid line.