dchublist.biz
robots.txt
Robots Exclusion Standard data for dchublist.biz
Resource Scan
Scan Details
Site Domain | dchublist.biz |
Base Domain | dchublist.biz |
Scan Status | Ok |
Last Scan | 2024-09-18T12:20:40+00:00 |
Next Scan | 2024-10-18T12:20:40+00:00 |
Last Scan
Scanned | 2024-09-18T12:20:40+00:00 |
URL | https://dchublist.biz/robots.txt |
Domain IPs | 37.187.127.179 |
Response IP | 37.187.127.179 |
Found | Yes |
Hash | 8e5b81453fc8415e1dbb63083b6675ee1b3881f04e8ec9cd19789303b2823e7e |
SimHash | 6b81367328e3 |
Groups
*
Rule | Path |
---|---|
Disallow | /.config |
Disallow | /.bz2 |
Disallow | /?do=hublist&get=hublist.xml.bz2 |
Disallow | /?do=hublist&get=hublist.xml |
Disallow | /?do=hublist&get=hublist.config.bz2 |
Disallow | /?do=hublist&get=hublist.config |
Disallow | /?do=hublist&get=hublist-ru.xml.bz2 |
Disallow | /?do=hublist&get=hublist-ru.xml |
Disallow | /?do=hublist&get=hublist-ru.config.bz2 |
Disallow | /?do=hublist&get=hublist-ru.config |
Disallow | /?do=hublist&get=hublist-ro.xml.bz2 |
Disallow | /?do=hublist&get=hublist-ro.xml |
Disallow | /?do=hublist&get=hublist-ro.config.bz2 |
Disallow | /?do=hublist&get=hublist-ro.config |
Disallow | /?do=hublist&get=hublist-pl.xml.bz2 |
Disallow | /?do=hublist&get=hublist-pl.xml |
Disallow | /?do=hublist&get=hublist-pl.config.bz2 |
Disallow | /?do=hublist&get=hublist-pl.config |
Disallow | /?do=hublist&get=hublist-ua.xml.bz2 |
Disallow | /?do=hublist&get=hublist-ua.xml |
Disallow | /?do=hublist&get=hublist-ua.config.bz2 |
Disallow | /?do=hublist&get=hublist-ua.config |
Disallow | /?do=hublist&get=hublist-lt.xml.bz2 |
Disallow | /?do=hublist&get=hublist-lt.xml |
Disallow | /?do=hublist&get=hublist-lt.config.bz2 |
Disallow | /?do=hublist&get=hublist-lt.config |
Disallow | /?do=hublist&get=hublist-hu.xml.bz2 |
Disallow | /?do=hublist&get=hublist-hu.xml |
Disallow | /?do=hublist&get=hublist-hu.config.bz2 |
Disallow | /?do=hublist&get=hublist-hu.config |
Disallow | /?do=hublist&get=hublist-se.xml.bz2 |
Disallow | /?do=hublist&get=hublist-se.xml |
Disallow | /?do=hublist&get=hublist-se.config.bz2 |
Disallow | /?do=hublist&get=hublist-se.config |
Disallow | /?do=hublist&get=hublist-it.xml.bz2 |
Disallow | /?do=hublist&get=hublist-it.xml |
Disallow | /?do=hublist&get=hublist-it.config.bz2 |
Disallow | /?do=hublist&get=hublist-it.config |
Disallow | /?do=hublist&get=hublist-fr.xml.bz2 |
Disallow | /?do=hublist&get=hublist-fr.xml |
Disallow | /?do=hublist&get=hublist-fr.config.bz2 |
Disallow | /?do=hublist&get=hublist-fr.config |
Disallow | /?do=hublist&get=hublist-cz.xml.bz2 |
Disallow | /?do=hublist&get=hublist-cz.xml |
Disallow | /?do=hublist&get=hublist-cz.config.bz2 |
Disallow | /?do=hublist&get=hublist-cz.config |
Disallow | /?do=hublist&get=hublist-gb.xml.bz2 |
Disallow | /?do=hublist&get=hublist-gb.xml |
Disallow | /?do=hublist&get=hublist-gb.config.bz2 |
Disallow | /?do=hublist&get=hublist-gb.config |
Disallow | /?do=hublist&get=hublist-nl.xml.bz2 |
Disallow | /?do=hublist&get=hublist-nl.xml |
Disallow | /?do=hublist&get=hublist-nl.config.bz2 |
Disallow | /?do=hublist&get=hublist-nl.config |
Disallow | /?do=hublist&get=hublist-bg.xml.bz2 |
Disallow | /?do=hublist&get=hublist-bg.xml |
Disallow | /?do=hublist&get=hublist-bg.config.bz2 |
Disallow | /?do=hublist&get=hublist-bg.config |
Disallow | /?do=hublist&get=hublist-lv.xml.bz2 |
Disallow | /?do=hublist&get=hublist-lv.xml |
Disallow | /?do=hublist&get=hublist-lv.config.bz2 |
Disallow | /?do=hublist&get=hublist-lv.config |
Disallow | /hublist.xml.bz2 |
Disallow | /hublist.xml |
Disallow | /hublist.config.bz2 |
Disallow | /hublist.config |
Disallow | /?do=hublist.xml.bz2 |
Disallow | /?do=hublist.xml |
Disallow | /?do=hublist.config.bz2 |
Disallow | /?do=hublist.config |
Disallow | /lists/hublist.xml.bz2 |
Disallow | /lists/hublist.xml |
Disallow | /lists/hublist.config.bz2 |
Disallow | /lists/hublist.config |
Disallow | /*.zip$ |
Disallow | /lists/ |
aboundexbot
adressendeutschland.de
ahrefsbot
amagit.com
b2w/0.1
backdoorbot/1.0
black hole
blinkacrawler
botalot
bot-pge.chlooe.com
bullseye/1.0
ccbot
cityreview
coccoc
compspybot
dcpbot
disco pump 3.1
download ninja
drupact
emailcollector
ezooms
findlinks
flightdeckreportsbot
followsite bot
fyberspider
fatbot
gurujibot
hloader
http://www.searchengineworld.com bot
huaweisymantecspider
influencebot
integromedb
jyxobot
kenjin spider
linkaider
livedoor screenshot
metamojicrawler
mister pix
netants
netseer
nuhk
plukkie
qirina hurdler
r6 bot
radiation retriever 1.1
ronzoobot
ruky-roboter
search.kumkie.com
search17bot
sitesnagger
sosospider
spanner
suggybot
thenomad
tighttwatbot
tocrawl/urldispatcher
twiceler
url control
urlfan-bot
urly warning
vagabondo
webbandit
webbandit/3.50
webcopier
webreaper
websauger
web-sniffer
webstripper
webstripper/2.02
winwebbot
xenu link sleuth/1.3.8
xenu's
xenu's link sleuth 1.1c
yacybot
yioopbot
yodaobot
youdaobot
yowedobot
yrspider
zealbot
zeus
zeus 32297 webster pro v2.9 win32
zeus link scout
jikespider
voilabot
germcrawler
mj12bot
seznambot
zmeu
morfeus
snoopy
perl lwp
megaindex.ru
megaindex.ru/2.0
megaindex.ru
mj12bot/v1.4.3
libwww-perl
java
gsa-crawler (enterprise; t4-knhh62cdkc2w3; gsa_manage@nikon-sys.co.jp)
uptimerobot/2.0
seokicks-robot
Rule | Path |
---|---|
Disallow | / |
Other Records
Field | Value |
---|---|
sitemap | https://dchublist.biz/sitemap.xml |
Warnings
- 11 invalid lines.