yespornbt.net
robots.txt
Robots Exclusion Standard data for yespornbt.net
Resource Scan
Scan Details
| Site Domain | yespornbt.net |
| Base Domain | yespornbt.net |
| Scan Status | Ok |
| Last Scan | 2024-11-01T12:19:31+00:00 |
| Next Scan | 2024-12-01T12:19:31+00:00 |
Last Scan
| Scanned | 2024-11-01T12:19:31+00:00 |
| URL | https://yespornbt.net/robots.txt |
| Domain IPs | 104.21.8.66, 172.67.138.161, 2606:4700:3036::ac43:8aa1, 2606:4700:3037::6815:842 |
| Response IP | 104.21.8.66 |
| Found | Yes |
| Hash | 62320d71b0c0064014b1754d08c0206c85331982b416a18df23b9eabe97555eb |
| SimHash | 591761c0a710 |
Groups
seekport
datanyze
experibot
indeedbot
extlinksbot
crawler4j
dataprovider
daum
mauibot
mauibot (crawler.feedback+wc@gmail.com)
panscient.com
vscooter
psbot
ia_archiver
mj12bot
twiceler
velenpublicwebcrawler
serpstatbot
megaindex.ru/2.0
megaindex.ru
megaindex.ru
bytespider
taptubot
googlebot-image
twengabot
sitebot
ahrefsbot
ezooms
sistrix
aihitbot
infopath
infopath.2
swebot
ec2linkfinder
turnitinbot
the knowledge ai
mappy
grapeshotcrawler
steeler
jooblebot
adsbot
petalbot
seekport
amazonbot
neevabot
awariobot
| Rule | Path |
|---|---|
| Disallow | / |
searchmetericsbot
wbsearchbot
exabot
sosospider
ip-web-crawler.com
netestate ne crawler
aboundexbot
aboundex
meanpathbot
mail.ru
spbot
archive.org_bot
linkpadbot
easouspider
seznambot
wotbox
blexbot
xovibot
semrushbot
a6-indexer
riddler
loadtimebot
obot
mojeekbot
memorybot
ltx71
| Rule | Path |
|---|---|
| Disallow | / |
advbot
smtbot
yisouspider
lssrocketcrawler
gsa-crawler
nutch
tbot-nutch
thunderstone
yacybot
ranksonicbot
betabot
parsijoo-bot
nextgensearchbot
gocrawl
plukkie
applebot
lipperhey
safednsbot
rankactivelinkbot
uptimebot
seeker
cliqzbot
domaincrawler
yoozbot
coccocbot-web
qwantify
siteexplorer
findxbot
garlikcrawler
zoominfobot
bubing
barkrowler
rogerbot
dotbot
jamesbot
contacts-crawler
ccbot
idbot
dnyzbot
piplbot
alphabot
alphaseobot
alphaseobot-sa
seokicks-robot
bingbot
| Rule | Path |
|---|---|
| Disallow | / |
| Allow | /zb_users/upload/ |
| Disallow | /zb_system/ |
| Disallow | /zb_users/ |
googlebot
| Rule | Path |
|---|---|
| Allow | /zb_system/function/c_html_js_add.asp |
| Allow | /zb_system/function/c_html_js.asp |
| Allow | /zb_system/function/script/common.js |
| Allow | /zb_users/THEME/Newdur/style/default.css |
googlebot
| Rule | Path |
|---|---|
| Allow | /zb_system/function/c_html_js_add.asp |
| Allow | /zb_system/function/c_html_js.asp |
| Allow | /zb_system/function/script/common.js |
| Allow | /zb_users/THEME/Newdur/style/default.css |
| Disallow | /*.jpg$ |
| Disallow | /*.JPG$ |
| Disallow | /*.png$ |
| Disallow | /*.PDF$ |
| Disallow | /*.pdf$ |
| Disallow | /*.mp3$ |
| Disallow | /*.MOV$ |
| Disallow | /*.mov$ |
| Disallow | /*.AVI$ |
| Disallow | /*.avi$ |
| Disallow | /*.csv$ |
| Disallow | /*.svg$ |
| Disallow | /*.data$ |
| Allow | /zb_users/upload/ |
| Disallow | /zb_system/ |
| Disallow | /zb_users/ |
*
| Rule | Path |
|---|---|
| Allow | /zb_system/function/c_html_js_add.asp |
| Allow | /zb_system/function/c_html_js.asp |
| Allow | /zb_system/function/script/common.js |
| Allow | /zb_users/THEME/Newdur/style/default.css |
| Disallow | /*.jpg$ |
| Disallow | /*.JPG$ |
| Disallow | /*.png$ |
| Disallow | /*.PDF$ |
| Disallow | /*.pdf$ |
| Disallow | /*.mp3$ |
| Disallow | /*.MOV$ |
| Disallow | /*.mov$ |
| Disallow | /*.AVI$ |
| Disallow | /*.avi$ |
| Disallow | /*.csv$ |
| Disallow | /*.svg$ |
| Disallow | /*.data$ |
| Allow | /zb_users/upload/ |
| Disallow | /zb_system/ |
| Disallow | /zb_users/ |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.yespornbt.com/sitemap.xml |
Warnings
- 1 invalid line.
Comments