md.gvdb.org
robots.txt
Robots Exclusion Standard data for md.gvdb.org
Resource Scan
Scan Details
| Site Domain | md.gvdb.org |
| Base Domain | gvdb.org |
| Scan Status | Ok |
| Last Scan | 2025-09-01T04:30:39+00:00 |
| Next Scan | 2025-10-01T04:30:39+00:00 |
Last Scan
| Scanned | 2025-09-01T04:30:39+00:00 |
| URL | https://md.gvdb.org/robots.txt |
| Domain IPs | 104.21.41.211, 172.67.151.123, 2606:4700:3033::6815:29d3, 2606:4700:3033::ac43:977b |
| Response IP | 104.21.41.211 |
| Found | Yes |
| Hash | fa610169ba7905f3c1ebeea6decb2c134e1793c5ef4afc31c890f2775a9060ed |
| SimHash | 51137100ef2d |
Groups
*
| Rule | Path |
|---|---|
| Allow | /wp-content/uploads/*.jpg$ |
| Allow | /wp-content/uploads/*.webp$ |
| Disallow | /*/?*=* |
| Disallow | /cdn-cgi/ |
| Disallow | /author/ |
| Disallow | /wp-content/uploads/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/ |
| Disallow | /*/trackback |
| Disallow | */trackback |
| Disallow | */comments |
| Disallow | /wp-login.php |
| Disallow | /*/comment-page-* |
| Disallow | /*?replytocom=* |
| Disallow | /category/*/page/ |
| Disallow | /category/ |
| Disallow | /tag/*/page/ |
| Disallow | /attachment/ |
| Disallow | /page/ |
| Disallow | /wp-json/ |
| Disallow | /*/feed/ |
| Disallow | /*/embed/ |
| Disallow | /date/ |
| Disallow | /?cat=* |
| Disallow | /?s=* |
| Disallow | /?mode=* |
| Disallow | */?mode=* |
| Disallow | *.png$ |
| Disallow | *.gif$ |
| Disallow | *.txt$ |
| Disallow | /comments/ |
| Disallow | /search/ |
| Disallow | /ranking/ |
| Disallow | /result/ |
| Disallow | /guestbook/ |
| Disallow | /contact/ |
| Disallow | /landing/ |
| Disallow | /test-search/ |
| Disallow | /search/ |
| Disallow | /model-list/ |
| Disallow | /list/ |
| Disallow | /tag/ |
| Disallow | /studio/ |
| Disallow | /random/ |
| Disallow | /terms-of-service/ |
| Disallow | /disclaimer/ |
| Disallow | /comment-policy/ |
| Disallow | /privacy-policy/ |
| Disallow | /2257-statement/ |
| Disallow | /test/ |
| Disallow | /home/ |
| Disallow | /ad/ |
| Disallow | /search-for/ |
| Disallow | /favorites/ |
| Disallow | /most-comment/ |
| Disallow | /buy/ |
| Disallow | /rvl/ |
| Disallow | */?wpfpaction=* |
| Disallow | */?_page=* |
| Disallow | /*?wpfpaction=* |
| Disallow | /special-thanks/ |
| Disallow | /87667/ |
| Disallow | /100613/ |
| Disallow | */?fbclid=* |
| Disallow | */?__cf_chl_jschl_tk__=* |
| Disallow | */?highlight=* |
| Disallow | /image-sitemap-index-1.xml |
| Disallow | /image-sitemap-* |
google-extended
gptbot
ccbot
beautifulsoup
scrapy
ia_archiver
archive.org_bot
ia_archiver-web.archive.org
pinterestbot
petalbot
haosouspider
mj12bot
mediapartners-google*
israbot
orthogaffe
ubicrawler
doc
zao
sitecheck.internetseer.com
zealbot
msiecrawler
sitesnagger
webstripper
webcopier
fetch
offline explorer
teleport
teleportpro
webzip
linko
httrack
microsoft.url.control
xenu
larbin
libwww
zyborg
download ninja
fast
wget
grub-client
k2spider
npbot
webreaper
proximic
wget
teleport
webcopy
sitesucker
goo
y!j*
y!j-srd/1.0
y!j-mbs/1.0
y!j-brw/1.0 crawler
y!j-brj/yats crawler
y!j-brl/yatss crawler
y!j-brm/yatsd crawler
y!j-brn/yatsa crawler
y!j-bry/yatsh crawler
y!j-brz/yatsha crawler
y!j-bri/0.0.1 crawler
cyotekwebcopy
cyotekwebcopy/1.0 cyotekwebcrawler/1.0
cyotekwebcopy/1.0 cyotekhttp/2.0
cyotekwebcopy/1.6 cyotekhttp/2.0
cyotekwebcopy/1.8 cyotekhttp/2.0
lightspeedsystemscrawler
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /wp-content/uploads/wp-import-export-lite/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://md.gvdb.org/wp-sitemap.xml |
Comments