gendou.com
robots.txt
Robots Exclusion Standard data for gendou.com
Resource Scan
Scan Details
Site Domain | gendou.com |
Base Domain | gendou.com |
Scan Status | Ok |
Last Scan | 2024-09-30T20:42:58+00:00 |
Next Scan | 2024-10-30T20:42:58+00:00 |
Last Scan
Scanned | 2024-09-30T20:42:58+00:00 |
URL | http://gendou.com/robots.txt |
Domain IPs | 208.97.136.254 |
Response IP | 208.97.136.254 |
Found | Yes |
Hash | 2c0da1d295f13dd3341cac5d8fb5efbe9abf18b3c2cf50d3ff46df4b0feaee24 |
SimHash | f55ed48343b5 |
Groups
apachebench
da
dap
digext
dotbot
download master
e-societyrobot
exabot
fdm
flashget
getleft
getright
gigabot
heritrix
httrack
ia_archiver
ida
leechget
libwww-perl
msnbot
msrbot
omniexplorer_bot
puxarapido
riddler
skreemrbot
stardownloader
webcopier
yandex
yeti
yodaobot
Rule | Path |
---|---|
Disallow | / |
*
Rule | Path |
---|---|
Disallow | /bot-trap.php |
Disallow | /images/ |
Disallow | /amusic/dl.php |
Disallow | /amusic/rate.php |
Disallow | /forum/search.php |
Disallow | /downloads/ |
Disallow | /uploads/ |
Disallow | /amusic/my-request.php |
Disallow | /*sort%3D |
Disallow | /*cols%3D |
Disallow | /*ent%3D |
Disallow | /*back%3D |
Disallow | /*rev%3D |
Disallow | /*do%3D |
Disallow | /forum/user.php |
Disallow | /forum/sticky.php |
Disallow | /forum/lock.php |
Disallow | /q/lyrics.php |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |