eelkat.com
robots.txt
Robots Exclusion Standard data for eelkat.com
Resource Scan
Scan Details
| Site Domain | eelkat.com |
| Base Domain | eelkat.com |
| Scan Status | Ok |
| Last Scan | 2024-11-09T07:26:07+00:00 |
| Next Scan | 2024-11-16T07:26:07+00:00 |
Last Scan
| Scanned | 2024-11-09T07:26:07+00:00 |
| URL | https://eelkat.com/robots.txt |
| Redirect | https://www.eelkat.com/robots.txt |
| Redirect Domain | www.eelkat.com |
| Redirect Base | eelkat.com |
| Domain IPs | 173.247.219.183 |
| Redirect IPs | 173.247.219.183 |
| Response IP | 173.247.219.183 |
| Found | Yes |
| Hash | 2966ce10312127863643fe7bb66f46e539a51f4d1f45a9e3a6043cc88adc2fc2 |
| SimHash | f1a258fb8ed2 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Allow | /images/ | - |
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /includes/ | - |
| Disallow | /misc/ | - |
| Disallow | /modules/ | - |
| Disallow | /profiles/ | - |
| Disallow | /scripts/ | - |
| Disallow | /themes/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | /admin/ | - |
| Disallow | /comment/reply/ | - |
| Disallow | /filter/tips/ | - |
| Disallow | /node/add/ | - |
| Disallow | /search/ | - |
| Disallow | /user/register/ | - |
| Disallow | /user/password/ | - |
| Disallow | /user/login/ | - |
| Disallow | /user/logout/ | - |
| Disallow | /?q=admin%2F | - |
| Disallow | /?q=comment%2Freply%2F | - |
| Disallow | /?q=filter%2Ftips%2F | - |
| Disallow | /?q=node%2Fadd%2F | - |
| Disallow | /?q=search%2F | - |
| Disallow | /?q=user%2Fpassword%2F | - |
| Disallow | /?q=user%2Fregister%2F | - |
| Disallow | /?q=user%2Flogin%2F | - |
| Disallow | /?q=user%2Flogout%2F | - |
| Disallow | *.xlsx$ | - |
| Disallow | *.xls$ | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/ | - |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
| Disallow | /*.php | - |
| Disallow | - | |
| Disallow | *.xlsx | - |
| Disallow | *.xls | - |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 25 |
googlebot
| Rule | Path | Comment |
|---|---|---|
| Allow | /images/ | - |
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
googlebot-mobile
| Rule | Path | Comment |
|---|---|---|
| Allow | /images/ | - |
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
googlebot-image
| Rule | Path | Comment |
|---|---|---|
| Allow | /images/ | - |
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*.js | - |
mediapartners-google
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
bingbot
| Rule | Path | Comment |
|---|---|---|
| Allow | /images/ | - |
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
pinterestbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
pinterest/0.2 (+https://www.pinterest.com/bot.html)
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/5.0 (compatible; pinterestbot/1.0; +https://www.pinterest.com/bot.html)
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/5.0 (linux; android 6.0.1; nexus 5x build/mmb29p) applewebkit/537.36 (khtml, like gecko) chrome/41.0.2272.96 mobile safari/537.36 (compatible; pinterestbot/1.0; +https://www.pinterest.com/bot.html)
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
msnbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
slurp
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/5.0 (compatible;contxbot/1.0)
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
baiduspider
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
yandex
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
duckduckbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
adbeat_bot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
ahrefsbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
ahrefssiteaudit
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
alexibot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
appengine
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
aqua_products
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
aspiegelbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
asterias
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
b2w/0.1
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
backdoorbot/1.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
backlinkcrawler
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
becomebot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
blekkobot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
blexbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
blowfish/1.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
bookmark search tool
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
botalot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
builtbottough
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
bullseye/1.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
bunnyslippers
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
ccbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
cheesebot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
cherrypicker
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
cherrypickerelite/1.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
cherrypickerse/1.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
chroot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
copernic
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
copyrightcheck
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
cosmos
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
crescent
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
crescent internet toolpak http ole control v.1.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
dittospyder
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
dotbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
dumbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
emailcollector
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
emailsiphon
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
emailwolf
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
enterprise_search
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
enterprise_search/1.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
erocrawler
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
es
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
exabot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
extractorpro
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
fairad client
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
flaming attackbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
foobot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
gaisbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
getright/4.2
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
gigabot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
go-http-client
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
gptbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
grub
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
grub-client
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
harvest/1.5
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
hatena antenna
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
hloader
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
http://www.searchengineworld.com bot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
http://www.webmasterworld.com bot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
httplib
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
humanlinks
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
infonavirobot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
iron33/1.0.2
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
jamesbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
jennybot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
jetbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
jetbot/1.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
jorgee
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
kenjin spider
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
keyword density/0.9
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
larbin
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
lexibot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
libweb/clshttp
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
linkextractorpro
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
linkpadbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
linkscan/8.1a unix
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
linkwalker
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
lnspiderguy
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
looksmart
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
lwp-trivial
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
lwp-trivial/1.34
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
lycos
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mata hari
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mauibot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
megalodon
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
microsoft url control
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
microsoft url control - 5.01.4511
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
microsoft url control - 6.00.8169
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
miixpc
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
miixpc/4.2
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mister pix
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mj12bot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
moget
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
moget/2.1
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/3
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/4
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/4.0 (compatible; bullseye; windows 95)
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/4.0 (compatible; msie 4.0; windows 2000)
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/4.0 (compatible; msie 4.0; windows 95)
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/4.0 (compatible; msie 4.0; windows 98)
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/4.0 (compatible; msie 4.0; windows nt)
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/4.0 (compatible; msie 4.0; windows xp)
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
mozilla/5
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
msiecrawler
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
msnbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
naver
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
naverbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
nerdybot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
netants
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
netmechanic
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
nicerspro
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
nutch
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
offline explorer
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
openbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
openfind
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
openfind data gathere
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
oracle ultra search
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
perman
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
petalbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
propowerbot/2.14
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
prowebwalker
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
psbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
python-urllib
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
queryn metasearch
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
radiation retriever 1.1
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
repomonkey
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
repomonkey bait & tackle/v1.01
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
rma
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
robozilla
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
rogerbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
scooter
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
screaming frog seo spider
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
searchmetricsbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
searchpreview
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
seekbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
semrushbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
semrushbot-sa
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
seokicks-robot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
sistrix
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
sitesnagger
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
sootle
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
spankbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
spanner
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
spbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
stanford
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
stanford comp sci
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
stanford compclub
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
stanford compsciclub
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
stanford spiderboys
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
surveybot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
surveybot_ignoreip
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
suzuran
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
szukacz/1.4
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
teleport
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
teleportpro
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
telesoft
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
teoma
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
the intraformant
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
thenomad
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
tocrawl/urldispatcher
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
true_robot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
true_robot/1.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
turingos
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
typhoeus
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
url control
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
url_spider_pro
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
urly warning
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
vci
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
vci webviewer vci webviewer win32
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
web image collector
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webauto
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webbandit
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webbandit/3.50
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webcopier
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webenhancer
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webmasterworld extractor
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webmasterworldforumbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
websauger
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
website quester
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webster pro
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webstripper
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webvac
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webzip
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
webzip/4.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
wget
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
wget/1.5.3
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
wget/1.6
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
www-collector-e
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
xenu's
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
xenu's link sleuth 1.1c
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
xovi
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
yahoo-blogs/v3.9
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
yahoo-mmcrawler
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
yeti
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
zeus
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
zeus 32297 webster pro v2.9 win32
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
zeus link scout
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /bin/ | - |
| Disallow | /dyn/ | - |
| Disallow | /objects/ | - |
| Disallow | /help/ | - |
| Disallow | /no-such-url.html | - |
| Disallow | /fb/channel.html | - |
| Disallow | /*.pdf$ | - |
| Disallow | /.drafts/ | - |
| Disallow | /TEMPLATE | - |
| Disallow | /*TEMPLATE | - |
| Disallow | /Template | - |
| Disallow | /*Template | - |
| Disallow | /template | - |
| Disallow | /*template | - |
| Disallow | /ARCHIVE | - |
| Disallow | /*ARCHIVE | - |
| Disallow | /Archive | - |
| Disallow | /*Archive | - |
| Disallow | /archive | - |
| Disallow | /*archive | - |
| Disallow | /DRAFT | - |
| Disallow | /*DRAFT | - |
| Disallow | /Draft | - |
| Disallow | /*Draft | - |
| Disallow | /draft | - |
| Disallow | /*draft | - |
| Disallow | /erotica | - |
| Disallow | /Erotica | - |
| Disallow | /EROTICA | - |
| Disallow | /*erotica | - |
| Disallow | /*Erotica | - |
| Disallow | /*EROTICA | - |
| Disallow | /nsfw | - |
| Disallow | /NSFW | - |
| Disallow | /*nsfw | - |
| Disallow | /*NSFW | - |
| Disallow | /porn | - |
| Disallow | /Porn | - |
| Disallow | /PORN | - |
| Disallow | /*porn | - |
| Disallow | /*Porn | - |
| Disallow | /*PORN | - |
| Disallow | /sex | - |
| Disallow | /Sex | - |
| Disallow | /SEX | - |
| Disallow | /*sex | - |
| Disallow | /*Sex | - |
| Disallow | /*SEX | - |
| Disallow | /yaoi | - |
| Disallow | /Yaoi | - |
| Disallow | /YAOI | - |
| Disallow | /*yaoi | - |
| Disallow | /*Yaoi | - |
| Disallow | /*YAOI | - |
| Disallow | /*? | - |
| Disallow | *? | - |
| Disallow | /*.*. | - |
| Disallow | *?p=* | block access to pages for which permalinks fails |
| Disallow | *%26p%3D* | block access to pages for which permalinks fails |
| Disallow | *%26preview%3D* | block access to preview pages |
| Disallow | /*/* | - |
| Disallow | /*.js | - |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.eelkat.com/sitemap.xml |
| sitemap | https://www.eelkat.com/support-files/html-sitemap.txt |
| sitemap | https://www.eelkat.com/author.xml |
| sitemap | https://www.eelkat.com/support-files/image-sitemap-works.xml |
| sitemap | https://www.eelkat.com/index-of-pages.html |
| sitemap | https://www.eelkat.com/eelkat.xml |
Warnings
- 18 invalid lines.
- `request-rate` is not a known field.
Comments