lightnovel.com
robots.txt
Robots Exclusion Standard data for lightnovel.com
Resource Scan
Scan Details
| Site Domain | lightnovel.com |
| Base Domain | lightnovel.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't establish SSL connection. |
| Last Scan | 2024-09-10T18:12:18+00:00 |
| Next Scan | 2024-12-09T18:12:18+00:00 |
Last Successful Scan
| Scanned | 2024-05-14T06:54:05+00:00 |
| URL | https://lightnovel.com/robots.txt |
| Redirect | https://www.lightnovel.com/robots.txt |
| Redirect Domain | www.lightnovel.com |
| Redirect Base | lightnovel.com |
| Domain IPs | 175.207.13.247 |
| Redirect IPs | 175.207.13.247 |
| Response IP | 175.207.13.247 |
| Found | Yes |
| Hash | b39f70e4a3dc9fbc35ff26ac38deb2fe329e9529b8db9ad7a9303f326b06512c |
| SimHash | 405c3d527381 |
Groups
googlebot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
googlebot-image
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
adsbot-google
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
mediapartners-google
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
naverbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
yeti
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
slurp
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
bingbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
ia_archiver
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
twitterbot/1.0
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
zumbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
mj12bot
| Rule | Path |
|---|---|
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/de |
| Disallow | /bbs/li |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/si |
| Disallow | /bbs/wr |
| Disallow | /ched |
| Disallow | /conv |
| Disallow | /exte |
| Disallow | /eyoo |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /them |
| Disallow | /g4_i |
| Disallow | /yc4_ |
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /.well |
| Disallow | /adm |
| Disallow | /bbs/aj |
| Disallow | /bbs/de |
| Disallow | /bbs/lo |
| Disallow | /bbs/pa |
| Disallow | /bbs/wr |
| Disallow | /exte |
| Disallow | /lib |
| Disallow | /plug |
| Disallow | /g4_i |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Comments