webnovel.0fees.us
robots.txt

Robots Exclusion Standard data for webnovel.0fees.us

Resource Scan

Scan Details

Site Domain webnovel.0fees.us
Base Domain 0fees.us
Scan Status Ok
Last Scan2024-10-10T11:08:53+00:00
Next Scan 2024-11-09T11:08:53+00:00

Last Scan

Scanned2024-10-10T11:08:53+00:00
URL http://webnovel.0fees.us/robots.txt
Domain IPs 185.27.134.125
Response IP 185.27.134.125
Found Yes
Hash 89c02d7f16f056ed69c3afe490fb5a949e76ae06eb21584013c90da75e5a7a2d
SimHash a531ffb09493

Groups

*

Rule Path
Disallow /wp-admin
Disallow /wp-includes
Disallow /httpd/
Disallow /cgi-bin/
Disallow /stats/
Disallow /wp-
Disallow /feed/
Allow /tag

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

yandexbot

Rule Path
Allow /

cuil

Rule Path
Allow /

slurp

Rule Path
Allow /

yahoo

Rule Path
Allow /

yahoo! slurp

Rule Path
Allow /

ia_archiver-web.archive.org

Rule Path
Disallow /

aipbot

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

fast

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

voyager

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

backrub/*.*

Rule Path
Disallow /

grub.org

Rule Path
Disallow /

botrighthere

Rule Path
Disallow /

larbin

Rule Path
Disallow /

psbot

Rule Path
Disallow /

walhello appie

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

crescent

Rule Path
Disallow /

Other Records

Field Value
sitemap http://webnovel.0fees.us/post-sitemap.xml
sitemap http://webnovel.0fees.us/page-sitemap.xml
sitemap http://webnovel.0fees.us/category-sitemap.xml
sitemap http://webnovel.0fees.us/post-tag-sitemap.xml