Log In Sign Up

en.novel5s.com
robots.txt

Robots Exclusion Standard data for en.novel5s.com

Resource Scan

Scan Details

Site Domain en.novel5s.com
Base Domain novel5s.com
Scan Status Ok
Last Scan2024-05-26T13:18:02+00:00
Next Scan 2024-06-25T13:18:02+00:00

Last Scan

Scanned2024-05-26T13:18:02+00:00
URL https://en.novel5s.com/robots.txt
Domain IPs 104.21.81.130, 172.67.160.232, 2606:4700:3031::ac43:a0e8, 2606:4700:3037::6815:5182
Response IP 104.21.81.130
Found Yes
Hash 64f5ead745f6ec631668d4c23e9bcd3a3335791f12a3052d9e52d4df3b2ae2c2
SimHash 301501d04f8b

Groups

*

Rule Path
Allow /
Disallow /auth/*

proximic

Rule Path
Disallow /

obot

Rule Path
Disallow /

archive.org

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

python-request

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

brute-force login bot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

spbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://novel5s.com/sitemap.xml
Back to top