Log In Sign Up

dxniu.com
robots.txt

Robots Exclusion Standard data for dxniu.com

Resource Scan

Scan Details

Site Domain dxniu.com
Base Domain dxniu.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-05-28T11:08:56+00:00
Next Scan 2024-08-26T11:08:56+00:00

Last Successful Scan

Scanned2024-01-30T04:12:48+00:00
URL https://dxniu.com/robots.txt
Domain IPs 104.21.68.48, 172.67.186.169, 2606:4700:3035::ac43:baa9, 2606:4700:3036::6815:4430
Response IP 172.67.186.169
Found Yes
Hash 1e50eb8c5de43c35191a55486dacf57c544ab7196160a1500347d1226465676d
SimHash 5802947ae9bb

Groups

*

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

feeddemon

Rule Path
Disallow /

bot/0.1 (bot for jce)

Rule Path
Disallow /

crawldaddy

Rule Path
Disallow /

java

Rule Path
Disallow /

jullo

Rule Path
Disallow /

feedly

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

apachebench

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

zmeu phpmyadmin

Rule Path
Disallow /

winhttp

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

httpclient

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

yyspider

Rule Path
Disallow /

jaunty

Rule Path
Disallow /

obot

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

flightdeckreports bot

Rule Path
Disallow /

indy library

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.cscuiju.com/sitemap.xml
Back to top