Log In Sign Up

wnet.icu
robots.txt

Robots Exclusion Standard data for wnet.icu

Resource Scan

Scan Details

Site Domain wnet.icu
Base Domain wnet.icu
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-25T15:04:08+00:00
Next Scan 2025-12-24T15:04:08+00:00

Last Successful Scan

Scanned2024-08-08T03:47:57+00:00
URL https://wnet.icu/robots.txt
Domain IPs 104.21.67.221, 172.67.182.12, 2606:4700:3031::ac43:b60c, 2606:4700:3037::6815:43dd
Response IP 172.67.182.12
Found Yes
Hash 8385dafd9f85f60cefee656efa17f91814a918a8c26f9187127fd5006e810b98
SimHash 2056d80b4482

Groups

baiduspider

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

slurp

Rule Path
Disallow /

teoma

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

scrubby

Rule Path
Disallow /

robozilla

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

psbot

Rule Path
Disallow /
Disallow /

Other Records

Field Value
crawl-delay 120
Back to top

Comments

  • robots.txt generated at http://tool.chinaz.com/robots/
Back to top