dl.taou.com
robots.txt

Robots Exclusion Standard data for dl.taou.com

Resource Scan

Scan Details

Site Domain dl.taou.com
Base Domain taou.com
Scan Status Ok
Last Scan2024-09-16T22:53:40+00:00
Next Scan 2024-10-16T22:53:40+00:00

Last Scan

Scanned2024-09-16T22:53:40+00:00
URL https://dl.taou.com/robots.txt
Domain IPs 180.87.173.158, 223.119.20.18, 23.251.120.81, 2602:ffe4:401:1b::11, 2602:ffe4:401:1b::12, 2602:ffe4:401:1b::13, 2602:ffe4:401:1b::14, 2602:ffe4:401:1b::15
Response IP 223.121.13.17
Found Yes
Hash 56a5012b64053f212a5f8c6e454b1eae716bc8fb81a274abfc2a873b435c3a18
SimHash 6457f90151c3

Groups

baiduspider

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

googlebot-mobile

Rule Path
Disallow /

yahoo-blogs/v3.9

Rule Path
Disallow /

psbot

Rule Path
Disallow /

*

Rule Path
Disallow /

Comments

  • robots.txt generated at http://portal.qiniu.com