www.a.shifen.com
robots.txt

Robots Exclusion Standard data for www.a.shifen.com

Resource Scan

Scan Details

Site Domain www.a.shifen.com
Base Domain shifen.com
Scan Status Ok
Last Scan2024-06-02T02:54:35+00:00
Next Scan 2024-07-02T02:54:35+00:00

Last Scan

Scanned2024-06-02T02:54:35+00:00
URL http://www.a.shifen.com/robots.txt
Domain IPs 119.63.197.139, 119.63.197.151
Response IP 45.113.192.102
Found Yes
Hash c28ebeffb7baf9bd6407109e80dc95d87c75c0eaecc20b23884cb33111cc5df0
SimHash d865dd3c2901

Groups

baiduspider

Rule Path
Disallow /baidu
Disallow /s?
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

googlebot

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

msnbot

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

baiduspider-image

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

youdaobot

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

sogou web spider

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

sogou inst spider

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

sogou spider2

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

sogou blog

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

sogou news spider

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

sogou orion spider

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

chinasospider

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

sosospider

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

yisouspider

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

easouspider

Rule Path
Disallow /baidu
Disallow /s?
Disallow /shifen/
Disallow /homepage/
Disallow /cpro
Disallow /ulink?
Disallow /link?
Disallow /home/news/data/
Disallow /bh

*

Rule Path
Disallow /