huajia.163.com
robots.txt
Robots Exclusion Standard data for huajia.163.com
Resource Scan
Scan Details
Site Domain | huajia.163.com |
Base Domain | 163.com |
Scan Status | Ok |
Last Scan | 2024-09-20T22:23:20+00:00 |
Next Scan | 2024-10-04T22:23:20+00:00 |
Last Scan
Scanned | 2024-09-20T22:23:20+00:00 |
URL | https://huajia.163.com/robots.txt |
Domain IPs | 34.111.121.110 |
Response IP | 34.111.121.110 |
Found | Yes |
Hash | 1633cb06091bd08922e9ee04d095d3ad0b24576c0fb25a01f3b99976480d1e62 |
SimHash | dc76dd362a12 |
Groups
googlebot
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
baiduspider-news
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
baiduspider
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
baiduspider-image
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
sosospider
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
bingbot
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
haosouspider
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
yisouspider
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
youdaobot
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
sogou orion spider
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
sogou news spider
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
sogou blog
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
sogou spider2
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
sogou inst spider
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
sogou web spider
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
easouspider
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
msnbot
Rule | Path |
---|---|
Disallow | /main/arts |
Disallow | /profile |
Disallow | /app/v1/config |
Disallow | /app/v1/session |
Disallow | /app/v2/tag/list/workupload |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
*
Rule | Path |
---|---|
Disallow | / |
Warnings
- 6 invalid lines.
- `request-rate` is not a known field.