Log In Sign Up

hainanfz.com
robots.txt

Robots Exclusion Standard data for hainanfz.com

Resource Scan

Scan Details

Site Domain hainanfz.com
Base Domain hainanfz.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2026-02-04T07:13:47+00:00
Next Scan 2026-04-05T07:13:47+00:00

Last Successful Scan

Scanned2025-12-07T02:12:24+00:00
URL https://hainanfz.com/robots.txt
Domain IPs 106.53.189.93
Response IP 106.53.189.93
Found Yes
Hash 9db0197a61710f36f21674f3e97a4997e1b27e1d8726c18ed7437b93ded4e397
SimHash 45b59b5743d9

Groups

*

Rule Path
Disallow /*?
Disallow /api/
Disallow /js/
Disallow /css/
Disallow /Verify/
Disallow /iy/
Disallow /tags/
Disallow /Files/
Disallow /files/
Allow /.%2B
Allow /$

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://hainanfz.com/sitemap/directory.xml
sitemap https://hainanfz.com/sitemap/house/index.xml
sitemap https://hainanfz.com/sitemap/house/huxing.xml
sitemap https://hainanfz.com/sitemap/house/xiangce.xml
sitemap https://hainanfz.com/sitemap/house/dongtai.xml
sitemap https://hainanfz.com/sitemap/house/ditu.xml
sitemap https://hainanfz.com/sitemap/house/video.xml
sitemap https://hainanfz.com/sitemap/house/discusslist.xml
sitemap https://hainanfz.com/sitemap/news/0.xml
sitemap https://hainanfz.com/sitemap/news/81.xml
sitemap https://hainanfz.com/sitemap/key/0.xml
sitemap https://hainanfz.com/sitemap/key/1.xml
sitemap https://hainanfz.com/sitemap/key/2.xml
sitemap https://hainanfz.com/sitemap/key/3.xml
sitemap https://hainanfz.com/sitemap/key/4.xml
sitemap https://hainanfz.com/sitemap/key/5.xml
sitemap https://hainanfz.com/sitemap/key/6.xml
sitemap https://hainanfz.com/sitemap/key/7.xml
sitemap https://hainanfz.com/sitemap/key/8.xml
sitemap https://hainanfz.com/sitemap/key/9.xml
Back to top

Comments

  • 禁止抓取所有带查询参数的URL（通常是动态页面、搜索、会话等）
  • 禁止访问敏感或无用目录
  • 为新式前端路由或纯目录路径的å†
  • 设置抓取延迟，建议爬虫每5秒发送一个请求
Back to top

Warnings

  • 1 invalid line.
Back to top