Log In Sign Up

sm160.com
robots.txt

Robots Exclusion Standard data for sm160.com

Resource Scan

Scan Details

Site Domain sm160.com
Base Domain sm160.com
Scan Status Ok
Last Scan2024-11-12T12:45:41+00:00
Next Scan 2024-11-19T12:45:41+00:00

Last Scan

Scanned2024-11-12T12:45:41+00:00
URL https://sm160.com/robots.txt
Redirect https://www.sm160.com/robots.txt
Redirect Domain www.sm160.com
Redirect Base sm160.com
Domain IPs 103.39.220.208
Redirect IPs 103.39.220.208, 121.201.67.107
Response IP 121.201.67.107
Found Yes
Hash 4f5c2f694e391886e0ece39b9d491ea46c611203aee0a1012c43c27878bcff39
SimHash 1f36eb548fd3

Groups

*

Rule Path
Disallow /*?*
Disallow /cp*/p*
Disallow /cb*/p*
Disallow /cc*/p*
Disallow /sp-
Disallow /sb-
Disallow /sc-

googlebot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.sm160.com/sitemap/productIndex
sitemap https://www.sm160.com/sitemap/shopIndex
sitemap https://www.sm160.com/sitemap/categoryindex
Back to top

Comments

  • created by chenWei 2024/6/29
  • 如果想要å
  • 禁止抓取带有查询字符串的URL
  • 禁止抓取列表页面（假设cp, cb, cc为特定å†
  • 禁止抓取搜索结果页面（假设sp, sb, sc为搜索结果页面的前缀）
  • Sitemap files
  • User-agent: Baiduspider
  • User-agent: Baiduspider-image
  • User-agent: 360Spider
  • User-agent: sogou spider
Back to top

Warnings

  • 2 invalid lines.
Back to top