Log In Sign Up

sm160.com
robots.txt

Robots Exclusion Standard data for sm160.com

Resource Scan

Scan Details

Site Domain sm160.com
Base Domain sm160.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-14T12:42:50+00:00
Next Scan 2024-09-28T12:42:50+00:00

Last Successful Scan

Scanned2024-08-30T12:30:25+00:00
URL https://sm160.com/robots.txt
Redirect https://www.sm160.com/robots.txt?WebShieldDRSessionVerify=XRJ5eEJxuQFCesaib3XN
Redirect Domain www.sm160.com
Redirect Base sm160.com
Domain IPs 103.39.220.208
Redirect IPs 103.39.220.208, 121.201.67.107
Response IP 103.39.220.208
Found Yes
Hash 4f5c2f694e391886e0ece39b9d491ea46c611203aee0a1012c43c27878bcff39
SimHash 1f36eb548fd3

Groups

*

Rule Path
Disallow /*?*
Disallow /cp*/p*
Disallow /cb*/p*
Disallow /cc*/p*
Disallow /sp-
Disallow /sb-
Disallow /sc-

googlebot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.sm160.com/sitemap/productIndex
sitemap https://www.sm160.com/sitemap/shopIndex
sitemap https://www.sm160.com/sitemap/categoryindex
Back to top

Comments

  • created by chenWei 2024/6/29
  • 如果想要å
  • 禁止抓取带有查询字符串的URL
  • 禁止抓取列表页面（假设cp, cb, cc为特定å†
  • 禁止抓取搜索结果页面（假设sp, sb, sc为搜索结果页面的前缀）
  • Sitemap files
  • User-agent: Baiduspider
  • User-agent: Baiduspider-image
  • User-agent: 360Spider
  • User-agent: sogou spider
Back to top

Warnings

  • 2 invalid lines.
Back to top