cn.nytimes.com
robots.txt
Robots Exclusion Standard data for cn.nytimes.com
Resource Scan
Scan Details
Site Domain | cn.nytimes.com |
Base Domain | nytimes.com |
Scan Status | Ok |
Last Scan | 2024-03-23T01:34:14+00:00 |
Next Scan | 2024-04-22T01:34:14+00:00 |
Last Scan
Scanned | 2024-03-23T01:34:14+00:00 |
URL | https://cn.nytimes.com/robots.txt |
Domain IPs | 108.156.133.20, 108.156.133.43, 108.156.133.60, 108.156.133.90 |
Response IP | 108.156.133.43 |
Found | Yes |
Hash | 998fc6fdea5cc32609422fb1b3a1b6ac0cd0883fa452f86c9981ba988e3581b0 |
SimHash | fc1c111baf35 |
Groups
*
Rule | Path |
---|---|
Disallow | /ad-test/ |
Disallow | /helix-test/ |
Disallow | /async/ |
Disallow | /users/ |
Disallow | /sso/ |
Disallow | /tools/ |
Disallow | /search/ |
Disallow | /email/ |
Disallow | /*?*changeLang=zh-hant |
Disallow | /*?*changeLang=zh-hans |
Other Records
Field | Value |
---|---|
sitemap | https://cn.nytimes.com/sitemap.xml |
Comments