newspicks.com
robots.txt
Robots Exclusion Standard data for newspicks.com
Resource Scan
Scan Details
| Site Domain | newspicks.com |
| Base Domain | newspicks.com |
| Scan Status | Ok |
| Last Scan | 2024-06-25T18:28:42+00:00 |
| Next Scan | 2024-07-02T18:28:42+00:00 |
Last Scan
| Scanned | 2024-06-25T18:28:42+00:00 |
| URL | https://newspicks.com/robots.txt |
| Domain IPs | 13.35.18.107, 13.35.18.60, 13.35.18.77, 13.35.18.87 |
| Response IP | 13.35.18.77 |
| Found | Yes |
| Hash | 59c277046a2199ebfb08339f7f535f5f250e3f726f9784cd205e96ca4e4caa79 |
| SimHash | 482c0826aa92 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /adv/logs/ |
| Disallow | /logs/ |
| Disallow | /news/*/log/ |
| Disallow | /policy/recruitment-agreement/ |
| Disallow | /top/fragments/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://newspicks.com/sitemap.xml |
Comments