it.cointelegraph.com
robots.txt
Robots Exclusion Standard data for it.cointelegraph.com
Resource Scan
Scan Details
| Site Domain | it.cointelegraph.com |
| Base Domain | cointelegraph.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-04-20T09:09:23+00:00 |
| Next Scan | 2024-07-19T09:09:23+00:00 |
Last Successful Scan
| Scanned | 2023-12-15T20:32:35+00:00 |
| URL | https://it.cointelegraph.com/robots.txt |
| Domain IPs | 104.19.210.104, 104.19.211.104 |
| Response IP | 104.19.211.104 |
| Found | Yes |
| Hash | da381bc275677a63720734b80e49ad3b3f42bcecbfa1250a7157f13ecdb29a3e |
| SimHash | 641186d0e253 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /submit-a-post |
| Disallow | /exchange |
| Disallow | /marketcap |
| Disallow | /events |
| Disallow | /ico-101 |
| Disallow | /ico-calendar |
| Disallow | /embed/post-sidebar/ |
| Disallow | /embed/post-recomend/ |
| Disallow | /ripple-101 |
| Disallow | /lightning-network-101 |
| Disallow | /category/quiz |
| Disallow | /search?query=* |
| Disallow | /search |
Other Records
| Field | Value |
|---|---|
| sitemap | https://it.cointelegraph.com/sitemap.xml |