nytimes.cf
robots.txt
Robots Exclusion Standard data for nytimes.cf
Resource Scan
Scan Details
| Site Domain | nytimes.cf |
| Base Domain | nytimes.cf |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-10-24T08:56:14+00:00 |
| Next Scan | 2025-01-22T08:56:14+00:00 |
Last Successful Scan
| Scanned | 2023-04-03T19:16:42+00:00 |
| URL | https://nytimes.cf/robots.txt |
| Domain IPs | 104.21.70.9, 172.67.217.174, 2606:4700:3036::6815:4609, 2606:4700:3037::ac43:d9ae |
| Response IP | 172.67.217.174 |
| Found | Yes |
| Hash | 2f53aa82366e5b63d5769be9a5a8e4a2b06dd75086ccd8d32870081eb9104ce8 |
| SimHash | 6cba0c50cd1f |
Groups
yandex
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Disallow | /login/* |
| Disallow | /en/* |
| Disallow | /admin |
| Disallow | /users/* |
| Disallow | /franchise/ |
| Disallow | /*?s=* |
| Disallow | /*?lng=* |
| Disallow | /terms_of_use |
| Disallow | /data_processing_agreement |
| Disallow | /feedback/ |
| Disallow | /cart/order |
| Disallow | /*/mobile/2cab |
| Disallow | /*?flavour=* |
| Disallow | /*_openstat |
| Disallow | /*gclid%3D |
| Disallow | /*giclickid%3D |
| Disallow | *utm |
| Disallow | |
| Disallow | /*.doc |
| Disallow | /*.rtf |
| Disallow | /*.xls |
| Disallow | /*.ppt |
| Disallow | /*.txt |
googlebot
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Disallow | /login |
| Disallow | /en/* |
| Disallow | /admin |
| Disallow | /users/* |
| Disallow | /franchise/ |
| Disallow | /*?s=* |
| Disallow | /*?lng=* |
| Disallow | /terms_of_use |
| Disallow | /data_processing_agreement |
| Disallow | /feedback/ |
| Disallow | /cart/order |
| Disallow | /*/mobile/2cab |
| Disallow | /*?flavour=* |
| Disallow | /*_openstat |
| Disallow | *utm |
| Disallow | /*?utm_source= |
| Disallow | /*?utm_medium= |
| Disallow | /*?utm_campaign= |
| Disallow | /*?utm_content= |
| Disallow | /*?utm_term= |
| Disallow | /*?sid= |
| Disallow | /*?gclid= |
| Disallow | /*giclickid%3D |
| Disallow | /*?yclid= |
| Disallow | |
| Disallow | /*.doc |
| Disallow | /*.rtf |
| Disallow | /*.xls |
| Disallow | /*.ppt |
| Disallow | /*.txt |
*
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Disallow | /login |
| Disallow | /en/* |
| Disallow | /admin |
| Disallow | /users/* |
| Disallow | /franchise/ |
| Disallow | /*?s=* |
| Disallow | /*?lng=* |
| Disallow | /terms_of_use |
| Disallow | /data_processing_agreement |
| Disallow | /feedback/ |
| Disallow | /cart/order |
| Disallow | /*/mobile/2cab |
| Disallow | /*?flavour=* |
| Disallow | /*_openstat |
| Disallow | *utm |
| Disallow | /*?utm_source= |
| Disallow | /*?utm_medium= |
| Disallow | /*?utm_campaign= |
| Disallow | /*?utm_content= |
| Disallow | /*?utm_term= |
| Disallow | /*?sid= |
| Disallow | /*?gclid= |
| Disallow | /*?yclid= |
| Disallow | |
| Disallow | /*.doc |
| Disallow | /*.rtf |
| Disallow | /*.xls |
| Disallow | /*.ppt |
| Disallow | /*.txt |
Other Records
| Field | Value |
|---|---|
| sitemap | https://greenhydra.ml/sitemap.xml |
Warnings
- `clean-param` is not a known field.