consumerreports.com
robots.txt
Robots Exclusion Standard data for consumerreports.com
Resource Scan
Scan Details
Site Domain | consumerreports.com |
Base Domain | consumerreports.com |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a client error. |
Last Scan | 2024-05-04T00:49:53+00:00 |
Next Scan | 2024-05-18T00:49:53+00:00 |
Last Successful Scan
Scanned | 2024-04-19T00:38:21+00:00 |
URL | https://consumerreports.com/robots.txt |
Redirect | https://www.consumerreports.org/robots.txt |
Redirect Domain | www.consumerreports.org |
Redirect Base | consumerreports.org |
Domain IPs | 2600:9000:20a6:3400:d:9b89:840:93a1, 2600:9000:20a6:4600:d:9b89:840:93a1, 2600:9000:20a6:4c00:d:9b89:840:93a1, 2600:9000:20a6:6e00:d:9b89:840:93a1, 2600:9000:20a6:cc00:d:9b89:840:93a1, 2600:9000:20a6:d000:d:9b89:840:93a1, 2600:9000:20a6:e400:d:9b89:840:93a1, 2600:9000:20a6:f000:d:9b89:840:93a1, 99.84.238.174, 99.84.238.199, 99.84.238.202, 99.84.238.99 |
Redirect IPs | 99.84.238.101, 99.84.238.179, 99.84.238.190, 99.84.238.217 |
Response IP | 99.84.238.190 |
Found | Yes |
Hash | a11b8fc4f14f1b85083045f4843ac4a95053a30faf024050a47b6742ad31a6d9 |
SimHash | eb3ad861af13 |
Groups
*
Rule | Path |
---|---|
Disallow | /access |
Disallow | /cro/search.htm |
Disallow | /search* |
Disallow | /health/search.htm |
Disallow | /content/dam/cro/interactive/PRO410_Sept%20Premium.pdf |
Disallow | /bin/home/calendar/ |
Disallow | /*jcr%3Acontent* |
Disallow | /*preview_print.html$ |
Disallow | /*.print.html$ |
Disallow | /data/* |
Disallow | /bin/feedinfo.name%3Drss.xml |
Disallow | /*?*page= |
Disallow | /*?*sort= |
Disallow | /*?*view= |