consumerreports.com
robots.txt

Robots Exclusion Standard data for consumerreports.com

Resource Scan

Scan Details

Site Domain consumerreports.com
Base Domain consumerreports.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-04T00:49:53+00:00
Next Scan 2024-05-18T00:49:53+00:00

Last Successful Scan

Scanned2024-04-19T00:38:21+00:00
URL https://consumerreports.com/robots.txt
Redirect https://www.consumerreports.org/robots.txt
Redirect Domain www.consumerreports.org
Redirect Base consumerreports.org
Domain IPs 2600:9000:20a6:3400:d:9b89:840:93a1, 2600:9000:20a6:4600:d:9b89:840:93a1, 2600:9000:20a6:4c00:d:9b89:840:93a1, 2600:9000:20a6:6e00:d:9b89:840:93a1, 2600:9000:20a6:cc00:d:9b89:840:93a1, 2600:9000:20a6:d000:d:9b89:840:93a1, 2600:9000:20a6:e400:d:9b89:840:93a1, 2600:9000:20a6:f000:d:9b89:840:93a1, 99.84.238.174, 99.84.238.199, 99.84.238.202, 99.84.238.99
Redirect IPs 99.84.238.101, 99.84.238.179, 99.84.238.190, 99.84.238.217
Response IP 99.84.238.190
Found Yes
Hash a11b8fc4f14f1b85083045f4843ac4a95053a30faf024050a47b6742ad31a6d9
SimHash eb3ad861af13

Groups

*

Rule Path
Disallow /access
Disallow /cro/search.htm
Disallow /search*
Disallow /health/search.htm
Disallow /content/dam/cro/interactive/PRO410_Sept%20Premium.pdf
Disallow /bin/home/calendar/
Disallow /*jcr%3Acontent*
Disallow /*preview_print.html$
Disallow /*.print.html$
Disallow /data/*
Disallow /bin/feedinfo.name%3Drss.xml
Disallow /*?*page=
Disallow /*?*sort=
Disallow /*?*view=

zing-bottabot/2.0

Rule Path
Disallow /

google-http-java-client/1.17.0-rc (gzip)

Rule Path
Disallow /video/.*

istellabot/t.1

Rule Path
Disallow /video/.*

pcore-http/v0.24.5

Rule Path
Disallow /video/.*

trendkite-akashic-crawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /