cusrev.com
robots.txt

Robots Exclusion Standard data for cusrev.com

Resource Scan

Scan Details

Site Domain cusrev.com
Base Domain cusrev.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-14T20:06:58+00:00
Next Scan 2026-01-12T20:06:58+00:00

Last Successful Scan

Scanned2024-12-19T18:29:36+00:00
URL https://cusrev.com/robots.txt
Redirect https://www.cusrev.com/robots.txt
Redirect Domain www.cusrev.com
Redirect Base cusrev.com
Domain IPs 3.160.188.102, 3.160.188.16, 3.160.188.29, 3.160.188.58
Redirect IPs 3.164.182.61, 3.164.182.67, 3.164.182.79, 3.164.182.80
Response IP 3.164.68.118
Found Yes
Hash 5afab9367a36fbee39eaf9bfe33154c148c9d4f1da58d2e920d285c7cce3c675
SimHash 8d74f1022f2c

Groups

dotbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /reviews/
Disallow /de/reviews/
Disallow /es/reviews/
Disallow /fi/reviews/
Disallow /fr/reviews/
Disallow /it/reviews/
Disallow /nl/reviews/
Disallow /pl/reviews/
Disallow /pt/reviews/
Disallow /sv/reviews/

*

Rule Path
Disallow /dashboard
Disallow /terms.html
Disallow /privacy-policy
Disallow /data-protection

Comments

  • Block Dotbot
  • Block Babbar bot
  • Block GeedoBot
  • Block MJ12bot
  • Block Amazonbot
  • Block service pages