brendan-power.com
robots.txt

Robots Exclusion Standard data for brendan-power.com

Resource Scan

Scan Details

Site Domain brendan-power.com
Base Domain brendan-power.com
Scan Status Ok
Last Scan2025-08-26T23:36:37+00:00
Next Scan 2025-09-25T23:36:37+00:00

Last Scan

Scanned2025-08-26T23:36:37+00:00
URL https://brendan-power.com/robots.txt
Domain IPs 57.129.136.239
Response IP 57.129.136.239
Found Yes
Hash 628a482e0a8edd50ec46f9a32bb3acf9fe11291c08ac25e620e17321c68f717c
SimHash 993889687dc3

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /test/
Disallow /ejunkie/

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path
Disallow /forum/

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

xenu’s

Rule Path
Disallow /

xenu’s link sleuth 1.1c

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

Comments

  • Disallow the following directories
  • Completely disallow the following bots