bensettle.com
robots.txt

Robots Exclusion Standard data for bensettle.com

Resource Scan

Scan Details

Site Domain bensettle.com
Base Domain bensettle.com
Scan Status Ok
Last Scan2025-06-23T03:43:15+00:00
Next Scan 2025-07-23T03:43:15+00:00

Last Scan

Scanned2025-06-23T03:43:15+00:00
URL https://bensettle.com/robots.txt
Domain IPs 104.26.6.215, 104.26.7.215, 172.67.68.195, 2606:4700:20::681a:6d7, 2606:4700:20::681a:7d7, 2606:4700:20::ac43:44c3
Response IP 104.26.6.215
Found Yes
Hash 65da8facd779cca890040b4fff1e9880211e71f5fe1bde748a4a9a9c6caa8874
SimHash 831648d307c3

Groups

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

Warnings

  • 2 invalid lines.