thebci.org
robots.txt

Robots Exclusion Standard data for thebci.org

Resource Scan

Scan Details

Site Domain thebci.org
Base Domain thebci.org
Scan Status Ok
Last Scan2024-05-26T12:19:18+00:00
Next Scan 2024-06-09T12:19:18+00:00

Last Scan

Scanned2024-05-26T12:19:18+00:00
URL https://thebci.org/robots.txt
Domain IPs 104.26.12.42, 104.26.13.42, 172.67.68.204, 2606:4700:20::681a:c2a, 2606:4700:20::681a:d2a, 2606:4700:20::ac43:44cc
Response IP 104.26.12.42
Found Yes
Hash 5c88cdad81714727016b7d7c2778aae0a80960a673b244c10614f0bffa0099ab
SimHash 0b82dbe15312

Groups

*

Rule Path
Disallow */asset/*
Disallow */login/*
Disallow */cpd-activity-detail/*
Disallow */resource/*
Disallow */member-detail/*
Disallow */message_centre/*
Disallow */trackResourceDownload/*
Disallow *?utm_campaign*

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 120

Comments

  • robots.txt for https://www.thebci.org/
  • Sitemap: https://www.thebci.org/sitemap.xml