counter-balance.org
robots.txt

Robots Exclusion Standard data for counter-balance.org

Resource Scan

Scan Details

Site Domain counter-balance.org
Base Domain counter-balance.org
Scan Status Ok
Last Scan2025-11-20T22:41:06+00:00
Next Scan 2025-11-27T22:41:06+00:00

Last Scan

Scanned2025-11-20T22:41:06+00:00
URL https://counter-balance.org/robots.txt
Domain IPs 94.237.108.238
Response IP 94.237.108.238
Found Yes
Hash 040b449543b2270684c3db963582929f559f72472772ea2117bbc8da7f49a93b
SimHash c1581872ff81

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/

Other Records

Field Value
crawl-delay 20

amazonbot
semrushbot
ahrefsbot
barkrowler
blexbot
bw/1.1
bytespider
censysinspect
dalvik/2.1.0
dataforseobot
dataprovider
dotbot
expanse
foregenix
gptbot
imagesiftbot
internet-measurement
ioncrawl
java
mj12bot
mozlila
orbbot
petalbot
python-requests
scrapy
wp_is_mobile
awariobot
claudebot
zoominfobot
yandexbot
seznambot
coccocbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://counter-balance.org/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://counter-balance.org/
  • live - don't allow web crawlers to index cpresources/ or vendor/