hivecpq.com
robots.txt

Robots Exclusion Standard data for hivecpq.com

Resource Scan

Scan Details

Site Domain hivecpq.com
Base Domain hivecpq.com
Scan Status Ok
Last Scan2025-08-18T01:03:36+00:00
Next Scan 2025-09-17T01:03:36+00:00

Last Scan

Scanned2025-08-18T01:03:36+00:00
URL https://hivecpq.com/robots.txt
Domain IPs 2a04:3544:1000:1510:3cc8:64ff:fefa:49e, 94.237.42.154
Response IP 94.237.42.154
Found Yes
Hash 6981692224610498ff77d7db6989ce577fb2a44569db4c143db4277fa2540f32
SimHash 75185e427c82

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/
Disallow /en/resources/updates?*
Allow /en/resources/updates?page=
Disallow /nl/resources/updates?*
Allow /nl/resources/updates?page=
Disallow /fr/ressources/actualites?*
Allow /fr/ressources/actualites?page=
Disallow /de/ressourcen/updates?*
Allow /de/ressourcen/updates?page=

Other Records

Field Value
crawl-delay 20

amazonbot
semrushbot
ahrefsbot
barkrowler
blexbot
bw/1.1
bytespider
censysinspect
dalvik/2.1.0
dataforseobot
dataprovider
dotbot
expanse
foregenix
imagesiftbot
internet-measurement
ioncrawl
java
mj12bot
mozlila
orbbot
petalbot
python-requests
scrapy
wp_is_mobile
awariobot
claudebot
zoominfobot
yandexbot
seznambot
coccocbot

No rules defined. All paths allowed.

Other Records

Field Value
sitemap https://hivecpq.com/en/sitemaps-1-sitemap.xml
sitemap https://hivecpq.com/nl/sitemaps-1-sitemap.xml
sitemap https://hivecpq.com/fr/sitemaps-1-sitemap.xml
sitemap https://hivecpq.com/de/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://hivecpq.com/
  • live - don't allow web crawlers to index cpresources/ or vendor/