Log In Sign Up

bitrix24site.ru
robots.txt

Robots Exclusion Standard data for bitrix24site.ru

Resource Scan

Scan Details

Site Domain bitrix24site.ru
Base Domain bitrix24site.ru
Scan Status Ok
Last Scan2025-11-12T09:59:52+00:00
Next Scan 2025-12-12T09:59:52+00:00

Last Scan

Scanned2025-11-12T09:59:52+00:00
URL https://bitrix24site.ru/robots.txt
Redirect https://www.bitrix24.com/robots.txt
Redirect Domain www.bitrix24.com
Redirect Base bitrix24.com
Domain IPs 185.137.235.2
Redirect IPs 185.104.211.34
Response IP 185.104.211.34
Found Yes
Hash 066751f80409c5440f163dd5ef37882674191379359194f9337c01d2a4679f2c
SimHash 2e15d3226467

Groups

*

Rule Path
Disallow /*rss
Disallow /auth.php
Disallow /auth/*
Disallow /bitrix/*
Disallow /email/*
Disallow /include/
Disallow /merge/
Disallow /register/*
Disallow /search/*
Disallow /merge/
Disallow /register/*
Disallow /search/*
Disallow /subscription-management.php
Disallow /prices/vacation.php
Disallow /my/prices/vacation.php
Disallow /th/prices/vacation.php
Disallow /self-hosted/trial-key-request.php
Disallow /tc/*
Disallow /hi/*
Disallow /report/*
Disallow /prices/space.php*
Disallow /promo/x2-x3/*
Disallow /promo/save-big-with-bitrix24/*
Disallow /*.pdf$
Disallow /*.xls$
Disallow /*.doc$
Disallow /*.ppt$
Disallow /*.txt$
Disallow /*.docx$
Disallow /*.xlsx$
Disallow /*.pptx$
Disallow /*.csv$
Allow */?product=
Allow */?app=
Allow /bitrix/cache/*
Allow /bitrix/blocks/*
Allow /bitrix/components/*
Allow /bitrix/content_fonts/*
Allow /bitrix/content_vendors/*
Allow /bitrix/css/*
Allow /bitrix/js/*
Allow /bitrix/images/*
Allow /bitrix/templates/*
Allow /bitrix/panel/*
Allow /local/components/*

bingbot
msnbot
duckduckbot
baiduspider
slurp

Rule Path
Disallow /*?*
Disallow /*%26*

googlebot
googlebot-image
googlebot-mobile
googlebot-news
googlebot-video

Rule Path
Allow /*?
Allow /*%26
Allow /*PAGEN_*

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

gptbot
chatgpt-user
anthropic-ai
claude-web
claudebot
google-extended
ccbot
cohere-ai
perplexity-bot
bingbot-copilot
oai-searchbot
facebookbot
youbot
duckassistbot

Rule Path
Disallow /bitrix/*
Disallow /local/*
Disallow /bitrix/cache/*
Disallow /bitrix/blocks/*
Disallow /bitrix/components/*
Disallow /bitrix/content_fonts/*
Disallow /bitrix/content_vendors/*
Disallow /bitrix/css/*
Disallow /bitrix/js/*
Disallow /bitrix/templates/*
Disallow /bitrix/panel/*
Disallow /local/components/*
Disallow /*?*
Disallow /*%26*
Allow */?product=
Allow */?app=
Allow /bitrix/images/*
Back to top

Other Records

Field Value
sitemap https://www.bitrix24.com/sitemap.xml
Back to top

Comments

  • All search engines
  • Admin section
  • Custom pages
  • Tmp files and Cache
  • Global Allow rules
  • Another search systems
  • Rules for Google
  • Pages with parameters
  • Rules for Yandex
  • Special parameters for Yandex
  • AI User-agents
  • Disallows for AI Bots
  • Allows for AI Bots
  • Sitemap
Back to top

Warnings

  • `clean-param` is not a known field.
Back to top