rb.ru
robots.txt

Robots Exclusion Standard data for rb.ru

Resource Scan

Scan Details

Site Domain rb.ru
Base Domain rb.ru
Scan Status Ok
Last Scan2024-11-18T08:43:29+00:00
Next Scan 2024-11-25T08:43:29+00:00

Last Scan

Scanned2024-11-18T08:43:29+00:00
URL https://rb.ru/robots.txt
Domain IPs 104.21.234.198, 104.21.234.199, 2606:4700:3038::6815:eac6, 2606:4700:3038::6815:eac7
Response IP 104.21.234.198
Found Yes
Hash a2333a733f1727b119bfccc300feb95c044ce914441233b2c463dbaa5a61ce1e
SimHash 777456693925

Groups

*

Rule Path
Disallow *?erid=
Disallow /admin
Disallow /search/*
Disallow /*/secretpreview/
Disallow /smi/
Disallow /events
Disallow /auth/begin/
Disallow /banners/view/*
Disallow /blogs/?tag=*&author=*
Disallow /edit/
Disallow /*/?&l=*
Disallow /*/?&e=*
Disallow /*/?next=*
Disallow /*/*%26next%3D*
Disallow /*/?type=*
Disallow /job/*
Disallow /blogs/?tag=*&author=*
Disallow /pipeline/*
Disallow /banners/go/
Disallow /organizer/
Disallow /office/
Disallow /accounts/
Disallow */?utm_source=*
Disallow */edit/?section=*
Disallow /blogs/author/asfaganova/*
Disallow */?niche=*
Disallow /api/*
Disallow /payment/*
Disallow /digital/get_mail/
Disallow /investors/?activity_niche=6
Disallow /investors/?activity_niche=9
Disallow /investors/?activity_niche=2
Disallow /investors/?activity_niche=4
Disallow /investors/?regions=103
Disallow /investors/?regions=473
Disallow /news/how-to-subscribe/?from=tag
Disallow /companies/?regions=473
Disallow /companies/?regions=481
Disallow /opinion/certification-china/?from=blogers
Disallow /investors/?regions=1
Disallow /opinion/23-neural-networks/?from=blogers
Disallow /investors/?regions=28
Disallow /companies/?activity_niche=2
Disallow /companies/?activity_niche=4
Disallow /companies/?activity_niche=51
Disallow /companies/?activity_niche=81
Disallow /companies/?activity_niche=45
Disallow /companies/?activity_niche=111
Disallow /companies/?activity_niche=114
Disallow /companies/?activity_niche=116
Disallow /companies/?activity_niche=148
Disallow /companies/?activity_niche=154
Disallow /companies/?activity_niche=155
Disallow /companies/?activity_niche=156
Disallow /companies/?activity_niche=165
Disallow /opinion/software-devepolment-trends/?from=blogers
Disallow /blogers/?from=opinion
Disallow /opinion/project-vs-gpt/?from=blogers
Disallow /opinion/markirovka-sep/?from=blogers
Disallow /companies/?regions=1
Disallow /awards/?from=article
Disallow /news/start-hackathon2025/?from=tag
Disallow /investors/?investment_round=3
Disallow /investors/?investment_round=2
Disallow /investors/?investment_round=1
Disallow /opinion/datasaentist/?from=blogers
Disallow /2018/?from=button

zen_bot/2.0

Rule Path
Allow *?utm_source=
Allow *?from=

yandexnews

Rule Path
Allow /rss/yandex/
Allow *?from=yanews
Allow */yanews/
Allow *?utm_source=

Other Records

Field Value
sitemap https://rb.ru/sitemap.xml

Warnings

  • `clean-param` is not a known field.