Log In Sign Up

forbes.ru
robots.txt

Robots Exclusion Standard data for forbes.ru

Resource Scan

Scan Details

Site Domain forbes.ru
Base Domain forbes.ru
Scan Status Ok
Last Scan2024-11-09T04:03:29+00:00
Next Scan 2024-11-16T04:03:29+00:00

Last Scan

Scanned2024-11-09T04:03:29+00:00
URL https://forbes.ru/robots.txt
Redirect https://www.forbes.ru/robots.txt
Redirect Domain www.forbes.ru
Redirect Base forbes.ru
Domain IPs 158.160.82.210
Redirect IPs 158.160.82.210
Response IP 158.160.82.210
Found Yes
Hash 9d5f0c89babe9749916e7e4a11c6ac0267113bcc825cf96e2d0582ac171c8202
SimHash 6c408e83cf33

Groups

yandexnews

Rule Path
Allow /newrss.xml

*

Rule Path
Disallow /*..*
Disallow /*embed/
Disallow /taxonomy/
Disallow /searchsite/
Disallow /nodesearch
Disallow /explain/
Disallow /blog/
Disallow /amp/
Disallow */photo/*
Disallow /preview/*
Disallow */slide/*
Disallow */slideshow*
Disallow /*.php
Disallow /*.html
Allow *.css
Allow *.js
Allow *.gif
Allow *.jpg
Allow *.jpeg
Allow *.png
Allow *.svg
Allow *.webp
Allow *.woff
Allow *.woff2
Allow /ads.txt

yandex

Rule Path
Disallow /*..*
Disallow /*embed/
Disallow /taxonomy/
Disallow /searchsite/
Disallow /search
Disallow /nodesearch
Disallow /explain/
Disallow /blog/
Disallow /amp/
Disallow */photo/*
Disallow /preview/*
Disallow */slide/*
Disallow */slideshow*
Disallow /*.php
Disallow /*.html
Allow *.css
Allow *.js
Allow *.gif
Allow *.jpg
Allow *.jpeg
Allow *.png
Allow *.svg
Allow *.webp
Allow *.woff
Allow *.woff2
Allow /ads.txt
Back to top

Other Records

Field Value
sitemap https://www.forbes.ru/sitemap.xml
sitemap https://www.forbes.ru/sitemap.xml
Back to top

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top