Log In Sign Up

forbes.ru
robots.txt

Robots Exclusion Standard data for forbes.ru

Resource Scan

Scan Details

Site Domain forbes.ru
Base Domain forbes.ru
Scan Status Ok
Last Scan2024-06-28T17:03:33+00:00
Next Scan 2024-07-05T17:03:33+00:00

Last Scan

Scanned2024-06-28T17:03:33+00:00
URL https://forbes.ru/robots.txt
Redirect https://www.forbes.ru/robots.txt
Redirect Domain www.forbes.ru
Redirect Base forbes.ru
Domain IPs 158.160.82.210
Redirect IPs 158.160.82.210
Response IP 158.160.82.210
Found Yes
Hash d09dd66e9288af8536c88d31eb348141e8acbb733bfbfdba4282f86a13c75c5c
SimHash 6800ce83cf33

Groups

yandexnews

Rule Path
Allow /newrss.xml

*

Rule Path
Disallow /*..*
Disallow /*embed/
Disallow /taxonomy/
Disallow /searchsite/
Disallow /nodesearch
Disallow /explain/
Disallow /blog/
Disallow /amp/
Disallow */photo/*
Disallow /preview/*
Disallow */slide/*
Disallow */slideshow*
Disallow /*.php
Disallow /*.html
Allow *.css
Allow *.js
Allow *.gif
Allow *.jpg
Allow *.jpeg
Allow *.png
Allow *.svg
Allow *.webp
Allow *.woff
Allow *.woff2
Allow /ads.txt

yandex

Rule Path
Disallow /*..*
Disallow /*embed/
Disallow /taxonomy/
Disallow /searchsite/
Disallow /search
Disallow /nodesearch
Disallow /explain/
Disallow /blog/
Disallow /amp/
Disallow */photo/*
Disallow /preview/*
Disallow */slide/*
Disallow */slideshow*
Disallow /*.php
Disallow /*.html
Allow *.css
Allow *.js
Allow *.gif
Allow *.jpg
Allow *.jpeg
Allow *.png
Allow *.svg
Allow *.webp
Allow *.woff
Allow *.woff2
Allow /ads.txt
Back to top

Other Records

Field Value
sitemap https://www.forbes.ru/sitemap.xml
sitemap https://www.forbes.ru/sitemap.xml
Back to top

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top