Log In Sign Up

news.mail.ru
robots.txt

Robots Exclusion Standard data for news.mail.ru

Resource Scan

Scan Details

Site Domain news.mail.ru
Base Domain mail.ru
Scan Status Ok
Last Scan2024-05-16T03:34:18+00:00
Next Scan 2024-05-23T03:34:18+00:00

Last Scan

Scanned2024-05-16T03:34:18+00:00
URL https://news.mail.ru/robots.txt
Domain IPs 5.61.236.237
Response IP 5.61.236.237
Found Yes
Hash 6ad0e4674dd20384699338ac6d08b35ddb6e6c4caaa1fab544076f8ae12321ad
SimHash 7459d5105730

Groups

*

Rule Path
Allow /infographics/
Disallow */ajax/
Disallow */page/
Disallow */inf/
Disallow */ext/
Disallow */search/
Disallow */monitoring/
Disallow /*q%3D
Disallow */dashboards/
Disallow */json/

googlebot

Rule Path
Allow /infographics/
Disallow */ajax/
Disallow */page/
Disallow */inf/
Disallow */ext/
Disallow */search/
Disallow */monitoring/
Disallow /*q%3D
Disallow */dashboards/
Disallow */json/

yandex

Rule Path
Allow /infographics/
Disallow */ajax/
Disallow */page/
Disallow */inf/
Disallow */ext/
Disallow */search/
Disallow */monitoring/
Disallow /*q%3D
Disallow */dashboards/
Disallow */json/
Disallow */amp/
Back to top

Other Records

Field Value
sitemap https://news.mail.ru/sitemap_index.xml
sitemap https://news.mail.ru/sitemap/news/google/news/
Back to top

Warnings

  • `clean-param` is not a known field.
Back to top