Log In Sign Up

top.mail.ru
robots.txt

Robots Exclusion Standard data for top.mail.ru

Resource Scan

Scan Details

Site Domain top.mail.ru
Base Domain mail.ru
Scan Status Ok
Last Scan2024-05-25T05:56:40+00:00
Next Scan 2024-06-24T05:56:40+00:00

Last Scan

Scanned2024-05-25T05:56:40+00:00
URL https://top.mail.ru/robots.txt
Domain IPs 95.163.52.44
Response IP 95.163.52.44
Found Yes
Hash 15a7f79aa42b66621c7c9d672c5aa81d2c7f014aaa963b46ba3dc6f0bf50b41d
SimHash 1011feb3e3ca

Groups

googlebot

Rule Path
Disallow /cgi-bin/
Disallow /json/
Disallow /xml/
Disallow /export/
Disallow /table/
Disallow /*.csv
Disallow /*.xls
Disallow /*.embed
Disallow /*.hc
Disallow /*.png
Disallow /jump
Disallow /add
Disallow /edit
Disallow /code
Disallow /passremind
Disallow /feedback
Disallow /diff
Disallow /mycounters
Disallow /visits
Disallow /dynamics
Disallow /sources
Disallow /complex
Disallow /rights
Disallow /conf
Disallow /realtime
Disallow /rt
Disallow /keywords
Allow /Rating
Allow /overall
Disallow /Rating/Adult
Disallow /Rating/adult
Disallow /*?

*

Rule Path
Disallow /cgi-bin/
Disallow /json/
Disallow /xml/
Disallow /export/
Disallow /table/
Disallow /*.csv
Disallow /*.xls
Disallow /*.embed
Disallow /*.hc
Disallow /*.png
Disallow /jump
Disallow /add
Disallow /edit
Disallow /code
Disallow /passremind
Disallow /feedback
Disallow /diff
Disallow /mycounters
Disallow /visits
Disallow /dynamics
Disallow /sources
Disallow /complex
Disallow /rights
Disallow /conf
Disallow /realtime
Disallow /rt
Disallow /keywords
Allow /Rating
Allow /overall
Disallow /Rating/Adult
Disallow /Rating/adult
Disallow /*?

Other Records

Field Value
crawl-delay 0.1

ia_archiver

Rule Path
Disallow /

ranksonicbot

Rule Path
Disallow /
Back to top

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top