Log In Sign Up

news29.ru
robots.txt

Robots Exclusion Standard data for news29.ru

Resource Scan

Scan Details

Site Domain news29.ru
Base Domain news29.ru
Scan Status Ok
Last Scan2024-11-11T21:02:32+00:00
Next Scan 2024-11-18T21:02:32+00:00

Last Scan

Scanned2024-11-11T21:02:32+00:00
URL https://news29.ru/robots.txt
Redirect https://www.news29.ru/robots.txt
Redirect Domain www.news29.ru
Redirect Base news29.ru
Domain IPs 84.201.172.196
Redirect IPs 84.201.172.196
Response IP 84.201.172.196
Found Yes
Hash d81f2454c05578c7d30d9a4e58f32a627dba6d16ef1903f19d5443b8005437e5
SimHash 68111f5e4676

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /reklama/?*
Disallow /reklama/*?*
Disallow /reklama/ban*
Disallow /reklama?*
Disallow /mobile*
Disallow /*?*
Disallow /index.asp
Disallow /index.php
Disallow /index.jsp
Disallow /index.pl
Disallow /index.py
Disallow /novosti_za_period/*
Disallow /novosti/*print
Disallow /*/page/*
Disallow /glavnye_novosti_arhangelska/*
Disallow */glavnye_novosti_arhangelska/*
Disallow /pda/*
Disallow /?remembered
Disallow /?oldSite
Disallow /admin*
Disallow /manager*
Disallow /admin/*
Disallow /manager/*
Disallow /user*
Back to top

Other Records

Field Value
sitemap http://www.news29.ru/sitemap.xml
Back to top

Comments

  • Disallow: /m/*
  • Host: xn--29-dlcyxgbyj.xn--p1ai
  • Request-rate: 1/3
Back to top

Warnings

  • `host` is not a known field.
Back to top