Log In Sign Up

hwp.ru
robots.txt

Robots Exclusion Standard data for hwp.ru

Resource Scan

Scan Details

Site Domain hwp.ru
Base Domain hwp.ru
Scan Status Ok
Last Scan2024-06-23T10:27:11+00:00
Next Scan 2024-07-23T10:27:11+00:00

Last Scan

Scanned2024-06-23T10:27:11+00:00
URL https://hwp.ru/robots.txt
Domain IPs 77.51.183.252
Response IP 77.51.183.252
Found Yes
Hash cb9458f9657e3a9499437d3b4071edf2bdad97f343d994d4532f251dd4bd10b9
SimHash 7f9c7cf3c6cd

Groups

*

Rule Path
Disallow /price/
Disallow /price2/
Disallow /private/
Disallow /search.php
Disallow /search/
Disallow /bitrix/admin/
Disallow /community/
Disallow /Newsarchive/
Disallow /releasesarchive/
Disallow /news2/
Disallow /news/
Disallow /News/
Disallow /forum/
Disallow /tags/
Disallow /releasesarchive/
Disallow /Files.html

Other Records

Field Value
crawl-delay 5

yandex

Rule Path
Disallow /articles/*/?PAGEN_1=1
Disallow *%26print%3Dy
Disallow /articles/*/?PAGEN_1=*%3F
Disallow /articles/*/?PAGEN_1=*&
Disallow /articles/*/?CODE=*
Disallow /articles/*/?CODE=*&
Disallow /price/
Disallow /price2/
Disallow /private/
Disallow /search.php
Disallow /search/
Disallow /bitrix/admin/
Disallow /community/
Disallow /Newsarchive/
Disallow /releasesarchive/
Disallow /news2/
Disallow /news/
Disallow /News/
Disallow /forum/
Disallow /tags/
Disallow /releasesarchive/
Disallow /Files.html
Disallow /*?
Disallow *?PAGEN_1=*%3F
Disallow *?PAGEN_1=*&
Disallow /%26print%3Dy
Disallow */%26SHOWALL_1%3D1
Disallow /*?SHOWALL_1=1%3F*
Disallow /*?SHOWALL_1=1&*
Disallow *.php?SHOWALL_1=1%3F*
Disallow *.php?SHOWALL_1=1&*
Disallow */article.php?ID=*%3F*
Disallow */article.php?ID=*&*
Disallow */article.php?ID=*%3F*
Disallow */article.php?ID=*%3FPAGEN_1=1
Disallow */article.php?ID=*%3FPAGEN_1=*%3F
Disallow */article.php?ID=*%3FSHOWALL_1=1%3F*
Disallow */article.php?ID=*&*
Disallow */article.php?ID=*%3FPAGEN_1=*&
Disallow */article.php?ID=*%3FSHOWALL_1=1&
Disallow /*%5B*
Disallow /*%3C
Disallow /*manifest.json
Disallow *html%
Disallow *?sphrase_id=*
Disallow */articles/*/index.php
Allow *?PAGEN_1=*
Allow */article.php?ID=*
Allow */article.php?ID=*%3FPAGEN_1=*

Other Records

Field Value
crawl-delay 2

googlebot

Rule Path
Disallow /*?
Disallow /price/
Disallow /price2/
Disallow /private/
Disallow /search.php
Disallow /search/
Disallow /bitrix/admin/
Disallow /community/
Disallow /Newsarchive/
Disallow /releasesarchive/
Disallow /news2/
Disallow /news/
Disallow /News/
Disallow /forum/
Disallow /tags/
Disallow /releasesarchive/
Disallow /Files.html
Disallow /*manifest.json
Disallow */?PAGEN_1=1
Disallow *.php?PAGEN_1=1
Disallow *.html?PAGEN_1=1
Disallow */article.php?ID=*%3FPAGEN_1=1
Disallow *?PAGEN_1=*%3F
Disallow *?PAGEN_1=*&
Disallow *?SHOWALL_1=1%3F*
Disallow *?SHOWALL_1=1&*
Disallow *.php?PAGEN_1=*%3F
Disallow *.php?PAGEN_1=*&
Disallow *.php?SHOWALL_1=1%3F*
Disallow *.php?SHOWALL_1=1&*
Disallow *?CODE=*
Disallow *?CODE=*&*
Allow *?PAGEN_1=*
Allow /article.php?ID=*
Disallow /article.php?ID=*%3F
Disallow /article.php?ID=*&
Allow /article.php?ID=*%3FPAGEN_1=*
Disallow /article.php?ID=*%3FPAGEN_1=*%3F
Disallow /article.php?ID=*%3FPAGEN_1=*&
Disallow /article.php?ID=*%3FSHOWALL_1=1%3F*
Disallow /article.php?ID=*%3FSHOWALL_1=1&*
Disallow /*%5B*
Disallow /*%3C
Disallow *html%
Disallow *?sphrase_id=*
Disallow */articles/*/index.php

sputnikbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ahrefsbot/5.2

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

spbot

Rule Path
Disallow /

linkdexbot/2.0

Rule Path
Disallow /

linkdexbot/2.1

Rule Path
Disallow /

linkdexbot/2.2

Rule Path
Disallow /

linkdexbot/2.3

Rule Path
Disallow /

ntentbot

Rule Path
Disallow /

converacrawler

Rule Path
Disallow /

smtbot

Rule Path
Disallow /
Back to top

Comments

  • ЯША!
  • ПРАВИЛО ДЛЯ /articles/
  • Disallow: /articles/*/*
  • Allow: /articles/*/?PAGEN_1=*
  • ОБЩАЯ КАРТА
  • ОТКЛЮЧАЕМ ПАРАМЕТРЫ
  • Disallow: *?print=y*
  • Disallow: *?PAGEN_1=1
  • Разрешаем определенные параметры
  • Allow: */?SHOWALL_1=1
  • Allow: *.php?SHOWALL_1=1
  • Allow: */article.php?ID=*?SHOWALL_1=1
  • ГОША
  • ОБЩАЯ КАРТА
  • ОБЩАЯ КАРТА
  • ЗАПРЕЩАЕМ ПЕРВУЮ СТРАНИЦУ УТОЧНЕННОЙ МАСКОЙ
  • Disallow: *?print=y*
  • Disallow: *?PAGEN_1=1
  • ЗАПРЕЩАЕМ СУММУ ПАРАМЕТРОВ ОБЩЕЙ МАСКОЙ
  • РАЗРЕШАЕМ ПАГИНАЦИЮ ОБЩЕЙ МАСКОЙ
  • Allow: */?SHOWALL_1=1
  • Allow: *.php?SHOWALL_1=1
  • ОТДЕЛЬНОЕ ПРАВИЛО ДЛЯ ARTICLE.PHP
  • Allow: /article.php?ID=*?SHOWALL_1=1
Back to top

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top