Log In Sign Up

auto-msk1.top
robots.txt

Robots Exclusion Standard data for auto-msk1.top

Resource Scan

Scan Details

Site Domain auto-msk1.top
Base Domain auto-msk1.top
Scan Status Ok
Last Scan2025-10-05T15:51:25+00:00
Next Scan 2025-11-04T15:51:25+00:00

Last Scan

Scanned2025-10-05T15:51:25+00:00
URL https://www.auto-msk1.top/robots.txt
Domain IPs 104.21.50.171, 172.67.164.105, 2606:4700:3033::6815:32ab, 2606:4700:3036::ac43:a469
Response IP 172.67.164.105
Found Yes
Hash 60af5e72d886d30b4bfebda5e92c2fc756967b058643dc79822dfa42a4abd705
SimHash 6918c2e0ce33

Groups

yandex

Rule Path
Allow /
Disallow /cgi-bin/
Disallow /tmp/

Other Records

Field Value
crawl-delay 1

*

Rule Path
Allow /
Disallow /cgi-bin/

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://auto-mskm.top/sitemap.xml
sitemap https://auto-mskm.top/sitemap.xml
Back to top

Comments

  • robots.txt — базовый шаблон (Яндекс + универсально)
  • ЗАМЕНИТЕ example.com на ваш домен
  • Игнорируем "мусорные" параметры в URL (Яндекс-специфично)
Back to top

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top