dia-dieta.ru
robots.txt

Robots Exclusion Standard data for dia-dieta.ru

Resource Scan

Scan Details

Site Domain dia-dieta.ru
Base Domain dia-dieta.ru
Scan Status Ok
Last Scan2024-05-31T14:00:47+00:00
Next Scan 2024-06-07T14:00:47+00:00

Last Scan

Scanned2024-05-31T14:00:47+00:00
URL https://dia-dieta.ru/robots.txt
Domain IPs 87.236.16.169
Response IP 87.236.16.169
Found Yes
Hash 1fe809f20e5ed9f25bbe96a88d2add6b00cf57af90bbd567c540a9adf1e2a24d
SimHash 6580d8629330

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /author
Disallow /search/
Disallow /search/*

yandex

Rule Path
Allow /feed/turbo/
Allow */uploads
Allow /*/*.js
Allow /*/*.css
Allow /wp-*.png
Allow /wp-*.jpg
Allow /wp-*.jpeg
Allow /wp-*.gif
Allow /wp-admin/admin-ajax.php
Disallow /cgi-bin
Disallow /wp-
Disallow /wp/
Disallow *?s=
Disallow *%26s%3D
Disallow *?do=
Disallow /search/
Disallow /search/*
Disallow /author/
Disallow /users/
Disallow /wp-json
Disallow /wp-admin/
Disallow /wp-login/
Disallow /wp-login*
Disallow */trackback
Disallow */embed
Disallow */wlwmanifest.xml
Disallow /wp-login.php?*
Disallow */?*

googlebot

Rule Path
Disallow /search/
Disallow /search/*

mail.ru

Rule Path
Disallow
Disallow /search/
Disallow /search/*

Other Records

Field Value
crawl-delay 250000

Other Records

Field Value
sitemap https://dia-dieta.ru/sitemap.xml

Comments

  • Disallow: /sitemap.html

Warnings

  • 1 invalid line.
  • `host` is not a known field.