proaeroporty.ru
robots.txt

Robots Exclusion Standard data for proaeroporty.ru

Resource Scan

Scan Details

Site Domain proaeroporty.ru
Base Domain proaeroporty.ru
Scan Status Ok
Last Scan2024-09-29T07:57:18+00:00
Next Scan 2024-10-06T07:57:18+00:00

Last Scan

Scanned2024-09-29T07:57:18+00:00
URL https://proaeroporty.ru/robots.txt
Domain IPs 45.8.97.198
Response IP 45.8.97.198
Found Yes
Hash 7eaf8b67eaa160f4d1e11766e0502bbc0542cf401214e9093a569a0045daee15
SimHash cb0eda51e610

Groups

blexbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

python-requests/2.12.4

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

senutobot

Rule Path
Disallow /

*

Rule Path
Disallow /oteli/
Disallow /generator_otel/
Disallow /generator_airport/
Disallow /error404.html
Disallow /modules/
Disallow /editor/
Disallow /weather/

yandex

Rule Path
Disallow /oteli/
Disallow /generator_otel/
Disallow /generator_airport/
Disallow /error404.html
Disallow /modules/
Disallow /editor/
Disallow /weather/

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://proaeroporty.ru/sitemap-airporty.xml
sitemap https://proaeroporty.ru/sitemap-airporty_en.xml
sitemap https://proaeroporty.ru/sitemap-timetable.xml
sitemap https://proaeroporty.ru/sitemap-airporty-goroda.xml
sitemap https://proaeroporty.ru/sitemap-airporty-goroda_en.xml

Comments

  • User-agent: Mediapartners-Google
  • Disallow:
  • Disallow: /airports/onlain-raspisanie-tablo-aeroporta
  • Disallow: /airports/onlain-raspisanie-tablo-aeroporta