misto.zp.ua
robots.txt

Robots Exclusion Standard data for misto.zp.ua

Resource Scan

Scan Details

Site Domain misto.zp.ua
Base Domain misto.zp.ua
Scan Status Ok
Last Scan2024-07-02T03:09:27+00:00
Next Scan 2024-07-09T03:09:27+00:00

Last Scan

Scanned2024-07-02T03:09:27+00:00
URL https://misto.zp.ua/robots.txt
Domain IPs 91.239.233.47
Response IP 91.239.233.47
Found Yes
Hash 6b8e8f42a20fb0f95b638e7a5efbc4657fbca3095151668f32f11b14fa776556
SimHash 4210fce28bf3

Groups

*

Rule Path
Disallow /business/catalog.php?main_category_name
Allow /

Other Records

Field Value
crawl-delay 30

yandex
yandexblogs

Rule Path
Disallow *

mozilla/5.0 (compatible; yandexblogs/0.99; robot; +http://yandex.com/bots)

Rule Path
Disallow *

twiceler

Rule Path
Disallow *

mozilla/5.0 (twiceler-0.9 http://www.cuill.com/twiceler/robot.html)

Rule Path
Disallow *

psbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow *

archive.org

Rule Path
Disallow *

Warnings

  • `host` is not a known field.