dumatobolsk.ru
robots.txt
Robots Exclusion Standard data for dumatobolsk.ru
Resource Scan
Scan Details
Site Domain | dumatobolsk.ru |
Base Domain | dumatobolsk.ru |
Scan Status | Ok |
Last Scan | 2024-09-18T08:35:35+00:00 |
Next Scan | 2024-10-18T08:35:35+00:00 |
Last Scan
Scanned | 2024-09-18T08:35:35+00:00 |
URL | https://dumatobolsk.ru/robots.txt |
Domain IPs | 37.140.192.24 |
Response IP | 37.140.192.24 |
Found | Yes |
Hash | 11c18daac0c027666c289139b6e5802f6e065c3eb7b493d6c81a70f87d9cb42e |
SimHash | 6ec8dfc081d4 |
Groups
*
Rule | Path | Comment |
---|---|---|
Disallow | empty Disallow instruction SHOULD be there | |
Disallow | /bitrix/ | - |
Disallow | /upload | - |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
yandex
Rule | Path | Comment |
---|---|---|
Disallow | empty Disallow instruction SHOULD be there | |
Disallow | /bitrix | - |
Disallow | /upload | - |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
yandexbot
Rule | Path | Comment |
---|---|---|
Disallow | empty Disallow instruction SHOULD be there |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
yandexmedia
Rule | Path | Comment |
---|---|---|
Disallow | empty Disallow instruction SHOULD be there |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
yandeximages
Rule | Path | Comment |
---|---|---|
Disallow | empty Disallow instruction SHOULD be there |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
yandexblogs
Rule | Path | Comment |
---|---|---|
Disallow | empty Disallow instruction SHOULD be there |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
yandexnews
Rule | Path | Comment |
---|---|---|
Disallow | empty Disallow instruction SHOULD be there |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
yandexmetrika
Rule | Path | Comment |
---|---|---|
Disallow | empty Disallow instruction SHOULD be there |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
yandexmarket
Rule | Path | Comment |
---|---|---|
Disallow | empty Disallow instruction SHOULD be there |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
googlebot
Rule | Path | Comment |
---|---|---|
Disallow | empty Disallow instruction SHOULD be there | |
Disallow | /bitrix | - |
Disallow | /upload | - |