petalbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
5 |
amazonbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
5 |
yandexbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
5 |
yandexantivirus
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
5 |
yandexblogs
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
5 |
googlebot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
5 |
mail.ru_bot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
5 |
bingbot/2.0
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
5 |
facebookexternalhit
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
5 |
mj12bot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
15 |
dotbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
15 |