ahrefsbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
30 |
semrushbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
30 |
semrushbot-sa
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
30 |
mediapartners-google
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
5 |
googlebot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
7 |
yandexbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
7 |