*
| Rule |
Path |
| Disallow |
/wp-admin/ |
| Disallow |
/wp-includes/ |
| Disallow |
/wp-content/themes/ |
| Disallow |
/wp-content/plugins/ |
| Disallow |
/wp-content/cache/ |
| Disallow |
/wp-content/widget-cache/ |
| Disallow |
*?replytocom |
Other Records
| Field |
Value |
| crawl-delay |
30 |
googlebot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
mediapartners-google
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
20 |
slurp
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
120 |
yahoo! slurp
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
20 |
msnbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
20 |
msnbot-media
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
20 |
bingbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
20 |
twitterbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
tweetedtimes bot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
60 |
tineye-bot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
webindex
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
marfeel-crawler
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
queryseekerspider
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
yandex
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
sogou spider
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
baiduspider
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
youdaobot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
mj12bot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
spbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
seznambot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
ahrefsbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
linkdex
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
linkdexbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
semrushbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
semrushbot-sa
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
cfnetwork
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
sputnikbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
turnitinbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
500 |
proximic
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
30 |
grapeshot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
30 |