*
| Rule |
Path |
| Disallow |
/wp-content/uploads/wc-logs/ |
| Disallow |
/wp-content/uploads/woocommerce_transient_files/ |
| Disallow |
/wp-content/uploads/woocommerce_uploads/ |
| Disallow |
/wp-admin/ |
| Allow |
/wp-admin/admin-ajax.php |
Other Records
| Field |
Value |
| crawl-delay |
10 |
ai2bot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
ai2bot-dolma
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
aihitbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
amazonbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
applebot-extended
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
anthropic-ai
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
bytespider
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
ccbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
chatgpt-user
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
claudebot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
claude-user
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
claude-searchbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
cohere-ai
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
cohere-training-data-crawler
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
cotoyogi
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
crawlspace
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
diffbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
facebookbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
factset_spyderbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
firecrawlagent
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
friendlycrawler
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
gptbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
google-extended
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
imagesiftbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
kangaroo bot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
meta-externalagent
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
meta-externalfetcher
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
oai-searchbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
omgili
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
omgilibot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
pangubot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
petalbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
perplexitybot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
perplexity‑user
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
scrapy
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
semrushbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
semrushbot-ocob
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
semrushbot-ft
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
sentibot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
sentibot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
timpibot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
turnitinbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
youbot
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
webzio
No rules defined. All paths allowed.
Other Records
| Field |
Value |
| crawl-delay |
10 |
webzio-extended
Other Records
| Field |
Value |
| crawl-delay |
10 |
Comments