craftself.com
robots.txt

Robots Exclusion Standard data for craftself.com

Resource Scan

Scan Details

Site Domain craftself.com
Base Domain craftself.com
Scan Status Ok
Last Scan2026-04-04T21:18:47+00:00
Next Scan 2026-04-18T21:18:47+00:00

Last Scan

Scanned2026-04-04T21:18:47+00:00
URL https://craftself.com/robots.txt
Domain IPs 104.21.16.118, 172.67.212.11, 2606:4700:3031::6815:1076, 2606:4700:3033::ac43:d40b
Response IP 104.21.16.118
Found Yes
Hash 3df9ac608432503647446a327726a4964bafbd3af1390eb4e8638a1a7f73992b
SimHash 3214d8738293

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Allow /wp-content/uploads/
Allow /*.png$
Allow /*.jpg$
Allow /*.jpeg$
Allow /*.gif$
Allow /*.webp$
Allow /wp-content/themes/*/assets/css/
Allow /wp-content/themes/*/assets/js/
Disallow /cart/
Disallow /checkout/
Disallow /my-account/
Disallow /order-tracking/
Disallow /feed/
Disallow /trackback/
Disallow /add-to-cart%3D*
Disallow /*?add-to-cart=
Disallow /*%26add-to-cart%3D
Disallow /*?orderby=
Disallow /shop/?orderby=
Disallow /*?filter_
Disallow /*%26filter_
Disallow /*?rating_filter
Disallow /*?min_price=
Disallow /*?max_price=
Disallow /*%26min_price%3D
Disallow /*%26max_price%3D
Disallow /product-category/*page/*
Disallow /product-tag/*page/*
Disallow /wc-api/*
Disallow /?s=
Disallow /?replytocom=
Disallow /?product_cat=0&s=
Disallow /*.php$
Disallow /*.js$
Disallow /*.css$

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

netsprintbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

openai

Rule Path
Disallow /

anthropic

Rule Path
Disallow /

llmbot

Rule Path
Disallow /

bard

Rule Path
Disallow /

crawlera

Rule Path
Disallow /

transformerbot

Rule Path
Disallow /

huggingface

Rule Path
Disallow /

aicrawler

Rule Path
Disallow /

datacollectorbot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

nlpbot

Rule Path
Disallow /

deeplearningbot

Rule Path
Disallow /

webdatabot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://craftself.com/sitemap_index.xml

Comments

  • ====================

Warnings

  • 2 invalid lines.