bobsredmill.com
robots.txt

Robots Exclusion Standard data for bobsredmill.com

Resource Scan

Scan Details

Site Domain bobsredmill.com
Base Domain bobsredmill.com
Scan Status Ok
Last Scan2025-07-05T16:15:37+00:00
Next Scan 2025-08-04T16:15:37+00:00

Last Scan

Scanned2025-07-05T16:15:37+00:00
URL https://bobsredmill.com/robots.txt
Redirect https://www.bobsredmill.com/robots.txt
Redirect Domain www.bobsredmill.com
Redirect Base bobsredmill.com
Domain IPs 94.247.142.1
Redirect IPs 94.247.142.1
Response IP 94.247.142.1
Found Yes
Hash 94ddfd0f3e2a38959013c88a2866b09e1fbb563ce3a5cd8ab90c9bdba614719d
SimHash 6a700d7d7682

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/
Disallow /search
Disallow /recipes/how-to-make
Disallow /articles/explore
Disallow /articles/explore/featured-articles
Disallow /articles/explore/healthy-living
Disallow /articles/explore/special-diets
Disallow /articles/explore/uncategorized
Disallow /articles/explore/recipes
Disallow /articles/explore/learning-center
Disallow /articles/explore/baking-101
Disallow /articles/explore/travel
Disallow /articles/explore/bobs-by-the-book
Disallow /articles/explore/sustainability
Disallow /recipes/how-to-make/category
Disallow /recipes/how-to-make/breakfast-and-brunch
Disallow /recipes/how-to-make/biscuits-and-scones
Disallow /recipes/how-to-make/granola-and-breakfast-bars
Disallow /recipes/how-to-make/muffins
Disallow /recipes/how-to-make/oatmeals-cereals-granolas
Disallow /recipes/how-to-make/pancakes-and-waffles
Disallow /recipes/how-to-make/quick-breads
Disallow /recipes/how-to-make/breakfast-sweets
Disallow /recipes/how-to-make/smoothies
Disallow /recipes/how-to-make/oats-baked-goods
Disallow /recipes/how-to-make/protein-oats
Disallow /recipes/how-to-make/oats-overnight
Disallow /recipes/how-to-make/sweets
Disallow /recipes/how-to-make/brownies-and-dessert-bars
Disallow /recipes/how-to-make/cakes-and-cupcakes
Disallow /recipes/how-to-make/cookies
Disallow /recipes/how-to-make/desserts-and-pastries
Disallow /recipes/how-to-make/pies-and-pie-crusts
Disallow /recipes/how-to-make/quick-cooking-rolled-oats
Disallow /recipes/how-to-make/holiday-pies
Disallow /recipes/how-to-make/holiday-soups
Disallow /recipes/how-to-make/hanukkah
Disallow /recipes/how-to-make/stuffing
Disallow /recipes/how-to-make/snack-cookies
Disallow /recipes/how-to-make/summer-salads
Disallow /recipes/how-to-make/fresh-and-fruity-desserts
Disallow /recipes/how-to-make/smores
Disallow /recipes/how-to-make/savory-breakfast
Disallow /recipes/how-to-make/breads
Disallow /recipes/how-to-make/rolls-and-buns
Disallow /recipes/how-to-make/tortillas-and-wraps
Disallow /recipes/how-to-make/yeasted-breads
Disallow /recipes/how-to-make/baking-mix-savory
Disallow /recipes/how-to-make/signature-blends-baking-mixes
Disallow /recipes/how-to-make/side-dishes
Disallow /recipes/how-to-make/entrees
Disallow /recipes/how-to-make/burgers
Disallow /recipes/how-to-make/meatless-options
Disallow /recipes/how-to-make/pastas-and-noodles
Disallow /recipes/how-to-make/pizzas-and-flatbreads
Disallow /recipes/how-to-make/baking-mix-sweet
Disallow /recipes/how-to-make/holiday-brunch
Disallow /recipes/how-to-make/savory-oats
Disallow /recipes/how-to-make/oats-hot-cereal
Disallow /recipes/how-to-make/basic-cooking-and-instructions
Disallow /recipes/how-to-make/on-the-package
Disallow /recipes/how-to-make/all-purpose-flour
Disallow /recipes/how-to-make/oats-ideas
Disallow /recipes/how-to-make/1-to-1-ideas
Disallow /recipes/how-to-make/holiday-desserts
Disallow /recipes/how-to-make/holiday-ap-flour
Disallow /recipes/how-to-make/holiday-cookies
Disallow /recipes/how-to-make/pizza
Disallow /recipes/how-to-make/00-flour
Disallow /recipes/how-to-make/holiday-sides
Disallow /recipes/how-to-make/almond-flour
Disallow /recipes/how-to-make/spooky-treats
Disallow /recipes/how-to-make/gf-holiday-desserts
Disallow /recipes/how-to-make/holiday-gf-1-to-1
Disallow /recipes/how-to-make/holiday-almond-flour
Disallow /recipes/how-to-make/dinner-party
Disallow /recipes/how-to-make/fall-desserts
Disallow /recipes/how-to-make/appetizers
Disallow /recipes/how-to-make/salads
Disallow /recipes/how-to-make/spring-celebrations
Disallow /recipes/how-to-make/granola-muesli
Disallow /recipes/how-to-make/steel-cut-oats
Disallow /recipes/how-to-make/pasta
Disallow /recipes/how-to-make/gf-holiday-sides
Disallow /recipes/how-to-make/holiday-gf-mixes
Disallow /recipes/how-to-make/brownies-bars
Disallow /recipes/how-to-make/appetizers-2
Disallow /recipes/how-to-make/crackers
Disallow /recipes/how-to-make/dips-sauces-spreads
Disallow /recipes/how-to-make/holiday-cassava-flour
Disallow /recipes/how-to-make/gf-holiday-bread-and-rolls
Disallow /recipes/how-to-make/artisan-bread-flour
Disallow /recipes/how-to-make/summer-grilling
Disallow /recipes/how-to-make/after-school-snacks
Disallow /recipes/how-to-make/stay-cool
Disallow /recipes/how-to-make/grain-bowls
Disallow /recipes/how-to-make/oats-cookies
Disallow /recipes/how-to-make/cassava-ideas
Disallow /recipes/how-to-make/smoothies-and-beverages
Disallow /recipes/how-to-make/smoothies-2
Disallow /recipes/how-to-make/soups-and-stews
Disallow /recipes/how-to-make/gf-holiday-soups
Disallow /recipes/how-to-make/valentines
Disallow /recipes/how-to-make/brunch
Disallow /recipes/how-to-make/pet-treats-making
Disallow /recipes/how-to-make/whole-wheat-flour
Disallow /recipes/how-to-make/pet-treats
Disallow /recipes/how-to-make/holiday-breads-rolls
Disallow /recipes/how-to-make/gingerbread
Disallow /recipes/how-to-make/see-whats-new
Disallow /recipes/how-to-make/bob-charlees-recipe-box
Disallow /recipes/how-to-make/simple-almond-flour-recipes
Disallow /recipes/how-to-make/simple-oats-recipes
Disallow /recipes/how-to-make/dietary-needs
Disallow /recipes/how-to-make/gluten-free
Disallow /recipes/how-to-make/dairy-lactose-free
Disallow /recipes/how-to-make/soy-free
Disallow /recipes/how-to-make/vegan
Disallow /recipes/how-to-make/vegetarian
Disallow /recipes/how-to-make/paleo
Disallow /recipes/how-to-make/grain-free
Disallow /recipes/how-to-make/nut-free
Disallow /recipes/how-to-make/keto
Disallow /recipes/how-to-make/occasion
Disallow /recipes/how-to-make/christmas
Disallow /recipes/how-to-make/easter
Disallow /recipes/how-to-make/fourth-of-july
Disallow /recipes/how-to-make/halloween
Disallow /recipes/how-to-make/new-years-eve
Disallow /recipes/how-to-make/thanksgiving
Disallow /recipes/how-to-make/rosh-hashanahoryom-kippur
Disallow /recipes/how-to-make/passover
Disallow /recipes/how-to-make/appliance
Disallow /recipes/how-to-make/bread-machine
Disallow /recipes/how-to-make/microwave
Disallow /recipes/how-to-make/slow-cooker
Disallow /recipes/how-to-make/pressure-cooker
Disallow /recipes/how-to-make/multi-cooker
Disallow /recipes/how-to-make/instant-pot
Disallow /recipes/how-to-make/season
Disallow /recipes/how-to-make/spring
Disallow /recipes/how-to-make/winter
Disallow /recipes/how-to-make/fall
Disallow /recipes/how-to-make/summer
Disallow /recipes/how-to-make/st-patricks-day
Disallow /recipes/how-to-make/baked-oats
Disallow /recipes/how-to-make/crunch-connoisseur
Disallow /recipes/how-to-make/good-health-guru
Disallow /recipes/how-to-make/high-fiber-2
Disallow /recipes/how-to-make/spicy-superfan
Disallow /recipes/how-to-make/protein-powerhouse
Disallow /recipes/how-to-make/v1-sweetness-savorer
Disallow /recipes/how-to-make/sweetness-savorer
Disallow /recipes/how-to-make/bars
Disallow /recipes/how-to-make/simplicity-seeker
Disallow /recipes/how-to-make/new-year-oats-recipes-2024
Disallow /recipes/how-to-make/v1-gf-holiday-desserts-2
Disallow /recipes/how-to-make/grab-and-go
Disallow /recipes/how-to-make/outdoor-adventures
Disallow /recipes/how-to-make/easy-breakfasts

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.bobsredmill.com/sitemaps-1-sitemap.xml
sitemap https://www.bobsredmill.com/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.bobsredmill.com/
  • live - don't allow web crawlers to index cpresources/ or vendor/
  • Disallow ChatGPT bot, as there's no benefit to allowing it to index your site
  • Disallow Google Bard and Vertex AI bots, as there's no benefit to allowing it to index your site
  • Disallow Perplexity bot, as there's no benefit to allowing it to index your site