Log In Sign Up

keurig.com
robots.txt

Robots Exclusion Standard data for keurig.com

Resource Scan

Scan Details

Site Domain keurig.com
Base Domain keurig.com
Scan Status Ok
Last Scan2024-10-28T10:02:21+00:00
Next Scan 2024-11-27T10:02:21+00:00

Last Scan

Scanned2024-10-28T10:02:21+00:00
URL https://www.keurig.com/robots.txt
Domain IPs 125.252.229.74
Response IP 125.252.229.74
Found Yes
Hash b593db39b6c791762ede0942d0c8064e4ab5be54d50509d253fd33d712fc2ff7
SimHash 25950e38eeb2

Groups

*

Rule Path
Disallow *brand%3A*
Disallow categoryCode%3D*
Disallow *%3Arelevance*
Disallow sort%3Dpopular-rec*
Disallow sort%3Dprice-asc*
Disallow sort%3Dprice-desc*
Disallow sort%3Dname-asc*
Disallow sort%3Dname-desc*
Disallow sort%3Dbrand-asc*
Disallow sort%3Dbrand-desc*
Disallow /getProductOOSRecommendation?*
Disallow /cart
Disallow /checkout
Disallow /my-account
Disallow /*/ajaxCategoryPage?*
Disallow /getSessionCartGuid
Disallow /replenishment/
Disallow /membership/

googlebot-image

Rule Path
Disallow

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

*

Rule Path
Disallow /b/Keurig-K475-Coffee-Maker-Starter-Bundle
Disallow /janship20
Disallow /beverages/perfect-cup/c/
Disallow /write-power-review
Disallow /powerreviews
Disallow relevance

ahrefsbot

Rule Path
Disallow /write-power-review
Disallow /*OfficeSize%3A*
Disallow /*Deals%3A*
Disallow /*BrewerWaterSource%3A*
Disallow /*features%3A*
Disallow /*ReservoirCapacity%3A*
Disallow /*MaxBrewCupSize%3A*
Disallow /*categoryCode%3D*
Disallow /*sort%3Dpopular-rec*
Disallow /*sort%3Dprice-asc*
Disallow /*sort%3Dprice-desc*
Disallow /*sort%3Dname-asc*
Disallow /*sort%3Dname-desc*
Disallow /*sort%3Dbrand-asc*
Disallow /*sort%3Dbrand-desc*
Disallow /*q%3D*%3A*%3A*
Disallow /*brand%3A*%2C*
Disallow /*boxCount%3A*%2C*
Disallow /*varieties%3A*%2C*
Disallow /*RoastType%3A*%2C*
Disallow /*BeverageType%3A*%2C*
Disallow /*blend%3A*%2C*
Disallow /*Caffeination%3A*%2C*
Disallow /*PackageType%3A*%2C*
Disallow /*flavor%3A*%2C*

bingbot

Rule Path
Disallow /write-power-review

yandexbot

Rule Path
Disallow /write-power-review

wotbox

Rule Path
Disallow /write-power-review

adsbot-google

Rule Path
Disallow /checkout/orderConfirmation
Back to top

Other Records

Field Value
sitemap https://www.keurig.com/sitemap.xml
Back to top

Comments

  • For all robots
  • Allow search crawlers to discover the sitemap
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot
  • Block targeted promo pages
  • 2015-10-15 added by Remi <remi@lesite.ca> for SEO purposes on k.com
  • end added by Remi <remi@lesite.ca> for SEO purposes on k.com
  • 01-08-2016 changes to block PR and OrderConfirmation calls
  • Block AhrefsBot power-review call
  • Disallowed Facet Choices
  • Disallow Muliple Facet Choices
  • Disallow Muliple Choice from Single Facet
  • Block bingbot power-review call
  • Block YandexBot power-review call
  • Block Wotbox power-review call
  • Block AdsBot-Google OrderConfirmation call
Back to top

Warnings

  • 15 invalid lines.
Back to top