secure.myrecipes.com
robots.txt

Robots Exclusion Standard data for secure.myrecipes.com

Resource Scan

Scan Details

Site Domain secure.myrecipes.com
Base Domain myrecipes.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-15T22:07:13+00:00
Next Scan 2024-12-14T22:07:13+00:00

Last Successful Scan

Scanned2023-01-31T19:09:05+00:00
URL https://secure.myrecipes.com/robots.txt
Domain IPs 13.227.254.46, 13.227.254.55, 13.227.254.62, 13.227.254.64
Response IP 13.33.88.74
Found Yes
Hash ab6e04d0b24071c29bad46a5bdabaf3d7bf95c8dbfbb2374b78e9eceaf5cf355
SimHash 64c64295c414

Groups

mediapartners-google

Rule Path
Disallow

slurp

Rule Path
Disallow /*___itemUrl___
Disallow /*requestid%3D
Disallow /*.jhtml
Disallow /app/costumegallery/search/
Disallow /app/decoratinggallery/search/
Disallow /app/duedatepredictor/results.jsp
Disallow /app/familyfunfinder/results.jsp
Disallow /app/fertilitypredictor/results.jsp
Disallow /app/giftgallery/search/
Disallow /app/mycookbook/search/
Disallow /app/pediatrichealth/results.jsp
Disallow /app/plantgallery/search/
Disallow /app/product-recalls/results.jsp
Disallow /app/qa/common/searchResults
Disallow /app/scrapbookexchange/searchResults.jsp
Disallow /app/sharemy/invokeSearchForDetailsPage.jsp
Disallow /app/sharemy/searchResults.jsp
Disallow /app/stayathomecalculator/results.jsp
Disallow /app/toygallery/search/
Disallow /app/travelfinder/results.jsp
Disallow /betterrecipes/search.jsp
Disallow /bhg/archives/search/searchResults.jsp
Disallow /bhg/betterrecipes/searchResults.jsp
Disallow /bhg/css/search
Disallow /bhg/decoratinggallery/search
Disallow /bhg/dgroups/search
Disallow /bhg/file.jsp?item=Search
Disallow /bhg/plantfinder/search
Disallow /bhg/printableStory
Disallow /bhg/recipe/search
Disallow /bhg/requestForInformation/
Disallow /bhg/search/results.jsp
Disallow /bhg/store/cart/
Disallow /bhg/store/search
Disallow /dlv/search/results.jsp
Disallow /fitness/search/results.jsp
Disallow /lcl/search/results.jsp
Disallow /lhj/recipe/search
Disallow /lhj/search/results.jsp
Disallow /more/printableStory
Disallow /more/search/results.jsp
Disallow /mwl/search/results.jsp
Disallow /parents/search/results.jsp
Disallow /partner/
Disallow /partners/
Disallow /product-recalls/search/
Disallow /recipes/mealideas/searchResults.jsp
Disallow /recipes/search
Disallow /recipe/search
Disallow /recipecom/search
Disallow /remodel/search/results.jsp
Disallow /scrapbookexchange/searchResults.jsp
Disallow /templatedata/
Disallow /wood/searchResults.jsp
Disallow /common/
Disallow /bhg/store/checkout/
Disallow /bhg/profile/
Disallow /home/
Disallow /gardening/plant-dictionary/search/
Disallow /app/plantencyclopedia/searchResults.jsp
Disallow /child/
Disallow /app/sharemy/invokeSearchForDetailsPage.jsp
Disallow /app/sharemy/searchResults.jsp
Disallow /diy/search/results.jsp
Disallow /app/advice/searchResults.jsp
Disallow /app/advice/userSearch.jsp
Disallow /app/advice/myQA.jsp
Disallow /app/advice/browseQuestions.jsp
Disallow /app/advice/question.jsp

yahoo-mmcrawler

Rule Path
Disallow /*___itemUrl___
Disallow /*requestid%3D
Disallow /*.jhtml
Disallow /app/costumegallery/search/
Disallow /app/decoratinggallery/search/
Disallow /app/duedatepredictor/results.jsp
Disallow /app/familyfunfinder/results.jsp
Disallow /app/fertilitypredictor/results.jsp
Disallow /app/giftgallery/search/
Disallow /app/mycookbook/search/
Disallow /app/pediatrichealth/results.jsp
Disallow /app/plantgallery/search/
Disallow /app/product-recalls/results.jsp
Disallow /app/qa/common/searchResults
Disallow /app/scrapbookexchange/searchResults.jsp
Disallow /app/sharemy/invokeSearchForDetailsPage.jsp
Disallow /app/sharemy/searchResults.jsp
Disallow /app/stayathomecalculator/results.jsp
Disallow /app/toygallery/search/
Disallow /app/travelfinder/results.jsp
Disallow /betterrecipes/search.jsp
Disallow /bhg/archives/search/searchResults.jsp
Disallow /bhg/betterrecipes/searchResults.jsp
Disallow /bhg/css/search
Disallow /bhg/decoratinggallery/search
Disallow /bhg/dgroups/search
Disallow /bhg/file.jsp?item=Search
Disallow /bhg/plantfinder/search
Disallow /bhg/printableStory
Disallow /bhg/recipe/search
Disallow /bhg/requestForInformation/
Disallow /bhg/search/results.jsp
Disallow /bhg/store/cart/
Disallow /bhg/store/search
Disallow /dlv/search/results.jsp
Disallow /fitness/search/results.jsp
Disallow /lcl/search/results.jsp
Disallow /lhj/recipe/search
Disallow /lhj/search/results.jsp
Disallow /more/printableStory
Disallow /more/search/results.jsp
Disallow /mwl/search/results.jsp
Disallow /parents/search/results.jsp
Disallow /partner/
Disallow /partners/
Disallow /product-recalls/search/
Disallow /recipes/mealideas/searchResults.jsp
Disallow /recipes/search
Disallow /remodel/search/results.jsp
Disallow /scrapbookexchange/searchResults.jsp
Disallow /templatedata/
Disallow /wood/searchResults.jsp
Disallow /common/
Disallow /bhg/store/checkout/
Disallow /bhg/profile/
Disallow /home/
Disallow /gardening/plant-dictionary/search/
Disallow /app/plantencyclopedia/searchResults.jsp
Disallow /child/
Disallow /app/advice/searchResults.jsp
Disallow /app/advice/userSearch.jsp
Disallow /app/advice/myQA.jsp
Disallow /app/advice/browseQuestions.jsp
Disallow /app/advice/question.jsp

yahooseeker/m1a1-r2d2

Rule Path
Disallow /*___itemUrl___
Disallow /*requestid%3D
Disallow /*.jhtml
Disallow /app/costumegallery/search/
Disallow /app/decoratinggallery/search/
Disallow /app/duedatepredictor/results.jsp
Disallow /app/familyfunfinder/results.jsp
Disallow /app/fertilitypredictor/results.jsp
Disallow /app/giftgallery/search/
Disallow /app/mycookbook/search/
Disallow /app/pediatrichealth/results.jsp
Disallow /app/plantgallery/search/
Disallow /app/product-recalls/results.jsp
Disallow /app/qa/common/searchResults
Disallow /app/scrapbookexchange/searchResults.jsp
Disallow /app/sharemy/invokeSearchForDetailsPage.jsp
Disallow /app/sharemy/searchResults.jsp
Disallow /app/stayathomecalculator/results.jsp
Disallow /app/toygallery/search/
Disallow /app/travelfinder/results.jsp
Disallow /betterrecipes/search.jsp
Disallow /bhg/archives/search/searchResults.jsp
Disallow /bhg/betterrecipes/searchResults.jsp
Disallow /bhg/css/search
Disallow /bhg/decoratinggallery/search
Disallow /bhg/dgroups/search
Disallow /bhg/file.jsp?item=Search
Disallow /bhg/plantfinder/search
Disallow /bhg/printableStory
Disallow /bhg/recipe/search
Disallow /bhg/requestForInformation/
Disallow /bhg/search/results.jsp
Disallow /bhg/store/cart/
Disallow /bhg/store/search
Disallow /dlv/search/results.jsp
Disallow /fitness/search/results.jsp
Disallow /lcl/search/results.jsp
Disallow /lhj/recipe/search
Disallow /lhj/search/results.jsp
Disallow /more/printableStory
Disallow /more/search/results.jsp
Disallow /mwl/search/results.jsp
Disallow /parents/search/results.jsp
Disallow /partner/
Disallow /partners/
Disallow /product-recalls/search/
Disallow /recipes/mealideas/searchResults.jsp
Disallow /recipes/search
Disallow /remodel/search/results.jsp
Disallow /scrapbookexchange/searchResults.jsp
Disallow /templatedata/
Disallow /wood/searchResults.jsp
Disallow /common/
Disallow /bhg/store/checkout/
Disallow /bhg/profile/
Disallow /home/
Disallow /gardening/plant-dictionary/search/
Disallow /app/plantencyclopedia/searchResults.jsp
Disallow /child/
Disallow /app/advice/searchResults.jsp
Disallow /app/advice/userSearch.jsp
Disallow /app/advice/myQA.jsp
Disallow /app/advice/browseQuestions.jsp
Disallow /app/advice/question.jsp

yahoo-newscrawler

Rule Path
Disallow /*___itemUrl___
Disallow /*requestid%3D
Disallow /*.jhtml
Disallow /app/costumegallery/search/
Disallow /app/decoratinggallery/search/
Disallow /app/duedatepredictor/results.jsp
Disallow /app/familyfunfinder/results.jsp
Disallow /app/fertilitypredictor/results.jsp
Disallow /app/giftgallery/search/
Disallow /app/mycookbook/search/
Disallow /app/pediatrichealth/results.jsp
Disallow /app/plantgallery/search/
Disallow /app/product-recalls/results.jsp
Disallow /app/qa/common/searchResults
Disallow /app/scrapbookexchange/searchResults.jsp
Disallow /app/sharemy/invokeSearchForDetailsPage.jsp
Disallow /app/sharemy/searchResults.jsp
Disallow /app/stayathomecalculator/results.jsp
Disallow /app/toygallery/search/
Disallow /app/travelfinder/results.jsp
Disallow /betterrecipes/search.jsp
Disallow /bhg/archives/search/searchResults.jsp
Disallow /bhg/betterrecipes/searchResults.jsp
Disallow /bhg/css/search
Disallow /bhg/decoratinggallery/search
Disallow /bhg/dgroups/search
Disallow /bhg/file.jsp?item=Search
Disallow /bhg/plantfinder/search
Disallow /bhg/printableStory
Disallow /bhg/recipe/search
Disallow /bhg/requestForInformation/
Disallow /bhg/search/results.jsp
Disallow /bhg/store/cart/
Disallow /bhg/store/search
Disallow /dlv/search/results.jsp
Disallow /fitness/search/results.jsp
Disallow /lcl/search/results.jsp
Disallow /lhj/recipe/search
Disallow /lhj/search/results.jsp
Disallow /more/printableStory
Disallow /more/search/results.jsp
Disallow /mwl/search/results.jsp
Disallow /parents/search/results.jsp
Disallow /partner/
Disallow /partners/
Disallow /product-recalls/search/
Disallow /recipes/mealideas/searchResults.jsp
Disallow /recipes/search
Disallow /remodel/search/results.jsp
Disallow /scrapbookexchange/searchResults.jsp
Disallow /templatedata/
Disallow /wood/searchResults.jsp
Disallow /common/
Disallow /bhg/store/checkout/
Disallow /bhg/profile/
Disallow /home/
Disallow /gardening/plant-dictionary/search/
Disallow /app/plantencyclopedia/searchResults.jsp
Disallow /child/
Disallow /app/sharemy/invokeSearchForDetailsPage.jsp
Disallow /app/sharemy/searchResults.jsp
Disallow /diy/search/results.jsp
Disallow /app/advice/searchResults.jsp
Disallow /app/advice/userSearch.jsp
Disallow /app/advice/myQA.jsp
Disallow /app/advice/browseQuestions.jsp
Disallow /app/advice/question.jsp

yahoo-mmaudvid

Rule Path
Disallow /*___itemUrl___
Disallow /*requestid%3D
Disallow /*.jhtml
Disallow /app/costumegallery/search/
Disallow /app/decoratinggallery/search/
Disallow /app/duedatepredictor/results.jsp
Disallow /app/familyfunfinder/results.jsp
Disallow /app/fertilitypredictor/results.jsp
Disallow /app/giftgallery/search/
Disallow /app/mycookbook/search/
Disallow /app/pediatrichealth/results.jsp
Disallow /app/plantgallery/search/
Disallow /app/product-recalls/results.jsp
Disallow /app/qa/common/searchResults
Disallow /app/scrapbookexchange/searchResults.jsp
Disallow /app/sharemy/invokeSearchForDetailsPage.jsp
Disallow /app/sharemy/searchResults.jsp
Disallow /app/stayathomecalculator/results.jsp
Disallow /app/toygallery/search/
Disallow /app/travelfinder/results.jsp
Disallow /betterrecipes/search.jsp
Disallow /bhg/archives/search/searchResults.jsp
Disallow /bhg/betterrecipes/searchResults.jsp
Disallow /bhg/css/search
Disallow /bhg/decoratinggallery/search
Disallow /bhg/dgroups/search
Disallow /bhg/file.jsp?item=Search
Disallow /bhg/plantfinder/search
Disallow /bhg/printableStory
Disallow /bhg/recipe/search
Disallow /bhg/requestForInformation/
Disallow /bhg/search/results.jsp
Disallow /bhg/store/cart/
Disallow /bhg/store/search
Disallow /dlv/search/results.jsp
Disallow /fitness/search/results.jsp
Disallow /lcl/search/results.jsp
Disallow /lhj/recipe/search
Disallow /lhj/search/results.jsp
Disallow /more/printableStory
Disallow /more/search/results.jsp
Disallow /mwl/search/results.jsp
Disallow /parents/search/results.jsp
Disallow /partner/
Disallow /partners/
Disallow /product-recalls/search/
Disallow /recipes/mealideas/searchResults.jsp
Disallow /recipes/search
Disallow /remodel/search/results.jsp
Disallow /scrapbookexchange/searchResults.jsp
Disallow /templatedata/
Disallow /wood/searchResults.jsp
Disallow /common/
Disallow /bhg/store/checkout/
Disallow /bhg/profile/
Disallow /home/
Disallow /gardening/plant-dictionary/search/
Disallow /app/plantencyclopedia/searchResults.jsp
Disallow /child/
Disallow /videos/topics/
Disallow /app/sharemy/invokeSearchForDetailsPage.jsp
Disallow /app/sharemy/searchResults.jsp
Disallow /diy/search/results.jsp
Disallow /app/advice/searchResults.jsp
Disallow /app/advice/userSearch.jsp
Disallow /app/advice/myQA.jsp
Disallow /app/advice/browseQuestions.jsp
Disallow /app/advice/question.jsp

*

Rule Path
Disallow /*___itemUrl___
Disallow /*.jhtml
Disallow /app/costumegallery/search/
Disallow /app/decoratinggallery/search/
Disallow /app/duedatepredictor/results.jsp
Disallow /app/familyfunfinder/results.jsp
Disallow /app/fertilitypredictor/results.jsp
Disallow /app/giftgallery/search/
Disallow /app/mycookbook/search/
Disallow /app/pediatrichealth/results.jsp
Disallow /app/plantgallery/search/
Disallow /app/product-recalls/results.jsp
Disallow /app/qa/common/searchResults
Disallow /app/scrapbookexchange/searchResults.jsp
Disallow /app/sharemy/invokeSearchForDetailsPage.jsp
Disallow /app/sharemy/searchResults.jsp
Disallow /app/stayathomecalculator/results.jsp
Disallow /app/toygallery/search/
Disallow /app/travelfinder/results.jsp
Disallow /betterrecipes/search.jsp
Disallow /bhg/archives/search/searchResults.jsp
Disallow /bhg/betterrecipes/searchResults.jsp
Disallow /bhg/css/search
Disallow /bhg/decoratinggallery/search
Disallow /bhg/dgroups/search
Disallow /bhg/file.jsp?item=Search
Disallow /bhg/plantfinder/search
Disallow /bhg/printableStory
Disallow /bhg/recipe/search
Disallow /bhg/requestForInformation/
Disallow /bhg/search/results.jsp
Disallow /bhg/store/cart/
Disallow /bhg/store/search
Allow /content/dam/bhg/Images/
Disallow /content/
Disallow /cq/
Disallow /dlv/search/results.jsp
Disallow /fitness/search/results.jsp
Disallow /lcl/search/results.jsp
Disallow /lhj/recipe/search
Disallow /lhj/search/results.jsp
Disallow /more/printableStory
Disallow /more/search/results.jsp
Disallow /mwl/search/results.jsp
Disallow /parents/search/results.jsp
Disallow /partner/
Disallow /partners/
Disallow /product-recalls/search/
Disallow /recipes/mealideas/searchResults.jsp
Disallow /recipes/search
Disallow /remodel/search/results.jsp
Disallow /scrapbookexchange/searchResults.jsp
Disallow /templatedata/
Disallow /wood/searchResults.jsp
Disallow /common/
Disallow /bhg/store/checkout/
Disallow /bhg/profile/
Disallow /home/
Disallow /gardening/plant-dictionary/search/
Disallow /app/plantencyclopedia/searchResults.jsp
Disallow /child/
Disallow /app/sharemy/invokeSearchForDetailsPage.jsp
Disallow /app/sharemy/searchResults.jsp
Disallow /diy/search/results.jsp
Disallow /app/advice/searchResults.jsp
Disallow /app/advice/userSearch.jsp
Disallow /app/advice/myQA.jsp
Disallow /app/advice/browseQuestions.jsp
Disallow /app/advice/question.jsp
Disallow /mobile/search/results.jsp
Disallow /shop/search/
Disallow /community/kudos/
Disallow /community/user/
Disallow /order/
Disallow /labs/

adsbot-google

Rule Path
Disallow /order/checkout/
Disallow /order/billpay/
Disallow /order/profile/logout.jsp
Disallow /order/global/generalError.jsp
Disallow /order/checkout/oneClickError.jsp

bingbot

Rule Path
Disallow /order/checkout/
Disallow /order/billpay/
Disallow /order/profile/logout.jsp
Disallow /order/global/generalError.jsp
Disallow /order/checkout/oneClickError.jsp

adidxbot

Rule Path
Disallow /order/checkout/
Disallow /order/billpay/
Disallow /order/profile/logout.jsp
Disallow /order/global/generalError.jsp
Disallow /order/checkout/oneClickError.jsp

Comments

  • The general rules do not apply for the AdsBot. So we had to explicitly exclude CRT pages for the 3 User-agents AdsBot-Google,Bingbot,Adidxbot
  • Check the section on “Excluding your landing pages from review” - https://support.google.com/adwords/answer/2404197?hl=en