recipezaar.com
robots.txt

Robots Exclusion Standard data for recipezaar.com

Resource Scan

Scan Details

Site Domain recipezaar.com
Base Domain recipezaar.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-10-05T22:02:56+00:00
Next Scan 2025-01-03T22:02:56+00:00

Last Successful Scan

Scanned2022-06-05T00:42:12+00:00
URL http://recipezaar.com/robots.txt
Redirect https://www.food.com/robots.txt
Redirect Domain www.food.com
Redirect Base food.com
Response IP 23.36.48.71
Found Yes
Hash 0cef92a6f155585fb4f2ac00a20f570a8fbc8b0895db27ccc6ecc7ea59be4d62
SimHash 402258c64539

Groups

*

Rule Path
Disallow /recipeprint.do*
Disallow /rssapi.do*
Disallow /bb/
Disallow /null/
Disallow /members/subm/
Disallow /mail/compose*
Disallow /cp_asset_stats.do
Disallow /dpn.do*
Disallow /mobile/api/
Disallow /ingredient-finder/
Disallow /cookbook-finder/
Disallow /menu-finder/
Disallow /recipe-finder/
Disallow /photo-finder/
Disallow /member-finder/
Disallow /search/
Disallow /books/
Disallow /community/
Disallow *.zsp*
Disallow *.php*
Disallow /~asset*

mediapartners-google

Rule Path
Disallow

googlebot-news

Rule Path
Allow /news
Allow /article
Disallow /

Other Records

Field Value
sitemap https://www.food.com/sitemap.xml
sitemap https://www.food.com/news/news-sitemap.xml

Comments

  • Directives for all crawlers
  • Disallow: /about/tour/
  • Disallow: /admin/
  • Disallow: /alt/
  • Disallow: /mail/
  • Disallow: /members/photos.php
  • Disallow: /recipe/photo.php
  • Disallow: /void/
  • Disallow: /services/
  • Exclude page construction elements (commented out some due to new google guidelines)
  • Disallow: /pushdownAd.do
  • Disallow: /global/header.jsp
  • Disallow: /global/pickle-header.jsp
  • Disallow: /footer.jsp
  • Disallow: /headerbreadcrumbs.do
  • Disallow: /getrzuserstats.do
  • Disallow: /seasonal-header.do
  • Exclude authenticated pages
  • Exclude search results pages
  • Block invalid Omniture injected URLs within javascript
  • Disallow: /*_1
  • Disallow: /*_2
  • Disallow: /*_3
  • Disallow: /*_4
  • Disallow: /*_5
  • Disallow: /*_6
  • Disallow: /*_7
  • Disallow: /*_8
  • Block .php & .zsp files
  • Directives for Mediapartners-Google
  • Directives for GoogleBot News
  • Directives for 008
  • Sitemap

Warnings

  • 2 invalid lines.