yummly.com
robots.txt

Robots Exclusion Standard data for yummly.com

Resource Scan

Scan Details

Site Domain yummly.com
Base Domain yummly.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-21T17:19:20+00:00
Next Scan 2024-05-28T17:19:20+00:00

Last Successful Scan

Scanned2023-04-07T13:19:48+00:00
URL https://yummly.com/robots.txt
Redirect https://www.yummly.com/robots.txt
Redirect Domain www.yummly.com
Redirect Base yummly.com
Domain IPs 18.155.68.59, 18.155.68.6, 18.155.68.76, 18.155.68.85
Redirect IPs 104.18.26.71, 104.18.27.71, 2606:4700::6812:1a47, 2606:4700::6812:1b47
Response IP 104.18.26.71
Found Yes
Hash 7e1bcc16607b09bfceb69be9549139a844561e8b22c862b2e01a54bb272ba3bd
SimHash e59c58e1d752

Groups

*

Rule Path
Disallow /email/
Disallow /login
Disallow /profile/
Disallow /discuss/
Disallow /private/
Disallow /article/
Disallow /video/

googlebot

Rule Path
Disallow /urb/
Disallow /services/
Disallow /email/
Disallow /login
Disallow /profile/
Disallow /discuss/
Disallow /private/
Disallow /article/
Disallow /video/

bingbot

Rule Path
Disallow /urb/
Disallow /services/
Disallow /email/
Disallow /login
Disallow /profile/
Disallow /discuss/
Disallow /private/
Disallow /article/
Disallow /video/

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.2

bhc.collectionbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.yummly.com/yummly-all-pages-us.xml

Comments

  • Misbehaving bot