web-website-spa-production.production.yummly.com
robots.txt

Resource Scan

Scan Details

Site Domain web-website-spa-production.production.yummly.com
Base Domain yummly.com
Scan Status Ok
Last Scan2024-06-06T17:09:28+00:00
Next Scan 2024-06-20T17:09:28+00:00

Last Scan

Scanned2024-06-06T17:09:28+00:00
URL https://web-website-spa-production.production.yummly.com/robots.txt
Domain IPs 18.233.212.92, 3.212.205.109, 3.222.220.173, 54.221.66.186
Response IP 3.212.205.109
Found Yes
Hash 787208018cf8e4ddfd4dde72b264e55d07362928adfdfe2ca176d168ab87e1ba
SimHash ed9c58f1d770

Groups

*

Rule Path
Disallow /email/
Disallow /login
Disallow /profile/
Disallow /discuss/
Disallow /private/
Disallow /article/
Disallow /video/

googlebot

Rule Path
Disallow /urb/
Disallow /services/
Disallow /email/
Disallow /login
Disallow /profile/
Disallow /discuss/
Disallow /private/
Disallow /article/
Disallow /video/

bingbot

Rule Path
Disallow /urb/
Disallow /services/
Disallow /email/
Disallow /login
Disallow /profile/
Disallow /discuss/
Disallow /private/
Disallow /article/
Disallow /video/

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

bhc.collectionbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.yummly.com/yummly-all-pages-us.xml

Comments

  • Misbehaving bot