food52.com
robots.txt
Robots Exclusion Standard data for food52.com
Resource Scan
Scan Details
Site Domain | food52.com |
Base Domain | food52.com |
Scan Status | Ok |
Last Scan | 2024-09-21T08:55:58+00:00 |
Next Scan | 2024-09-28T08:55:58+00:00 |
Last Scan
Scanned | 2024-09-21T08:55:58+00:00 |
URL | https://food52.com/robots.txt |
Domain IPs | 104.17.112.170, 104.17.239.199, 2606:4700::6811:70aa, 2606:4700::6811:efc7 |
Response IP | 104.17.239.199 |
Found | Yes |
Hash | 27ce5efd3ae33534f9a1bd945671481447567500b318fb0e7d80afdfd72a3d5a |
SimHash | 6a0e98368723 |
Groups
*
Rule | Path |
---|---|
Disallow | *ajax_* |
Disallow | */increment_view_count |
Disallow | /*/edit/ |
Disallow | /admin/ |
Disallow | /answers/ |
Disallow | /api/v1/promo_banner_active/ |
Disallow | /log-event/ |
Disallow | /recipes/cooking/ |
Disallow | /recipes/print/ |
Disallow | /recipes/ingredient/ |
Disallow | /session/ |
Disallow | /sweeps/ |
Disallow | /foodpickle/ |
Disallow | /ad-testing |
Disallow | /shop/admin/ |
Disallow | /shop/merchantadmin/ |
Disallow | /shop/checkout/ |
Disallow | /shop/track-order |
Disallow | /shop/referrals/ |
Disallow | /shop/claimcredit |
Disallow | /shop/basket_items/ |
Disallow | /users/sign_in |
Disallow | /users/sign_out |
Disallow | /users/sign_up |
Disallow | /blog/search |
Disallow | /hotline/search |
Disallow | /recipes/search |
Disallow | /shop/search |
Disallow | /users/search |
Allow | /recipes/search?o=newest&tag=test-kitchen-approved |
Disallow | /pantry |
Allow | / |
Other Records
Field | Value |
---|---|
sitemap | https://food52.com/sitemap-index.xml |
Comments