localharvest.org
robots.txt

Robots Exclusion Standard data for localharvest.org

Resource Scan

Scan Details

Site Domain localharvest.org
Base Domain localharvest.org
Scan Status Ok
Last Scan2024-06-18T16:29:12+00:00
Next Scan 2024-06-25T16:29:12+00:00

Last Scan

Scanned2024-06-18T16:29:12+00:00
URL https://localharvest.org/robots.txt
Redirect https://www.localharvest.org/robots.txt
Redirect Domain www.localharvest.org
Redirect Base localharvest.org
Domain IPs 139.177.201.237
Redirect IPs 139.177.201.237
Response IP 139.177.201.237
Found Yes
Hash 48f07fd7b3b149fb235e2346634a3ca9a207759f141617ec90b3c7d83a3f4cb6
SimHash 09545888c3f0

Groups

*

Rule Path
Disallow /ajax/
Disallow /members/
Disallow /maps/
Disallow /admin/
Disallow /feedback.jsp
Disallow /zipsearch.jsp
Disallow /search.jsp
Disallow /privacy.jsp
Disallow /postcomment.jsp
Disallow /comments.jsp
Disallow /keepmeposted.jsp
Disallow /inarea.jsp
Disallow /jackpot.jsp
Disallow /register.jsp
Disallow /register_conditions.jsp
Disallow /listing_email.jsp
Disallow /listing_events.jsp
Disallow /elink/
Disallow /album/
Disallow /legal.jsp
Disallow /store/basket.jsp
Disallow /store/busted.jsp
Disallow /store/myaccount.jsp
Disallow /store/item_email.jsp
Disallow /store/vendor_email.jsp
Disallow /store/giftcert.jsp
Disallow /store/giftcert_email.jsp
Disallow /store/getshipto.jsp
Disallow /store/notify-me.jsp
Disallow /store/zipcode_p.jsp
Disallow /features/cookies.jsp
Disallow /servlets/captcha.jpg
Disallow /farms/M286
Disallow /forum/search.jsp
Disallow /accounts/
Disallow /new-review.jsp
Disallow /takemeto.jsp
Disallow /new-comment.jsp
Disallow /forum/post.jsp
Disallow /events.jsp?zip=

adsbot-google

Rule Path
Disallow /

adsbot-google-mobile

Rule Path
Disallow /

mp3bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

yak

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Warnings

  • 4 invalid lines.