Log In Sign Up

auto-api.yelp.com
robots.txt

Robots Exclusion Standard data for auto-api.yelp.com

Resource Scan

Scan Details

Site Domain auto-api.yelp.com
Base Domain yelp.com
Scan Status Ok
Last Scan2024-05-12T00:17:39+00:00
Next Scan 2024-06-11T00:17:39+00:00

Last Scan

Scanned2024-05-12T00:17:39+00:00
URL https://auto-api.yelp.com/robots.txt
Domain IPs 151.101.0.116, 151.101.128.116, 151.101.192.116, 151.101.64.116
Response IP 199.232.44.116
Found Yes
Hash 62ac263feed2c4d1f23b857785a68627747dc6be394610f58adfc98f56b3c0cf
SimHash 3c1777116fdf

Groups

googlebot

Rule Path
Allow /

adsbot-google
applebot
bingpreview
google-inspectiontool
googlebot-image
googlebot-mobile
googlebot-video
mediapartners-google
stc-bot
scoutjet
seznambot
telegrambot
twitterbot
yahoo! slurp
yandex
archive.org_bot
bingbot
ia_archiver
msnbot
www.similar.ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

*

Rule Path
Disallow /
Back to top

Comments

  • By accessing Yelp's website (© 2024) you agree to Yelp's Terms of Service, available at
  • https://www.yelp.com/static?country=US&p=tos
  • If you would like to inquire about crawling Yelp, please contact us at
  • https://www.yelp.com/contact
  • As always, Asimov's Three Laws are in effect:
  • 1. A robot may not injure a human being or, through inaction, allow a human
  • being to come to harm.
  • 2. A robot must obey orders given it by human beings except where such
  • orders would conflict with the First Law.
  • 3. A robot must protect its own existence as long as such protection does
  • not conflict with the First or Second Law.
Back to top