Log In Sign Up

yelp.nl
robots.txt

Robots Exclusion Standard data for yelp.nl

Resource Scan

Scan Details

Site Domain yelp.nl
Base Domain yelp.nl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-17T16:39:50+00:00
Next Scan 2024-06-16T16:39:50+00:00

Last Successful Scan

Scanned2024-04-11T16:31:44+00:00
URL https://yelp.nl/robots.txt
Redirect https://www.yelp.nl/robots.txt
Redirect Domain www.yelp.nl
Redirect Base yelp.nl
Domain IPs 108.156.133.101, 108.156.133.104, 108.156.133.75, 108.156.133.98
Redirect IPs 108.156.133.101, 108.156.133.104, 108.156.133.75, 108.156.133.98
Response IP 108.156.133.98
Found Yes
Hash 1f4c871457b2604c690c06403dee44a861f518fd3673f207254a4fbc93c80510
SimHash ad173531e75e

Groups

*
bingbot
adsbot-google
adsbot-google-mobile
adsbot-google-mobile-apps
mediapartners-google

Rule Path
Disallow /
Back to top

Comments

  • By accessing Yelp's website (© 2024) you agree to Yelp's Terms of Service, available at
  • https://www.yelp.com/static?country=US&p=tos
  • If you would like to inquire about crawling Yelp, please contact us at
  • https://www.yelp.com/contact
  • As always, Asimov's Three Laws are in effect:
  • 1. A robot may not injure a human being or, through inaction, allow a human
  • being to come to harm.
  • 2. A robot must obey orders given it by human beings except where such
  • orders would conflict with the First Law.
  • 3. A robot must protect its own existence as long as such protection does
  • not conflict with the First or Second Law.
Back to top