Log In Sign Up

libroselectronicos.cervantes.es
robots.txt

Robots Exclusion Standard data for libroselectronicos.cervantes.es

Resource Scan

Scan Details

Site Domain libroselectronicos.cervantes.es
Base Domain cervantes.es
Scan Status Ok
Last Scan2025-05-15T23:50:28+00:00
Next Scan 2025-06-14T23:50:28+00:00

Last Scan

Scanned2025-05-15T23:50:28+00:00
URL https://libroselectronicos.cervantes.es/robots.txt
Domain IPs 34.107.57.161
Response IP 34.107.57.161
Found Yes
Hash 3a42b48e640402314bee9e9c81757ad9739896a0970ad2bb6e4000cc897b78b7
SimHash 3875d351c5d5

Groups

applebot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /

*

Rule Path
Disallow /
Back to top

Comments

  • Applebot is the web crawler for Apple.
  • https://www.apple.com/go/applebot
  • Bing search engine robot.
  • https://www.bing.com/bingbot.htm
  • Petal search engine Robot (HUAWEI).
  • https://aspiegel.com/petalbot
  • SEMrushBot is the search bot software used by SEMrush.
  • https://www.semrush.com/bot.html
  • Googlebot is the generic name for Google's web crawler.
  • https://support.google.com/webmasters/answer/182072
  • All other robots are disallowed from crawling our servers.
Back to top