planetadelibros.com
robots.txt

Robots Exclusion Standard data for planetadelibros.com

Resource Scan

Scan Details

Site Domain planetadelibros.com
Base Domain planetadelibros.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-05-24T05:23:05+00:00
Next Scan 2024-06-23T05:23:05+00:00

Last Successful Scan

Scanned2024-04-02T05:19:09+00:00
URL https://planetadelibros.com/robots.txt
Redirect https://www.planetadelibros.com/robots.txt
Redirect Domain www.planetadelibros.com
Redirect Base planetadelibros.com
Domain IPs 213.192.253.16
Redirect IPs 213.192.253.16
Response IP 213.192.253.16
Found Yes
Hash 7bb84be85be2e7923f53554bdbdfbbf56af52e0afe76841188ce1f87a459f6ac
SimHash 60cc5c524710

Groups

*

Rule Path
Disallow /usuaris/
Allow /usuaris/*.webp
Disallow */buscar?*
Disallow /*?access_token=*
Disallow /preview/pagina/
Disallow /usuaris/web_plataformas_venda/fotos/
Allow /usuaris/*/fotos/
Allow /usuaris/*.pdf
Allow *.js
Allow *.css
Allow *.json
Disallow /js/obfuscator.js
Allow /usuaris/libros_contenido/arxius/1/32_1_Lalineadeltiempodelanovela.doc

twitterbot

Rule Path
Allow /

blexbot
cliqzbot
mojeekbot
musobot
linkdexbot
seznambot
mj12bot
everyonesocialbot
gnowitnewsbot
leikibot
outclicksbot
sudorlybot
woobot
linkdexbot
epicbot
tweetmemebot
obot
yoozbot
msn-bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.planetadelibros.com/sitemap.xml

Comments

  • Robots V1.7
  • Bloqueo del crawling de ciertos directorios y del buscador
  • Permitiendo recursos del directorio usuaris
  • Permitiendo recursos de JS y CSS
  • Permitiendo páginas importantes
  • Sitemap