dibujoskawaii.pet
robots.txt

Robots Exclusion Standard data for dibujoskawaii.pet

Resource Scan

Scan Details

Site Domain dibujoskawaii.pet
Base Domain dibujoskawaii.pet
Scan Status Ok
Last Scan2024-11-14T14:20:52+00:00
Next Scan 2024-11-21T14:20:52+00:00

Last Scan

Scanned2024-11-14T14:20:52+00:00
URL https://dibujoskawaii.pet/robots.txt
Domain IPs 2406:da18:9d0:143f:2124:4e9c:36a9:d9de, 52.221.42.138
Response IP 52.221.42.138
Found Yes
Hash 5ee67bdfdd06837f75e9af0afde372dab5106cbe3dd50f2a79afde64a9595473
SimHash e8bc49822938

Groups

*

Rule Path
Allow /*.js$
Allow /*.css$
Allow */modules/*.css
Allow /feed/$
Allow /*.css
Allow /*.js
Allow /wp-admin/*.js
Allow /wp-admin/*.css
Allow /wp-admin/admin-ajax.php
Allow /wp-includes/js/jquery/jquery.js
Allow /wp-includes/js/*
Disallow /cgi-bin
Disallow /sitemap/
Disallow /author/
Disallow /?format=feed&type=rss
Disallow /wp-register.php
Disallow /xmlrpc.php
Disallow /template.html
Disallow /wp-comments
Disallow /cgi-bin
Disallow /trackback
Disallow /feed
Disallow /comments
Disallow /comment-page
Disallow /replytocom%3D
Disallow /author
Disallow /?author=
Disallow /tag
Disallow /?feed=
Disallow /?s=
Disallow /?se=
Disallow /prueba
Disallow *?replytocom
Disallow /?s=
Disallow /author/*/$
Disallow /wp-admin/admin-ajax.php*
Disallow /?controller=*
Disallow /comments/feed
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$
Disallow /comments/feed/
Disallow /wp-admin/
Disallow /wp-login/
Disallow /wp-content/uploads/woocommerce_pdf_invoices
Disallow /wp-includes/
Disallow /*/feed/
Disallow /*/trackback/
Disallow /*/attachment/
Disallow /author/
Allow /blog/page*
Disallow /*/page/
Disallow /*/feed/
Disallow /tag/*/page/
Disallow /tag/*/feed/
Disallow /page/
Disallow /comments/
Disallow /xmlrpc.php
Disallow /*?s=
Disallow /*/*/*/feed.xml
Disallow /?attachment_id*
Disallow /mi-cuenta/*
Disallow /carro/*
Disallow /total-pedido/*
Disallow /404/*
Disallow /?s=
Disallow /search

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /

Comments

  • Allow
  • Evita bloqueos de CSS y JS.
  • Indicamos que estas reglas son aplicables a todos los buscadores
  • Impedir que /permalink/feed/ sea indexado pues el feed de comentarios suele posicionarse antes de los post.
  • Impedir URLs terminadas en /trackback/ que sirven como Trackback URI (contenido duplicado).
  • Lista de bots que deberías permitir.
  • Lista de bots que generan consultas abusivas aunque siguen las pautas del archivo robots.txt