Log In Sign Up

webadictos.com
robots.txt

Robots Exclusion Standard data for webadictos.com

Resource Scan

Scan Details

Site Domain webadictos.com
Base Domain webadictos.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-27T03:22:24+00:00
Next Scan 2024-11-25T03:22:24+00:00

Last Successful Scan

Scanned2023-08-03T20:41:40+00:00
URL https://webadictos.com/robots.txt
Domain IPs 104.21.29.35, 172.67.171.75, 2606:4700:3032::6815:1d23, 2606:4700:3035::ac43:ab4b
Response IP 172.67.171.75
Found Yes
Hash fbfecf4f3b6dc8aceb23ccffd646231f009b91708c748266f9388627352548f3
SimHash ea68dc340c14

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Allow /media/
Disallow /wp-content/
Disallow /wp-includes/
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /wp-content/themes/
Allow /wp-content/plugins/
Allow /wp-includes/js/
Disallow /wp-*
Disallow /tag/*
Disallow /tag/
Disallow /embed.js?pname=wordpress&pver=2.74
Disallow /1026310/article_bottom
Disallow /1026310/article_top
Disallow /1026310/article_top_html
Disallow /1026310/Post_abajo300x250
Disallow /1026310/SideBar300Internas
Disallow /1026310/w_post_bottom_movil
Disallow /1026310/w_post_top_movil
Disallow /1026310/WA_468_home
Disallow /1026310/WA_468_Internas
Disallow /1026310/WA_Destacados_Home
Disallow /1026310/WA_HomeSidebarMiddle
Disallow /1026310/WA_Interna_SidebarSociales
Disallow /1026310/WA_Skin_home
Disallow /1026310/WA_Skin_internas
Disallow /1026310/WA_Sky_Top_home
Disallow /1026310/WA_Top_moviles
Disallow /1026310/home_iPhone_Web
Disallow /1026310/Interna_iPhone_Web
Disallow /rsswamx.php
Disallow /?s=
Disallow /*?comments=
Disallow /*?postcomment=
Allow /feed/$
Disallow /feed
Disallow /comments/feed
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap http://webadictos.com/news-sitemap.xml
sitemap http://webadictos.com/sitemap_index.xml
Back to top

Comments

  • Primero el contenido adjunto.
  • Tambien podemos desindexar todo lo que empiece
  • por wp-. Es lo mismo que los Disallow de arriba pero
  • incluye cosas como wp-rss.php
  • Disallow: /wp-
  • Sitemap permitido, busquedas no.asi es!
  • Permitimos el feed general para Google Blogsearch.
  • Impedimos que permalink/feed/ sea indexado ya que el
  • feed con los comentarios suele posicionarse en lugar de
  • la entrada y desorienta a los usuarios.
  • Lo mismo con URLs terminadas en /trackback/ que solo
  • sirven como Trackback URI (y son contenido duplicado).
Back to top

Warnings

  • `host` is not a known field.
Back to top