Log In Sign Up

davidbenavente.com
robots.txt

Robots Exclusion Standard data for davidbenavente.com

Resource Scan

Scan Details

Site Domain davidbenavente.com
Base Domain davidbenavente.com
Scan Status Ok
Last Scan2025-07-29T12:50:33+00:00
Next Scan 2025-08-28T12:50:33+00:00

Last Scan

Scanned2025-07-29T12:50:33+00:00
URL https://davidbenavente.com/robots.txt
Domain IPs 2001:8d8:100f:f000::271, 217.160.0.140
Response IP 217.160.0.140
Found Yes
Hash 2217816717cad650f62330e01a7e6e932fb9add015c84458ca514d8855dce5e7
SimHash a27cc8404cbc

Groups

*

Rule Path
Disallow

*

Rule Path
Disallow /wp-admin/
Disallow /feed/
Disallow /trackback/
Disallow /xmlrpc.php
Disallow /*?remove_item=
Disallow /*?attachment_id=
Disallow /*?redirect_to=
Disallow /*?noamp=mobile
Disallow /tag/
Disallow /etiqueta/

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://davidbenavente.com/sitemap_index.xml
Back to top

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK
  • Agentes de usuario permitidos (Reglas generales)
  • Bloqueo de parametros de URL no deseados para todos los bots
  • Disallow: /*?add-to-cart=
  • Bloqueo de taxonomias y rutas de poco valor SEO
  • Bloqueo de bots de SEO y analisis (Opcional, pero recomendado para ahorrar ancho de banda)
  • Permitir explicitamente a los bots importantes
Back to top