barcelonatourstransfers.com
robots.txt

Robots Exclusion Standard data for barcelonatourstransfers.com

Resource Scan

Scan Details

Site Domain barcelonatourstransfers.com
Base Domain barcelonatourstransfers.com
Scan Status Ok
Last Scan2025-05-13T17:07:56+00:00
Next Scan 2025-05-20T17:07:56+00:00

Last Scan

Scanned2025-05-13T17:07:56+00:00
URL https://barcelonatourstransfers.com/robots.txt
Domain IPs 34.149.120.3, 34.149.36.179, 34.160.81.203, 35.244.153.44
Response IP 34.120.190.48
Found Yes
Hash b5623a02c79bae34e64ecc883922b52e65a0586346e7afa7ca403e0a8f1c48be
SimHash cc8c4bd8b400

Groups

*

Rule Path
Disallow /?s=
Disallow /search/
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /xmlrpc.php
Disallow /category/
Disallow /tag/
Disallow /categoria/
Disallow /etiqueta/
Disallow /feed/
Disallow /*/feed/
Disallow /comments/feed/
Allow /

googlebot

Rule Path
Allow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.barcelonatourstransfers.com/sitemaps/sitemap-index.xml
sitemap https://www.barcelonatourstransfers.com/sitemaps/page-sitemap.xml
sitemap https://www.barcelonatourstransfers.com/post-sitemap.xml
sitemap https://www.barcelonatourstransfers.com/sitemaps/product-sitemap.xml

Comments

  • ROBOTS.TXT
  • ---------------------------
  • Configuración optimizada para SEO, rastreo eficiente y seguridad básica
  • Bloqueo de páginas de búsqueda internas
  • Bloqueo de páginas administrativas y sensibles
  • Bloquear etiquetas y categorías de productos
  • Bloquear feeds RSS si no se usan
  • Permitir rastreo general del sitio
  • Directiva explícita para Googlebot
  • Limitar frecuencia para bots secundarios (ajustable si saturan el crawl budget)
  • Permitir acceso completo a AhrefsBot
  • Sitemaps para mejorar la indexación