betterfly.com
robots.txt

Robots Exclusion Standard data for betterfly.com

Resource Scan

Scan Details

Site Domain betterfly.com
Base Domain betterfly.com
Scan Status Ok
Last Scan2024-05-19T17:14:21+00:00
Next Scan 2024-06-02T17:14:21+00:00

Last Scan

Scanned2024-05-19T17:14:21+00:00
URL https://betterfly.com/robots.txt
Domain IPs 108.156.133.101, 108.156.133.123, 108.156.133.32, 108.156.133.53
Response IP 108.156.133.123
Found Yes
Hash a03fd2fa72745fc3c8180073312b607c050729227be6e082528c402905b73ae7
SimHash 6c9a4e12c472

Groups

*

Rule Path
Allow /wp-content/uploads/
Allow /
Allow /

*

Rule Path
Allow /wp-admin/admin-ajax.php
Disallow /cgi-bin
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-includes/
Disallow /wp-admin/
Disallow /theme/
Disallow /laintranet/js/
Disallow /lib/
Disallow /_api/
Disallow /calendar/
Disallow /_partials

*

Rule Path
Allow /wp-admin/admin-ajax.php
Allow /wp-content/uploads/
Disallow /wp-
Disallow /wp-content/
Disallow /wp-admin/
Disallow /?s=
Disallow /search
Disallow /archives/
Disallow /index.php
Disallow /*.php$
Disallow /*.js$
Disallow /*.inc$
Disallow /*.css$
Disallow /*feed/
Disallow /*trackback/
Disallow /*page/
Disallow /*tag/
Disallow /*category/
Disallow /*etiquetas/
Disallow /*categorias/
Disallow /*author/
Disallow /aviso_legal/
Disallow /politica-de-cookies/
Disallow /politica-de-privacidad/
Disallow /*.sql$
Disallow /*.tgz$
Disallow /*.gz$
Disallow /*.tar$
Disallow /*.svn$
Disallow /*/*/*/trackback/$
Allow /*.js$
Allow /*.css$
Disallow /*.pdf$

Other Records

Field Value
sitemap https://betterfly.com/sitemap_index.xml

Comments

  • Robots.txt creado por gopoint.agency
  • Añadimos una indicación de la localización del sitemap
  • Lista de bots que deberías permitir.
  • User-agent: Googlebot-Image
  • User-agent: Adsbot-Google
  • User-agent: Googlebot-Mobile
  • Bloquear o permitir acceso a contenido adjunto. (Si la instalación está en /public_html).
  • Impedir el acceso a los diferentes feed que genere la página
  • Impedir URLs terminadas en /trackback/ que sirven como Trackback URL.
  • Evita bloqueos de CSS y JS.
  • Bloquear todos los pdfs