Log In Sign Up

petrolheadgarage.com
robots.txt

Robots Exclusion Standard data for petrolheadgarage.com

Resource Scan

Scan Details

Site Domain petrolheadgarage.com
Base Domain petrolheadgarage.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-11-25T23:07:05+00:00
Next Scan 2025-12-09T23:07:05+00:00

Last Successful Scan

Scanned2025-11-10T06:42:52+00:00
URL https://petrolheadgarage.com/robots.txt
Domain IPs 213.158.84.46
Response IP 213.158.84.46
Found Yes
Hash 1159a2402d6c7a4873a7775b4f39d6be613a06fb863e600203bbb844be8649c5
SimHash e0f45984057a

Groups

*

Rule Path
Allow /wp-content/uploads/
Disallow /cgi-bin
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-includes/
Disallow /wp-admin/
Disallow /wp-
Disallow /?s=
Disallow /search
Allow /feed/$
Disallow /feed
Disallow /comments/feed
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$
Allow /*.js$
Allow /*.css$

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://petrolheadgarage.com/sitemap_index.xml
Back to top

Comments

  • robots.txt para un blog WordPress.
  • Bloquear o permitir acceso a contenido adjunto. (Si la instalación está en /public_html).
  • Desindexar carpetas que empiecen por wp-
  • Permitir sitemap pero no las búsquedas.
  • Permitir Feed general para Google Blogsearch.
  • Impedir que /permalink/feed/ sea indexado pues el feed de comentarios suele posicionarse antes de los post.
  • Impedir URLs terminadas en /trackback/ que sirven como Trackback URI (contenido duplicado).
  • Evita bloqueos de CSS y JS.
  • Lista de bots que deberías permitir.
  • Lista de bots que generan consultas abusivas aunque siguen las pautas del archivo robots.txt
Back to top