Log In Sign Up

totalcomputer.it
robots.txt

Robots Exclusion Standard data for totalcomputer.it

Resource Scan

Scan Details

Site Domain totalcomputer.it
Base Domain totalcomputer.it
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-10T00:50:40+00:00
Next Scan 2025-11-09T00:50:40+00:00

Last Successful Scan

Scanned2025-09-11T00:14:20+00:00
URL https://totalcomputer.it/robots.txt
Domain IPs 75.102.58.54
Response IP 75.102.58.54
Found Yes
Hash 093ab459eaae0eaad0af87950f707d6c29f25154ebe36bdb1fa0767ca4a3f666
SimHash 60f25d000eb2

Groups

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

xenu

Rule Path
Disallow /

ahrefs

Rule Path
Disallow /

semrush

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin
Disallow /wp-includes/
Disallow /wp-admin/
Disallow /sitemap/
Disallow /author/
Disallow /?format=feed&amp%3Btype=rss
Disallow /wp-register.php
Disallow /xmlrpc.php
Disallow /template.html
Disallow /wp-comments
Disallow /cgi-bin
Disallow /trackback
Disallow /feed
Disallow /comments
Disallow /comment-page
Disallow /replytocom%3D
Disallow /author
Disallow /?author=
Disallow /tag
Disallow /?feed=
Disallow /?s=
Disallow /?se=
Disallow /prueba
Disallow *?replytocom
Disallow /?s=
Disallow /author/*/$
Disallow /avviso-legale/
Disallow /comments/feed
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$
Disallow /*?wordfence_lh=
Disallow /*.pdf
Back to top

Other Records

Field Value
sitemap https://totalcomputer.it/sitemap_index.xml
sitemap https://totalcomputer.it/sitemap_index.xml
sitemap https://totalcomputer.it/blog/feed/
Back to top

Comments

  • Permitir sitemap pero no las busquedas.
  • Evita bloqueos de CSS y JS.
  • Lista de bots que deberias permitir.
  • Lista de bots que generan consultas abusivas aunque siguen las pautas del archivo robots.txt
  • Indicamos que estas reglas son aplicables a todos los buscadores
  • Impedir que /permalink/feed/ sea indexado pues el feed de comentarios suele posicionarse antes de los post.
  • Impedir URLs terminadas en /trackback/ que sirven como Trackback URI (contenido duplicado).
Back to top

Warnings

  • 5 invalid lines.
  • `host` is not a known field.
Back to top