Log In Sign Up

smartertech.info
robots.txt

Robots Exclusion Standard data for smartertech.info

Resource Scan

Scan Details

Site Domain smartertech.info
Base Domain smartertech.info
Scan Status Ok
Last Scan2025-06-18T16:37:40+00:00
Next Scan 2025-06-25T16:37:40+00:00

Last Scan

Scanned2025-06-18T16:37:40+00:00
URL https://smartertech.info/robots.txt
Domain IPs 109.234.167.14
Response IP 109.234.167.14
Found Yes
Hash 2098e68078237633c0a3bc51e034d7e86fd54158c6f5f6a01d3785b799aa35df
SimHash 4146d8d2f245

Groups

*

Rule Path
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content/plugins
Disallow /wp-content/cache
Disallow /trackback
Disallow /feed
Disallow /comments
Disallow /category/*/*
Disallow */trackback
Disallow */feed
Disallow */comments
Disallow /*.pdf$
Disallow /*?*
Disallow /*?
Disallow /wp-login.php
Allow /wp-content/uploads

googlebot

Rule Path
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz$
Disallow /*.pdf$

googlebot-image

Rule Path
Disallow
Allow /*

mediapartners-google*

Rule Path
Disallow
Allow /*

ahrefssiteaudit

Rule Path
Allow /*

ahrefsbot

Rule Path
Allow /*
Disallow /wp-content/sabai/
Allow /wp-content/sabai/File/thumbnails/
Disallow /wp-content/plugins/sabai/
Disallow /wp-content/plugins/sabai-directory/
Back to top

Comments

  • On empeche l'indexation des dossiers sensibles
  • On desindexe toutes les URL ayant des parametres (duplication de contenu)
  • On desindexe la page de connexion (contenu inutile)
  • On autorise l'indexation des images
  • On empeche l'indexation des fichiers sensibles
  • Autoriser Google Image
  • Autoriser Google AdSense
  • Autoriser Ahrefs
Back to top