rx3.org
robots.txt

Robots Exclusion Standard data for rx3.org

Resource Scan

Scan Details

Site Domain rx3.org
Base Domain rx3.org
Scan Status Ok
Last Scan2025-10-13T09:16:09+00:00
Next Scan 2025-10-27T09:16:09+00:00

Last Scan

Scanned2025-10-13T09:16:09+00:00
URL https://rx3.org/robots.txt
Redirect https://www.rx3.org/robots.txt
Redirect Domain www.rx3.org
Redirect Base rx3.org
Domain IPs 62.210.209.131
Redirect IPs 62.210.209.131
Response IP 62.210.209.131
Found Yes
Hash 60f910aac092f34995e1c35d6cab820f77080398a222f9f7f939f404d86f0dae
SimHash 09711ecc418d

Groups

*

Rule Path
Disallow /cgi-private
Disallow /cgi-bin
Disallow /private

webcrawler

Rule Path
Disallow /cgi-private
Disallow /cgi-bin
Disallow /private

lycra

Rule Path
Disallow /cgi-private
Disallow /cgi-bin
Disallow /private

Comments

  • Fichier autorisant ou non la visite des robots de recherche Web
  • Pour connaitre les visites d'un robot, le serveur
  • doit supporter les User-agent
  • Rem: Pour limiter les robots sans etre administrateur
  • du serveur, utiliser dans la page HTML:
  • <META NAME="ROBOTS" CONTENT="INDEX|NOINDEX,FOLLOW|NOFOLLOW">
  • Nom des robots (* pour tous) et restrictions