Log In Sign Up

lelivretdesconcours.com
robots.txt

Robots Exclusion Standard data for lelivretdesconcours.com

Resource Scan

Scan Details

Site Domain lelivretdesconcours.com
Base Domain lelivretdesconcours.com
Scan Status Ok
Last Scan2026-03-23T04:30:13+00:00
Next Scan 2026-04-22T04:30:13+00:00

Last Scan

Scanned2026-03-23T04:30:13+00:00
URL https://lelivretdesconcours.com/robots.txt
Domain IPs 104.21.83.22, 172.67.167.84, 2606:4700:3033::ac43:a754, 2606:4700:3037::6815:5316
Response IP 104.21.83.22
Found Yes
Hash 5259bb3178bfca32e51ec925b6d2e21a726dee2ae151c83050ea0801e37d5601
SimHash c9180c60a755

Groups

*

Rule Path
Allow /
Disallow /dashboard
Disallow /dashboard/*
Disallow /admin
Disallow /admin/*
Disallow /consumer
Disallow /user
Disallow /subscriptions
Disallow /api/
Disallow /storage/
Disallow /account-deletion
Disallow /activate-account/
Disallow /*.json$
Disallow /*.env$
Allow /sitemap.xml

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0

gptbot

Rule Path
Allow /

claude-web

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

ccbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://lelivretdesconcours.com/sitemap.xml
Back to top

Comments

  • Robots.txt pour Le Livret des Concours
  • https://lelivretdesconcours.com
  • Bloquer les pages admin et privées
  • Bloquer les fichiers sensibles
  • Autoriser explicitement le sitemap
  • Sitemap
  • Crawl-delay pour ne pas surcharger le serveur
  • Règles spécifiques pour Googlebot
  • Règles pour les bots IA
Back to top