Log In Sign Up

labelaure.fr
robots.txt

Robots Exclusion Standard data for labelaure.fr

Resource Scan

Scan Details

Site Domain labelaure.fr
Base Domain labelaure.fr
Scan Status Ok
Last Scan2026-03-12T10:08:15+00:00
Next Scan 2026-03-19T10:08:15+00:00

Last Scan

Scanned2026-03-12T10:08:15+00:00
URL https://labelaure.fr/robots.txt
Domain IPs 109.234.167.37
Response IP 109.234.167.37
Found Yes
Hash 86e09c9d3305d452198ed859f2db9e4732cad1e410e6ff3356db368b5c2f84d8
SimHash 6188bb10c433

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

gptbot

Rule Path
Allow /wp-json/llm-endpoints/

Other Records

Field Value
crawl-delay 1

chatgpt-user

Rule Path
Allow /

perplexitybot

Rule Path
Allow /wp-json/llm-endpoints/

Other Records

Field Value
crawl-delay 1

claudebot

Rule Path
Allow /wp-json/llm-endpoints/

Other Records

Field Value
crawl-delay 1

google-extended

Rule Path
Allow /wp-json/llm-endpoints/

anthropic-ai

Rule Path
Allow /wp-json/llm-endpoints/

cohere-ai

Rule Path
Allow /wp-json/llm-endpoints/
Back to top

Other Records

Field Value
sitemap https://labelaure.fr/sitemap-news.xml
sitemap https://labelaure.fr/wp-json/llm-endpoints/v1/sitemap
sitemap https://labelaure.fr/sitemaps.xml
Back to top

Comments

  • AI-optimized sitemap
  • AI crawling guidelines
  • See: https://labelaure.fr/wp-json/llm-endpoints/v1/ai.txt
  • AI crawlers - prioritize optimized endpoints
Back to top