Log In Sign Up

pocoloco-restaurant.de
robots.txt

Robots Exclusion Standard data for pocoloco-restaurant.de

Resource Scan

Scan Details

Site Domain pocoloco-restaurant.de
Base Domain pocoloco-restaurant.de
Scan Status Ok
Last Scan2026-04-02T16:56:00+00:00
Next Scan 2026-04-16T16:56:00+00:00

Last Scan

Scanned2026-04-02T16:56:00+00:00
URL https://pocoloco-restaurant.de/robots.txt
Domain IPs 212.227.10.30, 2a01:239:2ac:df00::1
Response IP 212.227.10.30
Found Yes
Hash 871b3ce7dce777998ad8008dfcc0f76e7a390e7ed11b15aee2291990a7d1a1df
SimHash 2217921706e6

Groups

*

Rule Path
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/
Allow /

gptbot

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 1

chatgpt-user

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 1

oai-searchbot

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 1

claudebot

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 1

claude-user

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 1

claude-searchbot

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 1

perplexitybot

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 2

ccbot

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 2

google-extended

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

facebot

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 1

meta-externalagent

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 1

meta-externalfetcher

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 1

youbot

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 2

kagibot

Rule Path
Allow /
Disallow /admin/
Disallow /admin_dark/
Disallow /scripts/

Other Records

Field Value
crawl-delay 2
Back to top

Other Records

Field Value
sitemap https://pocoloco-restaurant.de/sitemap.xml
Back to top

Comments

  • robots.txt — PocoLoco Restaurants (pocoloco-restaurant.de)
  • Ziel: KI-/LLM-Bots aktiv einladen, sensible Bereiche ausnehmen,
  • und höfliche Crawl-Delays für Nicht-Google-Bots setzen.
  • Allgemeine Standardregeln
  • XML-Sitemap(s)
  • Hinweisdatei für KI-Agents (nur Info/Hinweis; kein offizieller Standard)
  • OpenAI / ChatGPT / Search (explizit erlauben)
  • Training-Crawler
  • On-demand Browsing (wenn ChatGPT-Nutzer Links öffnen)
  • OpenAI Search / SearchGPT
  • Anthropic / Claude
  • Perplexity
  • Common Crawl (Basis für viele LLM-Datensätze)
  • Google KI-Policy-Token
  • Erlaubt Verwendung für Gemini/AI-Features (kein klassischer Crawler)
  • Apple AI-Policy-Token
  • Erlaubt Verwendung für Apple Intelligence/AI-Features
  • Meta / Facebook / Instagram
  • Link-Previews (Open Graph)
  • Historischer/Ads-Crawler (wird teils noch gesehen)
  • AI-Training/Indexing (Meta AI / Llama)
  • User-initiierte Abrufe für Meta-AI-Funktionen
  • Weitere AI-/Suche-Crawler (freundlich erlauben)
Back to top

Warnings

  • `llms.txt` is not a known field.
Back to top