kanto.global
robots.txt

Robots Exclusion Standard data for kanto.global

Resource Scan

Scan Details

Site Domain kanto.global
Base Domain kanto.global
Scan Status Ok
Last Scan2025-10-16T05:30:29+00:00
Next Scan 2025-10-23T05:30:29+00:00

Last Scan

Scanned2025-10-16T05:30:29+00:00
URL https://kanto.global/robots.txt
Domain IPs 104.21.1.173, 172.67.129.169, 2606:4700:3032::ac43:81a9, 2606:4700:3035::6815:1ad
Response IP 104.21.1.173
Found Yes
Hash 403fdb2597483f2ae9734b311217e9f924a9d5d9df069277161751f606bf42ac
SimHash 3b364c708ec3

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

yandexbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

whatsapp

Rule Path
Allow /

telegrambot

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

semrushbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

amazonbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

openai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

scrapebot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

*

Rule Path
Disallow /api/
Disallow /admin/
Disallow /_next/
Disallow /wp-admin/
Disallow /wp-content/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /tmp/
Disallow /private/
Disallow /user/
Disallow /login
Disallow /register
Disallow /search?
Disallow /*?*sort=
Disallow /*?*filter=
Allow /robots.txt
Allow /sitemap.xml
Allow /favicon.ico
Allow /.well-known/

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://kanto.co/sitemap.xml

Comments

  • Robots.txt optimizado para kanto.co
  • Permite bots legítimos y bloquea scrapers maliciosos
  • Bots de búsqueda principales - PERMITIR TODO
  • Bots de redes sociales - PERMITIR TODO
  • Bots de SEO y análisis legítimos
  • Bots específicos - BLOQUEAR
  • Scrapers conocidos - BLOQUEAR
  • Reglas generales para todos los demás bots
  • Permitir archivos importantes
  • Sitemap
  • Información de contacto (opcional)
  • Contact: hola@kanto.co