Log In Sign Up

underc0de.org
robots.txt

Robots Exclusion Standard data for underc0de.org

Resource Scan

Scan Details

Site Domain underc0de.org
Base Domain underc0de.org
Scan Status Ok
Last Scan2026-02-07T10:39:42+00:00
Next Scan 2026-03-09T10:39:42+00:00

Last Scan

Scanned2026-02-07T10:39:42+00:00
URL https://underc0de.org/robots.txt
Domain IPs 104.21.53.222, 172.67.219.89, 2606:4700:3037::6815:35de, 2606:4700:3037::ac43:db59
Response IP 172.67.219.89
Found Yes
Hash 8b06b4b6338b47f42099eefe04d854662c8400e36f50dedcee5354d26b2ffd36
SimHash 7b1ea1017677

Groups

*

Rule Path
Disallow /foro/index.php?action=register2
Disallow /foro/index.php?action=register
Allow /

gptbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

duckduckbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ccbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

petalbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

google-inspectiontool

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

yandex

Rule Path
Allow /
Disallow /search

duckduckbot

Rule Path
Allow /

applebot

Rule Path
Allow /

ccbot

Rule Path
Allow /

petalbot

Rule Path
Allow /

google-extended

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://underc0de.org/sitemap-main.xml
sitemap https://underc0de.org/sitemap.xml
sitemap https://underc0de.org/sitemaps.xml
sitemap https://underc0de.org/sitemaps/almacen-de-manuales.xml
sitemap https://underc0de.org/sitemaps/android.xml
sitemap https://underc0de.org/sitemaps/apple.xml
sitemap https://underc0de.org/sitemaps/arduino-raspberry-pi.xml
sitemap https://underc0de.org/sitemaps/asm.xml
sitemap https://underc0de.org/sitemaps/base-de-datos.xml
sitemap https://underc0de.org/sitemaps/batch-bash.xml
sitemap https://underc0de.org/sitemaps/bugs-y-exploits.xml
sitemap https://underc0de.org/sitemaps/c-c.xml
sitemap https://underc0de.org/sitemaps/codigos-fuentes-56.xml
sitemap https://underc0de.org/sitemaps/codigos-fuentes.xml
sitemap https://underc0de.org/sitemaps/criptografia-253.xml
sitemap https://underc0de.org/sitemaps/criptomonedas.xml
sitemap https://underc0de.org/sitemaps/debates.xml
sitemap https://underc0de.org/sitemaps/delphi.xml
sitemap https://underc0de.org/sitemaps/desarrollo-movil.xml
sitemap https://underc0de.org/sitemaps/descargas-de-juegos.xml
sitemap https://underc0de.org/sitemaps/diseno-grafico-132.xml
sitemap https://underc0de.org/sitemaps/e-zines.xml
sitemap https://underc0de.org/sitemaps/dudas-generales-121.xml
sitemap https://underc0de.org/sitemaps/electronica-y-robotica.xml
sitemap https://underc0de.org/sitemaps/galeria-164.xml
sitemap https://underc0de.org/sitemaps/gnulinux.xml
sitemap https://underc0de.org/sitemaps/hacking-showoff.xml
sitemap https://underc0de.org/sitemaps/hacking.xml
sitemap https://underc0de.org/sitemaps/hardware.xml
sitemap https://underc0de.org/sitemaps/herramientas-hacking.xml
sitemap https://underc0de.org/sitemaps/impresiones-3d.xml
sitemap https://underc0de.org/sitemaps/informatica-en-general.xml
sitemap https://underc0de.org/sitemaps/informatica-forense.xml
sitemap https://underc0de.org/sitemaps/ingenieria-inversa.xml
sitemap https://underc0de.org/sitemaps/inteligencia-artificial-y-biohacking.xml
sitemap https://underc0de.org/sitemaps/international-forum.xml
sitemap https://underc0de.org/sitemaps/ios.xml
sitemap https://underc0de.org/sitemaps/java.xml
sitemap https://underc0de.org/sitemaps/jquery.xml
sitemap https://underc0de.org/sitemaps/malware.xml
sitemap https://underc0de.org/sitemaps/marketing.xml
sitemap https://underc0de.org/sitemaps/noticias-informaticas-120.xml
sitemap https://underc0de.org/sitemaps/off-topic-122.xml
sitemap https://underc0de.org/sitemaps/pentest.xml
sitemap https://underc0de.org/sitemaps/perl.xml
sitemap https://underc0de.org/sitemaps/php.xml
sitemap https://underc0de.org/sitemaps/phreaking.xml
sitemap https://underc0de.org/sitemaps/presentaciones-147.xml
sitemap https://underc0de.org/sitemaps/programacion-de-malwares.xml
sitemap https://underc0de.org/sitemaps/programacion-general.xml
sitemap https://underc0de.org/sitemaps/programacion-web-247.xml
sitemap https://underc0de.org/sitemaps/python.xml
sitemap https://underc0de.org/sitemaps/qa-testing.xml
sitemap https://underc0de.org/sitemaps/recursos-208.xml
sitemap https://underc0de.org/sitemaps/redes.xml
sitemap https://underc0de.org/sitemaps/root-exploits.xml
sitemap https://underc0de.org/sitemaps/ruby.xml
sitemap https://underc0de.org/sitemaps/scripting.xml
sitemap https://underc0de.org/sitemaps/seguridad-en-servidores.xml
sitemap https://underc0de.org/sitemaps/seguridad.xml
sitemap https://underc0de.org/sitemaps/software-155.xml
sitemap https://underc0de.org/sitemaps/soluciones-de-wargames.xml
sitemap https://underc0de.org/sitemaps/sugerencias-233.xml
sitemap https://underc0de.org/sitemaps/talleres-underc0de-213.xml
sitemap https://underc0de.org/sitemaps/testing-automatizado.xml
sitemap https://underc0de.org/sitemaps/tutoriales-y-manuales.xml
sitemap https://underc0de.org/sitemaps/tutoriales.xml
sitemap https://underc0de.org/sitemaps/underc0de.xml
sitemap https://underc0de.org/sitemaps/video-juegos.xml
sitemap https://underc0de.org/sitemaps/visual-basic.xml
sitemap https://underc0de.org/sitemaps/visual-studio-net.xml
sitemap https://underc0de.org/sitemaps/wargames-202.xml
sitemap https://underc0de.org/sitemaps/windows-123.xml
sitemap https://underc0de.org/sitemaps/wireless.xml
Back to top

Comments

  • --- Permitir a los bots de OpenAI ---
  • --- GOOGLE (no usa Crawl-delay; pone reglas finas si hace falta) ---
  • --- BING / YAHOO (Yahoo usa Bingbot hoy) ---
  • --- YANDEX (además respeta "Host" al final) ---
  • --- DUCKDUCKGO ---
  • --- APPLE (Siri/Spotlight previews) ---
  • --- COMMON CRAWL (datasets abiertos, usado por múltiples proyectos/LLMs) ---
  • --- PETAL (Huawei) ---
  • Extensión de Google para IA generativa
Back to top