Log In Sign Up

definiciona.com
robots.txt

Robots Exclusion Standard data for definiciona.com

Resource Scan

Scan Details

Site Domain definiciona.com
Base Domain definiciona.com
Scan Status Ok
Last Scan2024-11-11T10:14:41+00:00
Next Scan 2024-11-18T10:14:41+00:00

Last Scan

Scanned2024-11-11T10:14:41+00:00
URL https://definiciona.com/robots.txt
Domain IPs 104.21.42.119, 172.67.161.217, 2606:4700:3036::6815:2a77, 2606:4700:3037::ac43:a1d9
Response IP 104.21.42.119
Found Yes
Hash fceb7fdc1f1de54de1c882378714c77e225a41498656aac4d4b4e712a1b127e4
SimHash de52153a2e39

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /tag/
Disallow /*/amp/
Disallow /*/*/page/
Disallow /*/*/*/page/
Disallow /*/feed/
Disallow /*/*/feed/
Disallow /*/*/*/feed/
Disallow /*/rss/
Disallow /*/*/rss/
Disallow /*/*/*/rss/
Disallow /?
Disallow /*?
Disallow /*/?
Disallow /*/*/?
Disallow /*/*/*/?
Disallow /search
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

httrack

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

wget

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

acunetix

Rule Path
Disallow /

nikto

Rule Path
Disallow /

sqlmap

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

amazon-kendra

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20
Back to top

Other Records

Field Value
sitemap https://definiciona.com/sitemap.xml
Back to top

Comments

  • Specific robot directives / Directivas especí­ficas para robots:
  • Definiciona content is made available under our terms and conditions; for personal, non-commercial use / El contenido de Definiciona está disponible bajo nuestros términos y condiciones; para uso personal, no comercial;
  • Use of any device, tool, or process designed to data mine or scrape the content, using automated means, it is prohibited without prior written permission from Definiciona / El uso de cualquier dispositivo, herramienta o proceso diseñado para extraer datos o raspar el contenido, utilizando medios automatizados, está prohibido sin el permiso previo por escrito de Definiciona.
  • Prohibited uses include but are not limited to: / Los usos prohibidos del contenido incluyen, entre otros:
  • (1) text and data mining activities; / (1) actividades de minería de textos y datos;
  • (2) the development of any software, machine learning, artificial intelligence (AI), and/or large language models (LLMs); / (2) el desarrollo de cualquier software, aprendizaje automático, inteligencia artificial (IA) y/o grandes modelos de lenguaje (LLM);
  • (3) creating or providing archived or cached data sets containing our content to others; and/or / (3) crear o proporcionar conjuntos de datos archivados o en caché que contengan nuestro contenido a otros; y/o
  • (4) any commercial purposes / (4) cualquier propósito comercial.
Back to top