Log In Sign Up

onubenses.com
robots.txt

Robots Exclusion Standard data for onubenses.com

Resource Scan

Scan Details

Site Domain onubenses.com
Base Domain onubenses.com
Scan Status Ok
Last Scan2026-02-01T01:40:01+00:00
Next Scan 2026-02-08T01:40:01+00:00

Last Scan

Scanned2026-02-01T01:40:01+00:00
URL https://onubenses.com/robots.txt
Domain IPs 94.23.86.171
Response IP 94.23.86.171
Found Yes
Hash 6409327806313e3027ba7f4b82f22ca25ef6bbb911c3adb7d326f0d437801c69
SimHash 290b9e512497

Groups

*

Rule Path
Disallow /%40%40search
Disallow /%40%40search_rss$
Disallow /%40%40sendto_form$
Disallow /login_form
Disallow /mail_password_form
Disallow /contact-info
Disallow /change_password
Disallow /*atct_album_view$
Disallow /*folder_summary_view$
Disallow /*listing_view$
Disallow /*thumbnail_view$
Disallow /*summary_view$
Disallow /*folder_factories$
Disallow /*edit$
Disallow /portal_css/
Disallow /portal_javascripts/
Disallow /portal_factory/
Disallow /temporary_folder/
Disallow /%2B%2Bresource%2B%2B
Disallow /*?

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

duckduckbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

ecosia

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

ccbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

barkrowler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

petalbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60
Back to top

Other Records

Field Value
sitemap https://onubenses.com/sitemap.xml.gz
Back to top

Comments

  • Evitar búsquedas, formularios y vistas sin valor
  • Recursos internos de Plone
  • Evitar URLs con parámetros
  • --- Control por buscador ---
  • Google: respeta robots.txt pero ignora Crawl-delay
  • Bing y Yahoo! (comparten infraestructura, respetan Crawl-delay)
  • DuckDuckGo
  • Yandex
  • Ecosia (basado en Bing)
  • GPTBot (OpenAI)
  • CCBot (Common Crawl)
  • Barkrowler (Qwant)
  • PetalBot (Huawei)
Back to top