Log In Sign Up

millerotici.com
robots.txt

Robots Exclusion Standard data for millerotici.com

Resource Scan

Scan Details

Site Domain millerotici.com
Base Domain millerotici.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-01-09T03:16:59+00:00
Next Scan 2026-03-10T03:16:59+00:00

Last Successful Scan

Scanned2025-10-17T22:58:12+00:00
URL https://millerotici.com/robots.txt
Redirect https://www.millerotici.com/robots.txt
Redirect Domain www.millerotici.com
Redirect Base millerotici.com
Domain IPs 104.26.0.19, 104.26.1.19, 172.67.68.177, 2606:4700:20::681a:113, 2606:4700:20::681a:13, 2606:4700:20::ac43:44b1
Redirect IPs 104.26.0.19, 104.26.1.19, 172.67.68.177, 2606:4700:20::681a:113, 2606:4700:20::681a:13, 2606:4700:20::ac43:44b1
Response IP 104.26.0.19
Found Yes
Hash 370ccc9abbeb66700d2cc5f26d578d014ac1c83577f15a1aa6ba0bc5a0dc73f0
SimHash eb16415eeb22

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

*

Rule Path
Disallow /lib/
Disallow /assets/
Disallow /condiciones/
Disallow /politica/
Disallow /it/*
Disallow /mx/*
Disallow /publishers/*
Disallow /pub/*
Disallow /contactaranuncio/*
Disallow /favoritos/*
Disallow /comprar-creditos/*
Disallow /contactaranuncio/*
Disallow /programar-pack/*
Disallow /altrui/*
Disallow /contacto/*
Disallow /stats/*
Disallow /denunciar/*
Disallow /pagina/*
Disallow /user-panel/*
Disallow /user-listing-panel/*
Disallow /securitycheck/*
Disallow /publicar/*
Disallow /listajax/*
Disallow /embedvideo/*
Disallow /cancreateexperience/*
Back to top

Comments

  • Directivas para el bot de Bing para que deje un
  • delay (especificado en segundos) entre peticiones.
  • También se especifica el intervalo de horas en las
  • que se le permite hacer el crawl de la web
Back to top

Warnings

  • `visit-time` is not a known field.
Back to top