Log In Sign Up

eroticos.com
robots.txt

Robots Exclusion Standard data for eroticos.com

Resource Scan

Scan Details

Site Domain eroticos.com
Base Domain eroticos.com
Scan Status Ok
Last Scan2024-05-13T16:30:20+00:00
Next Scan 2024-06-12T16:30:20+00:00

Last Scan

Scanned2024-05-13T16:30:20+00:00
URL https://eroticos.com/robots.txt
Domain IPs 104.26.2.251, 104.26.3.251, 172.67.68.23, 2606:4700:20::681a:2fb, 2606:4700:20::681a:3fb, 2606:4700:20::ac43:4417
Response IP 104.26.2.251
Found Yes
Hash 370ccc9abbeb66700d2cc5f26d578d014ac1c83577f15a1aa6ba0bc5a0dc73f0
SimHash eb16415eeb22

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

*

Rule Path
Disallow /lib/
Disallow /assets/
Disallow /condiciones/
Disallow /politica/
Disallow /it/*
Disallow /mx/*
Disallow /publishers/*
Disallow /pub/*
Disallow /contactaranuncio/*
Disallow /favoritos/*
Disallow /comprar-creditos/*
Disallow /contactaranuncio/*
Disallow /programar-pack/*
Disallow /altrui/*
Disallow /contacto/*
Disallow /stats/*
Disallow /denunciar/*
Disallow /pagina/*
Disallow /user-panel/*
Disallow /user-listing-panel/*
Disallow /securitycheck/*
Disallow /publicar/*
Disallow /listajax/*
Disallow /embedvideo/*
Disallow /cancreateexperience/*
Back to top

Comments

  • Directivas para el bot de Bing para que deje un
  • delay (especificado en segundos) entre peticiones.
  • También se especifica el intervalo de horas en las
  • que se le permite hacer el crawl de la web
Back to top

Warnings

  • `visit-time` is not a known field.
Back to top