Log In Sign Up

andresmarin.com
robots.txt

Robots Exclusion Standard data for andresmarin.com

Resource Scan

Scan Details

Site Domain andresmarin.com
Base Domain andresmarin.com
Scan Status Ok
Last Scan2025-10-16T10:44:54+00:00
Next Scan 2025-11-15T10:44:54+00:00

Last Scan

Scanned2025-10-16T10:44:54+00:00
URL https://andresmarin.com/robots.txt
Redirect https://www.andresmarin.com/robots.txt
Redirect Domain www.andresmarin.com
Redirect Base andresmarin.com
Domain IPs 91.146.100.125
Redirect IPs 91.146.100.125
Response IP 91.146.100.125
Found Yes
Hash 0591ebf3378d335b32bf4a79a2cb33211674c603bd258a907e9aacfef922e0b5
SimHash d35e1c104cd7

Groups

*

Rule Path
Disallow /productos_detalle.php
Disallow /complementos_detalle.php

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap http://www.andresmarin.com/sitemap.xml
Back to top

Comments

  • Deshabilitar rastreadores abusadores
  • Slurp (Yahoo!), Noxtrum y el bot de MSN a veces tienen
  • idas de pinza, toca decirles que reduzcan la marcha.
  • El valor es en segundos y podéis dejarlo bajo e ir
  • subiendo hasta el punto óptimo.
Back to top