luzdefaro.es
robots.txt

Robots Exclusion Standard data for luzdefaro.es

Resource Scan

Scan Details

Site Domain luzdefaro.es
Base Domain luzdefaro.es
Scan Status Ok
Last Scan2025-05-02T13:15:41+00:00
Next Scan 2025-06-01T13:15:41+00:00

Last Scan

Scanned2025-05-02T13:15:41+00:00
URL https://luzdefaro.es/robots.txt
Redirect https://www.luzdefaro.es/robots.txt
Redirect Domain www.luzdefaro.es
Redirect Base luzdefaro.es
Domain IPs 2001:8d8:100f:f000::20c, 217.160.0.117
Redirect IPs 2001:8d8:100f:f000::20c, 217.160.0.117
Response IP 217.160.0.117
Found Yes
Hash 3768f8ace5e1b993e8346ea1d2e8f5afaf96bb5862e2e15d38eacec4e35ca9d6
SimHash 021ec8f2eab9

Groups

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

*

Rule Path
Disallow /shopping_cart.php

*

Rule Path
Disallow /checkout_shipping.php

*

Rule Path
Disallow /catalogo

twengabot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /