clubdia.dia.es
robots.txt

Robots Exclusion Standard data for clubdia.dia.es

Resource Scan

Scan Details

Site Domain clubdia.dia.es
Base Domain dia.es
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-07T03:28:08+00:00
Next Scan 2024-12-06T03:28:08+00:00

Last Successful Scan

Scanned2023-05-11T20:21:05+00:00
URL https://clubdia.dia.es/robots.txt
Domain IPs 184.87.193.132, 184.87.193.160, 2600:1413:b000:13::b857:c184, 2600:1413:b000:13::b857:c1a0
Response IP 42.99.140.179
Found Yes
Hash 774d3d591a7c274e326fcb95838896728c7fb3cac4fd78bbe113233dd4a61a37
SimHash a0041c219280

Groups

*

Rule Path
Disallow /login.html?action=conversionCliente
Disallow /login.html?action=errorLogin&errores=comun.error.sesion.expirada

bot*

Rule Path
Disallow /

discovery

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yahoo slurp

Rule Path
Disallow /

crawl

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

Other Records

Field Value
sitemap http://clubdia.dia.es/sitemap.xml