elmundo.cr
robots.txt

Robots Exclusion Standard data for elmundo.cr

Resource Scan

Scan Details

Site Domain elmundo.cr
Base Domain elmundo.cr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-11T13:38:04+00:00
Next Scan 2024-06-25T13:38:04+00:00

Last Successful Scan

Scanned2024-05-27T12:32:02+00:00
URL https://elmundo.cr/robots.txt
Domain IPs 2a02:4780:16:67a6:343f:3301:85ed:531, 77.37.48.41
Response IP 191.101.228.225
Found Yes
Hash b1c617e2c6839c99897c74744a4f0c547afe43b66fcefd505d4a4cbd46903f83
SimHash cc8c5900efd6

Groups

twitterbot

Rule Path
Disallow *
Allow /wp-content/uploads/*
Allow /archives

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /*/feed
Disallow /*/trackback
Disallow /*?
Disallow /*.js$
Disallow /*.inc$
Disallow /feed/
Disallow /feed
Disallow /tag/
Disallow /archives/
Disallow /2016/
Disallow /2015/
Disallow /2017/
Disallow /2018/
Disallow /portfolio/
Disallow /author/
Disallow /portfolio_tag/
Disallow /category/sin-categoria/
Disallow /?s=*