donbalonrosa.defensacentral.com
robots.txt

Robots Exclusion Standard data for donbalonrosa.defensacentral.com

Resource Scan

Scan Details

Site Domain donbalonrosa.defensacentral.com
Base Domain defensacentral.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-07T06:00:19+00:00
Next Scan 2024-11-05T06:00:19+00:00

Last Successful Scan

Scanned2023-09-20T05:29:45+00:00
URL https://donbalonrosa.defensacentral.com/robots.txt
Domain IPs 104.26.8.119, 104.26.9.119, 172.67.68.26, 2606:4700:20::681a:877, 2606:4700:20::681a:977, 2606:4700:20::ac43:441a
Response IP 172.67.68.26
Found Yes
Hash 1eee673e7ae724dca3b659d5bca56304a0583127578d11919d022687c374f4ad
SimHash 68445c808395

Groups

*

Rule Path
Allow /public/
Disallow /admin/
Disallow /es_ES/resultados/noticias/?q=
Allow /feed/$
Disallow /feed
Disallow /comments/feed
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://donbalonrosa.defensacentral.com/sitemaps/sitemap.xml