europapress.tv
robots.txt

Robots Exclusion Standard data for europapress.tv

Resource Scan

Scan Details

Site Domain europapress.tv
Base Domain europapress.tv
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-02-28T22:01:55+00:00
Next Scan 2024-05-28T22:01:55+00:00

Last Successful Scan

Scanned2023-11-01T20:40:01+00:00
URL https://www.europapress.tv/robots.txt
Domain IPs 138.199.44.209, 185.103.37.72, 185.103.37.73, 51.210.0.138, 51.89.172.162
Response IP 51.210.0.138
Found Yes
Hash 16c88553544f6915800d11010ad37731a7a16526cfae235b80d67e7783e62640
SimHash fa1c4253edb9

Groups

*

Rule Path
Disallow /api/*
Disallow /buscar/
Disallow /buscar?
Disallow /archivo/
Disallow /archivo?
Allow /

yandex

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

freshbot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

trendictionbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

grapeshot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

heritrix

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.europapress.tv/sitemap.xml