finance.co.uk
robots.txt

Robots Exclusion Standard data for finance.co.uk

Resource Scan

Scan Details

Site Domain finance.co.uk
Base Domain finance.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-05T07:06:15+00:00
Next Scan 2024-12-05T07:06:15+00:00

Last Successful Scan

Scanned2024-10-07T07:05:07+00:00
URL https://finance.co.uk/robots.txt
Redirect https://www.finance.co.uk/robots.txt
Redirect Domain www.finance.co.uk
Redirect Base finance.co.uk
Domain IPs 35.178.252.65, 52.56.211.198
Redirect IPs 35.178.252.65, 52.56.211.198
Response IP 52.56.211.198
Found Yes
Hash fbb6c87909a6967a4f212c0953de1f1360988b70dbc08d417d0c8fe847109f00
SimHash 431cd472ca9a

Groups

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

dataforseobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.finance.co.uk/sitemap_index.xml