datalog.co.uk
robots.txt

Robots Exclusion Standard data for datalog.co.uk

Resource Scan

Scan Details

Site Domain datalog.co.uk
Base Domain datalog.co.uk
Scan Status Ok
Last Scan2025-10-02T03:07:33+00:00
Next Scan 2025-10-09T03:07:33+00:00

Last Scan

Scanned2025-10-02T03:07:33+00:00
URL https://datalog.co.uk/robots.txt
Domain IPs 92.24.2.52
Response IP 92.24.2.52
Found Yes
Hash a1aa3b3191206fd0a88accee58bb69ce9bfd67fead896afc31c6b647f981fe55
SimHash 620f56e0d211

Groups

gptbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

amazonbot/0.1

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot/v1.4.8

Rule Path
Disallow /

ahrefsbot/6.1

Rule Path
Disallow /

ahrefsbot/7.0

Rule Path
Disallow /

qwantify/2.4w

Rule Path
Disallow /

dotbot/1.1

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

emailmarketingrobot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

Other Records

Field Value
sitemap http://www.datalog.co.uk/sitemap/sitemap_index_2017-01-01.xml