bgmcd.co.uk
robots.txt

Robots Exclusion Standard data for bgmcd.co.uk

Resource Scan

Scan Details

Site Domain bgmcd.co.uk
Base Domain bgmcd.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-02-01T13:35:55+00:00
Next Scan 2026-02-15T13:35:55+00:00

Last Successful Scan

Scanned2025-12-25T12:50:41+00:00
URL http://bgmcd.co.uk/robots.txt
Domain IPs 52.16.25.241
Response IP 52.16.25.241
Found Yes
Hash 89704d1137aabeec93afa037dc8dbb403ff6af09f464c977e16e2a1b28712f1b
SimHash 4057f512211b

Groups

baiduspider

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /*?

googlebot

Rule Path
Disallow /*.html?$

googlebot

Rule Path
Disallow /*.php$

googlebot

Rule Path
Disallow /*.aspx?$

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

applebot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

anthropicbot

Rule Path
Disallow /

claude

Rule Path
Disallow /

claudebot

Rule Path
Disallow /