Log In Sign Up

clivetully.co.uk
robots.txt

Robots Exclusion Standard data for clivetully.co.uk

Resource Scan

Scan Details

Site Domain clivetully.co.uk
Base Domain clivetully.co.uk
Scan Status Ok
Last Scan2026-03-01T05:23:09+00:00
Next Scan 2026-03-08T05:23:09+00:00

Last Scan

Scanned2026-03-01T05:23:09+00:00
URL http://clivetully.co.uk/robots.txt
Domain IPs 178.18.126.53
Response IP 178.18.126.53
Found Yes
Hash 64ba6f9718d354d20a321c4e03cb57229b3c112cb1affaa8c2ee85b43fc6d1ed
SimHash 7557c110e605

Groups

mauibot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

indeedbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dataprovider.com

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

bitsightbot

Rule Path
Disallow /

meta-externalagent
meta-externalagent

Rule Path
Disallow /

meta-externalfetcher
meta-externalfetcher

Rule Path
Disallow /

mycentralaiscraperbot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

perplexity-user

Rule Path
Disallow /

claude-searchbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-user

Rule Path
Disallow /
Back to top

Warnings

  • 2 invalid lines.
Back to top