fusilier.co.uk
robots.txt

Robots Exclusion Standard data for fusilier.co.uk

Archived Snapshots

Resource Scan

Scan Details

Site Domain	fusilier.co.uk
Base Domain	fusilier.co.uk
Scan Status	Ok
Last Scan	2025-10-19T16:06:31+00:00
Next Scan	2025-10-26T16:06:31+00:00

Last Scan

Scanned	2025-10-19T16:06:31+00:00
URL	https://fusilier.co.uk/robots.txt
Domain IPs	5.134.9.165
Response IP	5.134.9.165
Found	Yes
Hash	0a5a8ab82ad9db8d8bde3325e478906f887a0e3ccc7c7b7a2dafd3a77ae69167
SimHash	701f0313c08e

Groups

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

lwp-simple/5.65

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier v3.0.1

Rule	Path
Disallow	/

Rule

Path

Disallow

webripper

Rule	Path
Disallow	/

Rule

Path

Disallow

nicebot

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider+

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mauibot

Rule	Path
Disallow	/

Rule

Path

Disallow

webcollage/1.140

Rule	Path
Disallow	/

Rule

Path

Disallow

bulkdown pictures

Rule	Path
Disallow	/

Rule

Path

Disallow

webgobbler/1.2.6

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

chatgpt-user

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

google-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

google-cloudvertexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

anthropic-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

omgilibot

Rule	Path
Disallow	/

Rule

Path

Disallow

omgili

Rule	Path
Disallow	/

Rule

Path

Disallow

facebookbot

Rule	Path
Disallow	/

Rule

Path

Disallow

diffbot

Rule	Path
Disallow	/

Rule

Path

Disallow

duckassistbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ai2bot

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/

Rule

Path

Disallow

kangaroo bot

Rule	Path
Disallow	/

Rule

Path

Disallow

pangubot

Rule	Path
Disallow	/

Rule

Path

Disallow

imagesiftbot

Rule	Path
Disallow	/

Rule

Path

Disallow

perplexitybot

Rule	Path
Disallow	/

Rule

Path

Disallow

cohere-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

cohere-training-data-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalagent

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalfetcher

Rule	Path
Disallow	/

Rule

Path

Disallow

timpibot

Rule	Path
Disallow	/

Rule

Path

Disallow

webzio-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

youbot

Rule	Path
Disallow	/

Rule

Path

Disallow

Comments

Baiduspider

Warnings

2 invalid lines.

fusilier.co.ukrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

ia_archiver

lwp-simple/5.65

webcopier v3.0.1

webripper

nicebot

baiduspider

baiduspider/2.0

baiduspider+

ahrefsbot

mj12bot

semrushbot

dotbot

mauibot

webcollage/1.140

bulkdown pictures

webgobbler/1.2.6

ccbot

chatgpt-user

gptbot

google-extended

google-cloudvertexbot

applebot-extended

anthropic-ai

claudebot

omgilibot

omgili

facebookbot

diffbot

duckassistbot

ai2bot

bytespider

kangaroo bot

pangubot

imagesiftbot

perplexitybot

cohere-ai

cohere-training-data-crawler

meta-externalagent

meta-externalfetcher

timpibot

webzio-extended

youbot

Comments

Warnings

fusilier.co.uk
robots.txt