smartme.co.uk
robots.txt

Robots Exclusion Standard data for smartme.co.uk

Resource Scan

Scan Details

Site Domain smartme.co.uk
Base Domain smartme.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-29T08:34:58+00:00
Next Scan 2025-11-28T08:34:58+00:00

Last Successful Scan

Scanned2025-09-29T17:48:42+00:00
URL https://smartme.co.uk/robots.txt
Redirect https://www.smartme.co.uk/robots.txt
Redirect Domain www.smartme.co.uk
Redirect Base smartme.co.uk
Domain IPs 77.72.2.114
Redirect IPs 77.72.2.114
Response IP 77.72.2.114
Found Yes
Hash 0ea346646e2c51a21c919df0bf6d96ff8bad9d406dd18e7a00c92b1e58b0508d
SimHash 7c1d1947a292

Groups

*

Rule Path
Disallow /glossary
Disallow /q-and-a
Disallow /components
Disallow /js
Disallow /php
Disallow /contact-php
Disallow /login
Disallow /qa-update
Disallow /qa-delete
Disallow /qa-view
Disallow /qa-create

amazonbot
anthropic-ai
applebot-extended
bytespider
cohere-ai
cohere-training-data-crawler
chatgpt-user
claudebot
claude-web
ccbot
diffbot
facebookbot
googlebot-extended
gptbot
meltwater
meta-externalagent
oai-searchbot
omgili
omgilibot
perplexity-ai
perplexitybot
seekr
youbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.smartme.co.uk/sitemap.xml