Log In Sign Up

designcrowd.co.uk
robots.txt

Robots Exclusion Standard data for designcrowd.co.uk

Resource Scan

Scan Details

Site Domain designcrowd.co.uk
Base Domain designcrowd.co.uk
Scan Status Ok
Last Scan2024-06-24T04:56:44+00:00
Next Scan 2024-07-08T04:56:44+00:00

Last Scan

Scanned2024-06-24T04:56:44+00:00
URL https://designcrowd.co.uk/robots.txt
Redirect https://www.designcrowd.co.uk:443/robots.txt
Redirect Domain www.designcrowd.co.uk
Redirect Base designcrowd.co.uk
Domain IPs 52.84.229.14, 52.84.229.17, 52.84.229.51, 52.84.229.56
Redirect IPs 52.84.229.14, 52.84.229.17, 52.84.229.51, 52.84.229.56
Response IP 52.84.229.56
Found Yes
Hash a69d6f589d25c166bfffac38042dc611a73c819d500891be10eacf5761b898cb
SimHash 4280fb6702c1

Groups

trovitbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

seoengbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

spbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

obot

Rule Path
Disallow /

changedetection

Rule Path
Disallow /

slurp

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

proximic

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

adbeat_bot

Rule Path
Disallow /

peerindex

Rule Path
Disallow /

genieo

Rule Path
Disallow /

www.integromedb.org/crawler

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

kraken/0.1

Rule Path
Disallow /

steeler

Rule Path
Disallow /

flipboard

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

zenbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

jooblebot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

netseer

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

znajdzfoto

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

griffinbot

Rule Path
Disallow /

deusu

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

femtosearchbot

Rule Path
Disallow /

buckyohare

Rule Path
Disallow /

semanticscholarbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

bubing

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

applebot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

lcc

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

linespider

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

*

Rule Path
Disallow /cgi-bin/
Disallow /admin/
Disallow /*.axd
Disallow /WebResource.axd
Disallow /ScriptResource.axd
Disallow /search.aspx*
Disallow /report-issue-re-design.aspx?*
Disallow /manage-project/
Disallow /manage-account/
Disallow /designer-account/
Disallow /manage-profile/
Disallow /customer-handover/
Disallow /designer-handover/
Disallow /submit-design/
Disallow /login.aspx?returnUrl*
Disallow /login?returnUrl*
Disallow /s/
Disallow /poll/
Disallow /manage-poll/
Disallow /vote/
Back to top

Other Records

Field Value
sitemap https://www.designcrowd.co.uk/sitemap-index.xml
Back to top

Comments

  • robots.txt
Back to top

Warnings

  • 4 invalid lines.
Back to top