Log In Sign Up

howtocreate.co.uk
robots.txt

Robots Exclusion Standard data for howtocreate.co.uk

Resource Scan

Scan Details

Site Domain howtocreate.co.uk
Base Domain howtocreate.co.uk
Scan Status Ok
Last Scan2024-09-22T00:24:30+00:00
Next Scan 2024-10-22T00:24:30+00:00

Last Scan

Scanned2024-09-22T00:24:30+00:00
URL https://www.howtocreate.co.uk/robots.txt
Domain IPs 49.13.230.79
Response IP 49.13.230.79
Found Yes
Hash f9b174ba4fc90804646c61e63308c8b182acd9783f10db13d15f0239eaf71dea
SimHash 631d49ab4491

Groups

googlebot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

slurp

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

msnbot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

teoma

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

fast

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

surveybot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

mediapartners-google

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

robozilla

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

openbot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

openfind

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

pompos

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

looksmart

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

scooter

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

voilabot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

mozilla/4.0 (compatible; msie 5.0; windows 95) voilabot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

mirago

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

grub-client

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

nutch

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

mj12bot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

geonabot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

it-bot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

iltrovatore-setaccio

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

peerbot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

gridbot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

fyberspider

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

asterias

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

bilbo

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

vagabondo

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

mojeekbot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

docomo

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

webrover

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

netnosecrawler

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

mozilla/4.0 (compatible; msie 5.0; netnose-crawler

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

wotbox

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

webyield robot

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

spidermonkey

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

scrubby

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

overture-webcrawler

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

orcaspider

Rule Path
Disallow /noentry
Disallow /working
Disallow /tutorials
Disallow /jslibs
Disallow /operaStyles
Disallow /emails
Disallow /php
Disallow /wrongWithIE
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

http://www.almaden.ibm.com/cs/crawler

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

w3c-checklink

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

w3c_validator

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

ia_archiver

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php

msiecrawler

Rule Path
Disallow /

snappreviewbot

Rule Path
Disallow /

*

Rule Path
Disallow /noentry
Disallow /working
Disallow /php/pushimage.php
Disallow /jslibs/oldindex.php
Back to top

Comments

  • major
  • Yahoo
  • Ask Jeeves
  • dmoz
  • localised
  • distributed/open source - allowed
  • minor - allowed
  • private - allowed
  • special use - allowed
  • IE favourite saving to depth - denied
  • Snap thumbnails - not a search engine, and ignores default rule - denied
  • Default rule
Back to top