Log In Sign Up

type-r-owners.co.uk
robots.txt

Robots Exclusion Standard data for type-r-owners.co.uk

Resource Scan

Scan Details

Site Domain type-r-owners.co.uk
Base Domain type-r-owners.co.uk
Scan Status Ok
Last Scan2025-10-20T15:55:40+00:00
Next Scan 2025-10-27T15:55:40+00:00

Last Scan

Scanned2025-10-20T15:55:40+00:00
URL https://type-r-owners.co.uk/robots.txt
Domain IPs 104.21.87.91, 172.67.169.79, 2606:4700:3030::6815:575b, 2606:4700:3037::ac43:a94f
Response IP 172.67.169.79
Found Yes
Hash 78cd8da470f888a53a48b4027b4c56970b8755772428824124e978241f0ed013
SimHash 846355fcaa35

Groups

mediapartners-google*

Rule Path
Disallow

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

axmo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

booch

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

downloader

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

expired domain sleuth

Rule Path
Disallow /

franklin locator

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

grub

Rule Path
Disallow /

hughcrawler

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

lcabotaccept

Rule Path
Disallow /

iconsurf

Rule Path
Disallow /

iltrovatore-setaccio

Rule Path
Disallow /

indy library

Rule Path
Disallow /

iupui

Rule Path
Disallow /

kittiecentral

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

larbin

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

metatagrobot

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

nextgensearch

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

oracle ultra search

Rule Path
Disallow /

peerbot

Rule Path
Disallow /

pictureofinternet

Rule Path
Disallow /

plantynet

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

scspider

Rule Path
Disallow /

soft411

Rule Path
Disallow /

spider.acont.de

Rule Path
Disallow /

sqworm

Rule Path
Disallow /

ssm agent

Rule Path
Disallow /

tamu

Rule Path
Disallow /

theusefulbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

tutorial crawler

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webzip

Rule Path
Disallow /

zipppbot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wget

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

mozdex

Rule Path
Disallow /

*

Rule Path
Disallow /backups_homepage/
Disallow /bkp.phpBB2/
Disallow /cgi-bin/
Disallow /cotm/
Disallow /design2_imgs/
Disallow /links/
Disallow /gallery/
Disallow /howto/
Disallow /mmedia/
Disallow /phpBB2.bkp/
Disallow /public_docs/
Disallow /test/
Disallow /toolk/
Disallow /trstats/
Disallow /Videos/

*

Rule Path
Disallow /admin.html
Disallow /articles.php
Disallow /commonscripts.php
Disallow /design2.css
Disallow /index.old
Disallow /mmedia.php
Disallow /pagefooter.php
Disallow /pageheader.php
Disallow /sidemenu.php
Disallow /slideshow.php
Disallow /test.php
Disallow /troprocess.php
Disallow /viewgallery.php
Back to top

Comments

  • AUTHOR : SharkyUK
  • CREATED : 25Jan2006
  • UPDATED : 13Feb2007
  • -----------------------------------------------------------------------
  • the following lines disallow access to common spambots, trawlers, etc.
  • (should help reduce unnecessary bandwidth usage)
  • -----------------------------------------------------------------------
  • the following lines disallow access to specific directories and files
  • -----------------------------------------------------------------------
Back to top

Warnings

  • 1 invalid line.
Back to top