Log In Sign Up

uoflsports.com
robots.txt

Robots Exclusion Standard data for uoflsports.com

Resource Scan

Scan Details

Site Domain uoflsports.com
Base Domain uoflsports.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-08-19T22:43:43+00:00
Next Scan 2024-11-17T22:43:43+00:00

Last Successful Scan

Scanned2023-10-25T17:08:54+00:00
URL http://uoflsports.com/robots.txt
Domain IPs 67.192.28.17
Response IP 67.192.28.17
Found Yes
Hash eb52e388a12c4b90011847b5bef290e8d19156ea414e014f742e4def61b51bde
SimHash 6a14d2e2c0b1

Groups

blp_bbot/0.1

Rule Path
Disallow /

blp_bbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

psbot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

ask jeeves

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

jaxified

Rule Path
Disallow /

yeti

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

yesupbot

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

dotspotsbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

willow internet crawler by twotrees

Rule Path
Disallow /

largesmall crawler

Rule Path
Disallow /

spbot

Rule Path
Disallow /

mxbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

influencebot/0.9

Rule Path
Disallow /

kwaclebot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

msrbot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

yahoo-newscrawler

Rule Path
Disallow /

lycos_spider

Rule Path
Disallow /

yahoomobile/1.0

Rule Path
Disallow /

domaincrawler 1.0

Rule Path
Disallow /

yahoo pipes 1.0

Rule Path
Disallow /

kscrawler

Rule Path
Disallow /

synapse

Rule Path
Disallow /

yandex

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /admin/
Disallow /services/
Disallow /*.axd
Allow /

Other Records

Field Value
crawl-delay 2

inagist.com url crawler

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

lmspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

r6_feedfetcher

Rule Path
Disallow /

nu_tch-princeton

Rule Path
Disallow /

sheenbot

Rule Path
Disallow /

msr-isrccrawler

Rule Path
Disallow /

abby

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

purebot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

msubot

Rule Path
Disallow /

cyberpatrol sitecat webbot

Rule Path
Disallow /

diribot

Rule Path
Disallow /

envolk

Rule Path
Disallow /

fast enterprise crawler 6

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

postrank

Rule Path
Disallow /

hailoobot

Rule Path
Disallow /

agbot

Rule Path
Disallow /

unwindfetchor/1.0

Rule Path
Disallow /

voyager/1.0

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mozilla/5.0 (compatible; butterfly/1.0; +http://labs.topsy.com/butterfly/) gecko/2009032608 firefox/3.0.8

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

suggybot

Rule Path
Disallow /

lemurwebcrawler

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

www.integromedb.org/crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

siteimprovebot-crawler

Rule Path
Disallow /admin/
Disallow /services/
Disallow /*.axd
Disallow /*print%3Dtrue*
Allow /

powermapper

Rule Path
Disallow /admin/
Disallow /services/
Disallow /*.axd
Disallow /*print%3Dtrue*
Disallow /common/
Allow /

googlebot

Rule Path
Disallow /admin/
Disallow /services/
Disallow /*.axd
Disallow /*print%3Dtrue*
Disallow /common/
Allow /services/podcast_rss.ashx
Allow /

bingbot

Rule Path
Disallow /admin/
Disallow /images/
Disallow /admin/
Disallow /common/
Disallow /editor/
Disallow /services/
Disallow /site/
Disallow /*.js$
Disallow /*.css$
Disallow /*.jpg$
Disallow /*.gif$
Disallow /*.axd
Allow /documents/
Allow /

Other Records

Field Value
crawl-delay 30

msnbot

Rule Path
Disallow /admin/
Disallow /images/
Disallow /admin/
Disallow /common/
Disallow /editor/
Disallow /services/
Disallow /site/
Disallow /*.js$
Disallow /*.css$
Disallow /*.jpg$
Disallow /*.gif$
Disallow /*.axd
Allow /documents/
Allow /

Other Records

Field Value
crawl-delay 30

ltx71

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /admin/
Disallow /services/
Disallow /*.axd
Allow /

*

Rule Path
Disallow /common/
Disallow /images/
Disallow /documents/
Disallow /admin/
Disallow /services/
Disallow /site/
Disallow /hidden/
Disallow /*.js$
Disallow /*.css$
Disallow /*.jpg$
Disallow /*.gif$
Disallow /*.axd
Disallow /*print%3Dtrue*
Allow /

Other Records

Field Value
crawl-delay 30

heritrix

Rule Path
Disallow /admin/
Disallow /images/
Disallow /documents/
Disallow /admin/
Disallow /common/
Disallow /editor/
Disallow /services/
Disallow /site/
Disallow /*.axd
Allow /

Other Records

Field Value
crawl-delay 5

twitterbot/1.0

Rule Path
Allow /

mozilla/4.0+(compatible;+t-h-u-n-d-e-r-s-t-o-n-e)

Rule Path
Disallow /admin/
Disallow /services/
Disallow /site/
Disallow /common/
Disallow /*.axd
Allow /

Other Records

Field Value
crawl-delay 2

swiftbot

Rule Path
Disallow /admin/
Disallow /services/
Disallow /site/
Disallow /*.axd
Allow /

Other Records

Field Value
crawl-delay 3

gsa-crawler

Rule Path
Disallow /admin/
Disallow /common/
Disallow /services/
Disallow /*.axd
Allow /

Other Records

Field Value
crawl-delay 2
Back to top

Other Records

Field Value
sitemap https://uoflsports.com/sitemap.xml
Back to top

Warnings

  • 2 invalid lines.
Back to top