Log In Sign Up

ubuffalo.sidearmsports.com
robots.txt

Robots Exclusion Standard data for ubuffalo.sidearmsports.com

Resource Scan

Scan Details

Site Domain ubuffalo.sidearmsports.com
Base Domain sidearmsports.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-27T04:28:32+00:00
Next Scan 2024-11-25T04:28:32+00:00

Last Successful Scan

Scanned2023-05-06T03:36:49+00:00
URL https://ubuffalo.sidearmsports.com/robots.txt
Redirect https://ubbulls.com/robots.txt
Redirect Domain ubbulls.com
Redirect Base ubbulls.com
Domain IPs 74.205.57.15
Redirect IPs 67.192.28.17
Response IP 67.192.28.17
Found Yes
Hash 75474214528424c4921c406e21a0c8b7a5530de5d552091809a9df6ce8b75c2f
SimHash 6a14d2e2c2b1

Groups

blp_bbot/0.1

Rule Path
Disallow /

blp_bbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

psbot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

ask jeeves

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

jaxified

Rule Path
Disallow /

yeti

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

yesupbot

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

dotspotsbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

willow internet crawler by twotrees

Rule Path
Disallow /

largesmall crawler

Rule Path
Disallow /

spbot

Rule Path
Disallow /

mxbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

influencebot/0.9

Rule Path
Disallow /

kwaclebot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

msrbot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

yahoo-newscrawler

Rule Path
Disallow /

lycos_spider

Rule Path
Disallow /

yahoomobile/1.0

Rule Path
Disallow /

domaincrawler 1.0

Rule Path
Disallow /

yahoo pipes 1.0

Rule Path
Disallow /

kscrawler

Rule Path
Disallow /

synapse

Rule Path
Disallow /

yandex

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /admin/
Disallow /services/
Disallow /*.axd
Allow /

Other Records

Field Value
crawl-delay 2

inagist.com url crawler

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

lmspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

r6_feedfetcher

Rule Path
Disallow /

nu_tch-princeton

Rule Path
Disallow /

sheenbot

Rule Path
Disallow /

msr-isrccrawler

Rule Path
Disallow /

abby

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

purebot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

msubot

Rule Path
Disallow /

cyberpatrol sitecat webbot

Rule Path
Disallow /

diribot

Rule Path
Disallow /

envolk

Rule Path
Disallow /

fast enterprise crawler 6

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

postrank

Rule Path
Disallow /

hailoobot

Rule Path
Disallow /

agbot

Rule Path
Disallow /

unwindfetchor/1.0

Rule Path
Disallow /

voyager/1.0

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mozilla/5.0 (compatible; butterfly/1.0; +http://labs.topsy.com/butterfly/) gecko/2009032608 firefox/3.0.8

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

suggybot

Rule Path
Disallow /

lemurwebcrawler

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

www.integromedb.org/crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

siteimprovebot-crawler

Rule Path
Disallow /admin/
Disallow /services/
Disallow /*.axd
Disallow /*print%3Dtrue*
Allow /

powermapper

Rule Path
Disallow /admin/
Disallow /services/
Disallow /*.axd
Disallow /*print%3Dtrue*
Disallow /common/
Allow /

googlebot

Rule Path
Disallow /admin/
Disallow /services/
Disallow /*.axd
Disallow /*print%3Dtrue*
Disallow /common/
Allow /services/podcast_rss.ashx
Allow /

bingbot

Rule Path
Disallow /admin/
Disallow /images/
Disallow /admin/
Disallow /common/
Disallow /editor/
Disallow /services/
Disallow /site/
Disallow /*.js$
Disallow /*.css$
Disallow /*.jpg$
Disallow /*.gif$
Disallow /*.axd
Allow /documents/
Allow /

Other Records

Field Value
crawl-delay 30

msnbot

Rule Path
Disallow /admin/
Disallow /images/
Disallow /admin/
Disallow /common/
Disallow /editor/
Disallow /services/
Disallow /site/
Disallow /*.js$
Disallow /*.css$
Disallow /*.jpg$
Disallow /*.gif$
Disallow /*.axd
Allow /documents/
Allow /

Other Records

Field Value
crawl-delay 30

ltx71

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /admin/
Disallow /services/
Disallow /*.axd
Allow /

*

Rule Path
Disallow /common/
Disallow /images/
Disallow /documents/
Disallow /admin/
Disallow /services/
Disallow /site/
Disallow /hidden/
Disallow /*.js$
Disallow /*.css$
Disallow /*.jpg$
Disallow /*.gif$
Disallow /*.axd
Disallow /*print%3Dtrue*
Allow /

Other Records

Field Value
crawl-delay 30

heritrix

Rule Path
Disallow /admin/
Disallow /images/
Disallow /documents/
Disallow /admin/
Disallow /common/
Disallow /editor/
Disallow /services/
Disallow /site/
Disallow /*.axd
Allow /

Other Records

Field Value
crawl-delay 5

twitterbot/1.0

Rule Path
Allow /

mozilla/4.0+(compatible;+t-h-u-n-d-e-r-s-t-o-n-e)

Rule Path
Disallow /admin/
Disallow /services/
Disallow /site/
Disallow /common/
Disallow /*.axd
Allow /

Other Records

Field Value
crawl-delay 2

swiftbot

Rule Path
Disallow /admin/
Disallow /services/
Disallow /site/
Disallow /*.axd
Allow /

Other Records

Field Value
crawl-delay 3

gsa-crawler

Rule Path
Disallow /admin/
Disallow /common/
Disallow /services/
Disallow /*.axd
Allow /

Other Records

Field Value
crawl-delay 2
Back to top

Warnings

  • 2 invalid lines.
Back to top