Log In Sign Up

ballotpedia.org
robots.txt

Robots Exclusion Standard data for ballotpedia.org

Resource Scan

Scan Details

Site Domain ballotpedia.org
Base Domain ballotpedia.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-05T02:49:27+00:00
Next Scan 2025-01-03T02:49:27+00:00

Last Successful Scan

Scanned2024-06-08T02:45:05+00:00
URL https://ballotpedia.org/robots.txt
Domain IPs 3.213.98.57, 34.204.141.218, 52.205.137.186
Response IP 3.213.98.57
Found Yes
Hash 9ee7e3173b95264f16694aea98cf37fcdb81b3e9b29352dc4de5305bda463cc5
SimHash 7242c147c85b

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /wiki/images/
Disallow /tmp/
Disallow /private/
Disallow /phpshell-2.1/
Disallow /fckeditor/
Disallow /ballotpe/
Disallow /ballotpedia.org/
Disallow /cache/
Disallow /images/
Disallow /openx/
Disallow /samftp/
Disallow /winkmv77/
Disallow /wiki/bin/
Disallow /wiki/config/
Disallow /wiki/docs/
Disallow /wiki/extensions/
Disallow /wiki/htmlets/
Disallow /wiki/includes/
Disallow /wiki/languages/
Disallow /wiki/locale/
Disallow /wiki/maintenance/
Disallow /wiki/math/
Disallow /wiki/t/
Disallow /wiki/tests/
Disallow /wiki/index.php/User%3A*
Disallow /Wikipedia%3A*
Disallow /wiki/index.php/Wikipedia%3A*
Disallow /wiki/index.php?title=Wikipedia%3A*
Disallow /User%3A

squider

Rule Path
Disallow /

updown_tester

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

komodiabot

Rule Path
Disallow /

aboundex

Rule Path
Disallow /

jakarta

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

obot

Rule Path
Disallow /

crawldaddy

Rule Path
Disallow /

nutch

Rule Path
Disallow /

nutch

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

blogsearch

Rule Path
Disallow /

cloudservermarketspider

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

zemlyacrawl

Rule Path
Disallow /

webtarantula.com crawler

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

seo robot

Rule Path
Disallow /

cms crawler

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

screenerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blog search

Rule Path
Disallow /

xenu link sleuth

Rule Path
Disallow /

webcapture

Rule Path
Disallow /

lssrocketcrawler

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou

Rule Path
Disallow /

webmasteraid

Rule Path
Disallow /

larbin

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linkdexbot-mobile

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

webster

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

netseer

Rule Path
Disallow /

exabot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

unwindfetchor

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

moget

Rule Path
Disallow /

abot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

urlfilterdb-crawler

Rule Path
Disallow /

urlfilterdb

Rule Path
Disallow /

ufdb

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

netestate

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

seostats

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

yyspider

Rule Path
Disallow /

yrspider

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

aboutusbot

Rule Path
Disallow /

beetlebot

Rule Path
Disallow /

babya discoverer

Rule Path
Disallow /

hivabot

Rule Path
Disallow /

niki-bot

Rule Path
Disallow /

inagist

Rule Path
Disallow /

percolatecrawler

Rule Path
Disallow /

jetslide

Rule Path
Disallow /

newsme

Rule Path
Disallow /

eventmachine

Rule Path
Disallow /

openwebspider

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

httrack

Rule Path
Disallow /

nigma.ru

Rule Path
Disallow /

thumbnailagent

Rule Path
Disallow /

feedbot

Rule Path
Disallow /

spiderling

Rule Path
Disallow /

betasearch

Rule Path
Disallow /

academic beta search

Rule Path
Disallow /

wget

Rule Path
Disallow /

netshelter contentscan

Rule Path
Disallow /

netshelter

Rule Path
Disallow /

madaali.de

Rule Path
Disallow /

proximic

Rule Path
Disallow /

powermarks

Rule Path
Disallow /

wminer

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

lipperhey seo service

Rule Path
Disallow /

industry cortex webcrawler

Rule Path
Disallow /

wscheck.com

Rule Path
Disallow /

symfony spider

Rule Path
Disallow /

perviibot

Rule Path
Disallow /

webinatorbot

Rule Path
Disallow /

lbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

p4bot

Rule Path
Disallow /

prlog

Rule Path
Disallow /

domnutch-bot

Rule Path
Disallow /

prooxibot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

adnormcrawler

Rule Path
Disallow /

kulturarw3

Rule Path
Disallow /

googlebot-news

Rule Path
Disallow /
Allow /Fact_check
Back to top

Other Records

Field Value
sitemap /wiki/sitemap/sitemap-index-ballotpedia.xml
Back to top

Comments

  • Disallow: /wiki/skins/
  • Disallow: /wiki/index.php/
  • Crawl-delay: 5
  • Request-rate: 1/5 # maximum rate is one page every 5 seconds
  • Visit-time: 0600-0845 # only visit between 06:00 and 08:45 UTC (GMT)
  • User-agent: Slurp
  • Disallow: /
  • --------------------------
Back to top

Warnings

  • 4 invalid lines.
Back to top