Log In Sign Up

missingin.org
robots.txt

Robots Exclusion Standard data for missingin.org

Resource Scan

Scan Details

Site Domain missingin.org
Base Domain missingin.org
Scan Status Ok
Last Scan2024-07-03T20:24:16+00:00
Next Scan 2024-07-10T20:24:16+00:00

Last Scan

Scanned2024-07-03T20:24:16+00:00
URL http://missingin.org/robots.txt
Domain IPs 209.15.249.2
Response IP 209.15.249.2
Found Yes
Hash 95e5edc414771644553f573faf05bf04cba85ff4c828116b0fac7eb662dd75fa
SimHash a32712028dd3

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /cgi-bin/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 4

a1 website download

Rule Path
Disallow /

accelobot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow

accelovation

Rule Path
Disallow /

amazon

Rule Path
Disallow /

amazonaws.com

Rule Path
Disallow /

archive.org

Rule Path
Disallow /

archive.orgbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

atraxbot

Rule Path
Disallow /

attrakt

Rule Path
Disallow /

attributor

Rule Path
Disallow /

cloudacl

Rule Path
Disallow /

creative autoupdate

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

ccbot/1.0

Rule Path
Disallow /

charlotte.betaspider

Rule Path
Disallow /

cogent

Rule Path
Disallow /

comodospider

Rule Path
Disallow /

crawler.archive.org

Rule Path
Disallow /

discobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotnetdotcom

Rule Path
Disallow /

dropdowndeals

Rule Path
Disallow /

findfiles.net/0.98

Rule Path
Disallow /

getleft 1.2

Rule Path
Disallow /

gootkit auto-rooter scanner

Rule Path
Disallow /

heratrix

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

heretix

Rule Path
Disallow /

heratix

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ichiro/2.0

Rule Path
Disallow /

ilial

Rule Path
Disallow /

influencebot/0.9

Rule Path
Disallow /

itsapic.com_crawler

Rule Path
Disallow /

kimsufi.com

Rule Path
Disallow /

lemurproject nutch spider

Rule Path
Disallow /

litefinder

Rule Path
Disallow /

lssbot.com

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

msr-isrccrawler

Rule Path
Disallow /

msrputnik

Rule Path
Disallow /

mvk-it.com

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

openx

Rule Path
Disallow /

pagenest

Rule Path
Disallow /

proxyway.com

Rule Path
Disallow /

ptd-crawler

Rule Path
Disallow /

purebot

Rule Path
Disallow /

qbikspider

Rule Path
Disallow /

ripper

Rule Path
Disallow /

sbider/sbider-0.8.2-dev

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

semrushbot/0.9

Rule Path
Disallow /

shablastbot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sixtrix

Rule Path
Disallow /

softbytelabs.com

Rule Path
Disallow /

sogou

Rule Path
Disallow /

spider06.yandex.ru

Rule Path
Disallow /

squid-prefetch

Rule Path
Disallow /

superbot

Rule Path
Disallow /

swish

Rule Path
Disallow /

swish-e

Rule Path
Disallow /

twtelecom.net

Rule Path
Disallow /

webalta

Rule Path
Disallow /

webpix

Rule Path
Disallow /

webster pro v3.4

Rule Path
Disallow /

wells search ii

Rule Path
Disallow /

wep

Rule Path
Disallow /

wget

Rule Path
Disallow /

wget/1.9

Rule Path
Disallow /

wget/1.10 devel

Rule Path
Disallow /

wget/1.10.2

Rule Path
Disallow /

wget/1.12

Rule Path
Disallow /

winhttp

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wow64

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

yanga worldsearch bot v1.1/beta

Rule Path
Disallow /

yeti

Rule Path
Disallow /

your-server.de

Rule Path
Disallow /

zermelo

Rule Path
Disallow /
Back to top

Warnings

  • 6 invalid lines.
Back to top