/.well-known/

Log In Sign Up

мото-барахолка.рф
robots.txt

Robots Exclusion Standard data for мото-барахолка.рф

Archived Snapshots

Resource Scan

Scan Details

Site Domain	мото-барахолка.рф
Base Domain	мото-барахолка.рф
Scan Status	Ok
Last Scan	2024-11-02T12:43:13+00:00
Next Scan	2024-11-09T12:43:13+00:00

Last Scan

Scanned	2024-11-02T12:43:13+00:00
URL	https://мото-барахолка.рф/robots.txt
Redirect	http://xn----8sbaac0dgdsaewv2c.xn--p1ai/robots.txt
Domain IPs	104.21.18.152, 172.67.182.160, 2606:4700:3030::6815:1298, 2606:4700:3030::ac43:b6a0
Response IP	172.67.182.160
Found	Yes
Hash	4693491225795f3e39d026154e8d24e3a3a3aeffb506292c774c89cedcebf32b
SimHash	d1975272cba3

Groups

googlebot

Rule

Path

Allow

/

aipbot

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

alexibot

Rule

Path

Disallow

/

aqua_products

Rule

Path

Disallow

/

asterias

Rule

Path

Disallow

/

b2w/0.1

Rule

Path

Disallow

/

backdoorbot/1.0

Rule

Path

Disallow

/

becomebot

Rule

Path

Disallow

/

blowfish/1.0

Rule

Path

Disallow

/

bookmark search tool

Rule

Path

Disallow

/

botalot

Rule

Path

Disallow

/

botrighthere

Rule

Path

Disallow

/

builtbottough

Rule

Path

Disallow

/

bullseye/1.0

Rule

Path

Disallow

/

bunnyslippers

Rule

Path

Disallow

/

cheesebot

Rule

Path

Disallow

/

cherrypicker

Rule

Path

Disallow

/

cherrypickerelite/1.0

Rule

Path

Disallow

/

cherrypickerse/1.0

Rule

Path

Disallow

/

copernic

Rule

Path

Disallow

/

copyrightcheck

Rule

Path

Disallow

/

cosmos

Rule

Path

Disallow

/

crescent

Rule

Path

Disallow

/

crescent internet toolpak http ole control v.1.0

Rule

Path

Disallow

/

dittospyder

Rule

Path

Disallow

/

dotbot

Rule

Path

Disallow

/

emailcollector

Rule

Path

Disallow

/

emailsiphon

Rule

Path

Disallow

/

emailwolf

Rule

Path

Disallow

/

erocrawler

Rule

Path

Disallow

/

exabot

Rule

Path

Disallow

/

extractorpro

Rule

Path

Disallow

/

fairad client

Rule

Path

Disallow

/

fasterfox

Rule

Path

Disallow

/

flaming attackbot

Rule

Path

Disallow

/

foobot

Rule

Path

Disallow

/

gigabot

Rule

Path

Disallow

/

gaisbot

Rule

Path

Disallow

/

getright/4.2

Rule

Path

Disallow

/

harvest/1.5

Rule

Path

Disallow

/

hloader

Rule

Path

Disallow

/

httplib

Rule

Path

Disallow

/

httrack 3.0

Rule

Path

Disallow

/

humanlinks

Rule

Path

Disallow

/

infonavirobot

Rule

Path

Disallow

/

iron33/1.0.2

Rule

Path

Disallow

/

jennybot

Rule

Path

Disallow

/

kenjin spider

Rule

Path

Disallow

/

keyword density/0.9

Rule

Path

Disallow

/

larbin

Rule

Path

Disallow

/

lexibot

Rule

Path

Disallow

/

libweb/clshttp

Rule

Path

Disallow

/

linkextractorpro

Rule

Path

Disallow

/

linkscan/8.1a unix

Rule

Path

Disallow

/

linkwalker

Rule

Path

Disallow

/

lnspiderguy

Rule

Path

Disallow

/

lwp-trivial

Rule

Path

Disallow

/

lwp-trivial/1.34

Rule

Path

Disallow

/

mata hari

Rule

Path

Disallow

/

microsoft url control

Rule

Path

Disallow

/

microsoft url control ð â ðâ ð â ð²ðâ ð â ðâ ð ðð âð¡â¢ð â ð â ð â ð²ðñð ðð¡ñ 5.01.4511

Rule

Path

Disallow

/

microsoft url control ð â ðâ ð â ð²ðâ ð â ðâ ð ðð âð¡â¢ð â ð â ð â ð²ðñð ðð¡ñ 6.00.8169

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

miixpc

Rule

Path

Disallow

/

miixpc/4.2

Rule

Path

Disallow

/

mister pix

Rule

Path

Disallow

/

moget

Rule

Path

Disallow

/

moget/2.1

Rule

Path

Disallow

/

mozilla/4.0 (compatible; bullseye; windows 95)

Rule

Path

Disallow

/

msiecrawler

Rule

Path

Disallow

/

netants

Rule

Path

Disallow

/

nicerspro

Rule

Path

Disallow

/

offline explorer

Rule

Path

Disallow

/

openbot

Rule

Path

Disallow

/

openfind

Rule

Path

Disallow

/

openfind data gatherer

Rule

Path

Disallow

/

oracle ultra search

Rule

Path

Disallow

/

perman

Rule

Path

Disallow

/

propowerbot/2.14

Rule

Path

Disallow

/

prowebwalker

Rule

Path

Disallow

/

psbot

Rule

Path

Disallow

/

python-urllib

Rule

Path

Disallow

/

queryn metasearch

Rule

Path

Disallow

/

radiation retriever 1.1

Rule

Path

Disallow

/

repomonkey

Rule

Path

Disallow

/

repomonkey bait & tackle/v1.01

Rule

Path

Disallow

/

rma

Rule

Path

Disallow

/

rogerbot

Rule

Path

Disallow

/

searchpreview

Rule

Path

Disallow

/

sitesnagger

Rule

Path

Disallow

/

spankbot

Rule

Path

Disallow

/

spanner

Rule

Path

Disallow

/

surveybot

Rule

Path

Disallow

/

suzuran

Rule

Path

Disallow

/

szukacz/1.4

Rule

Path

Disallow

/

teleport

Rule

Path

Disallow

/

teleportpro

Rule

Path

Disallow

/

telesoft

Rule

Path

Disallow

/

the intraformant

Rule

Path

Disallow

/

thenomad

Rule

Path

Disallow

/

tighttwatbot

Rule

Path

Disallow

/

tocrawl/urldispatcher

Rule

Path

Disallow

/

true_robot

Rule

Path

Disallow

/

true_robot/1.0

Rule

Path

Disallow

/

turingos

Rule

Path

Disallow

/

turnitinbot

Rule

Path

Disallow

/

turnitinbot/1.5

Rule

Path

Disallow

/

url control

Rule

Path

Disallow

/

url_spider_pro

Rule

Path

Disallow

/

urly warning

Rule

Path

Disallow

/

vci

Rule

Path

Disallow

/

vci webviewer vci webviewer win32

Rule

Path

Disallow

/

web image collector

Rule

Path

Disallow

/

webauto

Rule

Path

Disallow

/

webbandit

Rule

Path

Disallow

/

webbandit/3.50

Rule

Path

Disallow

/

webcapture 2.0

Rule

Path

Disallow

/

webcopier

Rule

Path

Disallow

/

webcopier v.2.2

Rule

Path

Disallow

/

webcopier v3.2a

Rule

Path

Disallow

/

webenhancer

Rule

Path

Disallow

/

websauger

Rule

Path

Disallow

/

website quester

Rule

Path

Disallow

/

webster pro

Rule

Path

Disallow

/

webstripper

Rule

Path

Disallow

/

webzip

Rule

Path

Disallow

/

webzip

Rule

Path

Disallow

/

webzip/4.0

Rule

Path

Disallow

/

webzip/4.21

Rule

Path

Disallow

/

webzip/5.0

Rule

Path

Disallow

/

wget

Rule

Path

Disallow

/

wget

Rule

Path

Disallow

/

wget/1.5.3

Rule

Path

Disallow

/

wget/1.6

Rule

Path

Disallow

/

www-collector-e

Rule

Path

Disallow

/

xenuð â ðâ ð â ð²ðâ ð â ðâ ð ðð âð¡â¢ð â ð â ð ðð âð¡âºð ðð¡âºs

Rule

Path

Disallow

/

xenuð â ðâ ð â ð²ðâ ð â ðâ ð ðð âð¡â¢ð â ð â ð ðð âð¡âºð ðð¡âºs link sleuth 1.1c

Rule

Path

Disallow

/

zeus

Rule

Path

Disallow

/

zeus 32297 webster pro v2.9 win32

Rule

Path

Disallow

/

zeus link scout

Rule

Path

Disallow

/

megaindex

Rule

Path

Disallow

/

baiduspider

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

runet-research-crawler

Rule

Path

Disallow

/

sogou

Rule

Path

Disallow

/

python

Rule

Path

Disallow

/

pingdom

Rule

Path

Disallow

/

riddler

Rule

Path

Disallow

/

java

Rule

Path

Disallow

/

njuicebot

Rule

Path

Disallow

/

js-kit

Rule

Path

Disallow

/

voyager

Rule

Path

Disallow

/

postrank

Rule

Path

Disallow

/

pycurl

Rule

Path

Disallow

/

aport

Rule

Path

Disallow

/

surveybot

Rule

Path

Disallow

/

butterfly

Rule

Path

Disallow

/

libwww

Rule

Path

Disallow

/

sweb

Rule

Path

Disallow

/

linkexchanger

Rule

Path

Disallow

/

soup

Rule

Path

Disallow

/

wordpress

Rule

Path

Disallow

/

spbot

Rule

Path

Disallow

/

mlbot

Rule

Path

Disallow

/

internetseer

Rule

Path

Disallow

/

fairshare

Rule

Path

Disallow

/

yeti

Rule

Path

Disallow

/

birubot

Rule

Path

Disallow

/

yottosbot

Rule

Path

Disallow

/

linguee

Rule

Path

Disallow

/

ezooms

Rule

Path

Disallow

/

purebot

Rule

Path

Disallow

/

kmsearchbot

Rule

Path

Disallow

/

sitebot

Rule

Path

Disallow

/

camontspider

Rule

Path

Disallow

/

ptd-crawler

Rule

Path

Disallow

/

suggybot

Rule

Path

Disallow

/

ttcrawler

Rule

Path

Disallow

/

nutch

Rule

Path

Disallow

/

Back to top