/.well-known/

Log In Sign Up

территориясмыслов.рф
robots.txt

Robots Exclusion Standard data for территориясмыслов.рф

Archived Snapshots

Resource Scan

Scan Details

Site Domain	территориясмыслов.рф
Base Domain	территориясмыслов.рф
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Server returned a client error.
Last Scan	2024-08-23T02:28:48+00:00
Next Scan	2024-11-21T02:28:48+00:00

Last Successful Scan

Scanned	2024-01-27T01:47:16+00:00
URL	https://территориясмыслов.рф/robots.txt
Redirect	https://xn--b1afjapfmdmacnbee3mrc.xn--p1ai/robots.txt
Domain IPs	104.21.19.182, 172.67.187.14, 2606:4700:3034::6815:13b6, 2606:4700:3036::ac43:bb0e
Response IP	104.21.19.182
Found	Yes
Hash	9793223a1f6cab442e4647028dfa42ea7cd56ca9d2d75751f67d2150bf7d8580
SimHash	d1965262cba3

Groups

googlebot

Rule

Path

Allow

/

aipbot

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

alexibot

Rule

Path

Disallow

/

aqua_products

Rule

Path

Disallow

/

asterias

Rule

Path

Disallow

/

b2w/0.1

Rule

Path

Disallow

/

backdoorbot/1.0

Rule

Path

Disallow

/

becomebot

Rule

Path

Disallow

/

blowfish/1.0

Rule

Path

Disallow

/

bookmark search tool

Rule

Path

Disallow

/

botalot

Rule

Path

Disallow

/

botrighthere

Rule

Path

Disallow

/

builtbottough

Rule

Path

Disallow

/

bullseye/1.0

Rule

Path

Disallow

/

bunnyslippers

Rule

Path

Disallow

/

cheesebot

Rule

Path

Disallow

/

cherrypicker

Rule

Path

Disallow

/

cherrypickerelite/1.0

Rule

Path

Disallow

/

cherrypickerse/1.0

Rule

Path

Disallow

/

copernic

Rule

Path

Disallow

/

copyrightcheck

Rule

Path

Disallow

/

cosmos

Rule

Path

Disallow

/

crescent

Rule

Path

Disallow

/

crescent internet toolpak http ole control v.1.0

Rule

Path

Disallow

/

dittospyder

Rule

Path

Disallow

/

dotbot

Rule

Path

Disallow

/

emailcollector

Rule

Path

Disallow

/

emailsiphon

Rule

Path

Disallow

/

emailwolf

Rule

Path

Disallow

/

erocrawler

Rule

Path

Disallow

/

exabot

Rule

Path

Disallow

/

extractorpro

Rule

Path

Disallow

/

fairad client

Rule

Path

Disallow

/

fasterfox

Rule

Path

Disallow

/

flaming attackbot

Rule

Path

Disallow

/

foobot

Rule

Path

Disallow

/

gigabot

Rule

Path

Disallow

/

gaisbot

Rule

Path

Disallow

/

getright/4.2

Rule

Path

Disallow

/

harvest/1.5

Rule

Path

Disallow

/

hloader

Rule

Path

Disallow

/

httplib

Rule

Path

Disallow

/

httrack 3.0

Rule

Path

Disallow

/

humanlinks

Rule

Path

Disallow

/

infonavirobot

Rule

Path

Disallow

/

iron33/1.0.2

Rule

Path

Disallow

/

jennybot

Rule

Path

Disallow

/

kenjin spider

Rule

Path

Disallow

/

keyword density/0.9

Rule

Path

Disallow

/

larbin

Rule

Path

Disallow

/

lexibot

Rule

Path

Disallow

/

libweb/clshttp

Rule

Path

Disallow

/

linkextractorpro

Rule

Path

Disallow

/

linkscan/8.1a unix

Rule

Path

Disallow

/

linkwalker

Rule

Path

Disallow

/

lnspiderguy

Rule

Path

Disallow

/

lwp-trivial

Rule

Path

Disallow

/

lwp-trivial/1.34

Rule

Path

Disallow

/

mata hari

Rule

Path

Disallow

/

microsoft url control

Rule

Path

Disallow

/

microsoft url control — 5.01.4511

Rule

Path

Disallow

/

microsoft url control — 6.00.8169

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

miixpc

Rule

Path

Disallow

/

miixpc/4.2

Rule

Path

Disallow

/

mister pix

Rule

Path

Disallow

/

moget

Rule

Path

Disallow

/

moget/2.1

Rule

Path

Disallow

/

mozilla/4.0 (compatible; bullseye; windows 95)

Rule

Path

Disallow

/

msiecrawler

Rule

Path

Disallow

/

netants

Rule

Path

Disallow

/

nicerspro

Rule

Path

Disallow

/

offline explorer

Rule

Path

Disallow

/

openbot

Rule

Path

Disallow

/

openfind

Rule

Path

Disallow

/

openfind data gatherer

Rule

Path

Disallow

/

oracle ultra search

Rule

Path

Disallow

/

perman

Rule

Path

Disallow

/

propowerbot/2.14

Rule

Path

Disallow

/

prowebwalker

Rule

Path

Disallow

/

psbot

Rule

Path

Disallow

/

python-urllib

Rule

Path

Disallow

/

queryn metasearch

Rule

Path

Disallow

/

radiation retriever 1.1

Rule

Path

Disallow

/

repomonkey

Rule

Path

Disallow

/

repomonkey bait & tackle/v1.01

Rule

Path

Disallow

/

rma

Rule

Path

Disallow

/

rogerbot

Rule

Path

Disallow

/

searchpreview

Rule

Path

Disallow

/

sitesnagger

Rule

Path

Disallow

/

spankbot

Rule

Path

Disallow

/

spanner

Rule

Path

Disallow

/

surveybot

Rule

Path

Disallow

/

suzuran

Rule

Path

Disallow

/

szukacz/1.4

Rule

Path

Disallow

/

teleport

Rule

Path

Disallow

/

teleportpro

Rule

Path

Disallow

/

telesoft

Rule

Path

Disallow

/

the intraformant

Rule

Path

Disallow

/

thenomad

Rule

Path

Disallow

/

tighttwatbot

Rule

Path

Disallow

/

tocrawl/urldispatcher

Rule

Path

Disallow

/

true_robot

Rule

Path

Disallow

/

true_robot/1.0

Rule

Path

Disallow

/

turingos

Rule

Path

Disallow

/

turnitinbot

Rule

Path

Disallow

/

turnitinbot/1.5

Rule

Path

Disallow

/

url control

Rule

Path

Disallow

/

url_spider_pro

Rule

Path

Disallow

/

urly warning

Rule

Path

Disallow

/

vci

Rule

Path

Disallow

/

vci webviewer vci webviewer win32

Rule

Path

Disallow

/

web image collector

Rule

Path

Disallow

/

webauto

Rule

Path

Disallow

/

webbandit

Rule

Path

Disallow

/

webbandit/3.50

Rule

Path

Disallow

/

webcapture 2.0

Rule

Path

Disallow

/

webcopier

Rule

Path

Disallow

/

webcopier v.2.2

Rule

Path

Disallow

/

webcopier v3.2a

Rule

Path

Disallow

/

webenhancer

Rule

Path

Disallow

/

websauger

Rule

Path

Disallow

/

website quester

Rule

Path

Disallow

/

webster pro

Rule

Path

Disallow

/

webstripper

Rule

Path

Disallow

/

webzip

Rule

Path

Disallow

/

webzip

Rule

Path

Disallow

/

webzip/4.0

Rule

Path

Disallow

/

webzip/4.21

Rule

Path

Disallow

/

webzip/5.0

Rule

Path

Disallow

/

wget

Rule

Path

Disallow

/

wget

Rule

Path

Disallow

/

wget/1.5.3

Rule

Path

Disallow

/

wget/1.6

Rule

Path

Disallow

/

www-collector-e

Rule

Path

Disallow

/

xenu’s

Rule

Path

Disallow

/

xenu’s link sleuth 1.1c

Rule

Path

Disallow

/

zeus

Rule

Path

Disallow

/

zeus 32297 webster pro v2.9 win32

Rule

Path

Disallow

/

zeus link scout

Rule

Path

Disallow

/

megaindex

Rule

Path

Disallow

/

baiduspider

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

runet-research-crawler

Rule

Path

Disallow

/

sogou

Rule

Path

Disallow

/

python

Rule

Path

Disallow

/

pingdom

Rule

Path

Disallow

/

riddler

Rule

Path

Disallow

/

java

Rule

Path

Disallow

/

njuicebot

Rule

Path

Disallow

/

js-kit

Rule

Path

Disallow

/

voyager

Rule

Path

Disallow

/

postrank

Rule

Path

Disallow

/

pycurl

Rule

Path

Disallow

/

aport

Rule

Path

Disallow

/

surveybot

Rule

Path

Disallow

/

butterfly

Rule

Path

Disallow

/

libwww

Rule

Path

Disallow

/

sweb

Rule

Path

Disallow

/

linkexchanger

Rule

Path

Disallow

/

soup

Rule

Path

Disallow

/

wordpress

Rule

Path

Disallow

/

spbot

Rule

Path

Disallow

/

mlbot

Rule

Path

Disallow

/

internetseer

Rule

Path

Disallow

/

fairshare

Rule

Path

Disallow

/

yeti

Rule

Path

Disallow

/

birubot

Rule

Path

Disallow

/

yottosbot

Rule

Path

Disallow

/

linguee

Rule

Path

Disallow

/

ezooms

Rule

Path

Disallow

/

purebot

Rule

Path

Disallow

/

kmsearchbot

Rule

Path

Disallow

/

sitebot

Rule

Path

Disallow

/

camontspider

Rule

Path

Disallow

/

ptd-crawler

Rule

Path

Disallow

/

suggybot

Rule

Path

Disallow

/

ttcrawler

Rule

Path

Disallow

/

nutch

Rule

Path

Disallow

/

Back to top