check-you.vip
robots.txt

Robots Exclusion Standard data for check-you.vip

Archived Snapshots

Resource Scan

Scan Details

Site Domain	check-you.vip
Base Domain	check-you.vip
Scan Status	Ok
Last Scan	2026-02-17T02:18:55+00:00
Next Scan	2026-03-03T02:18:55+00:00

Last Scan

Scanned	2026-02-17T02:18:55+00:00
URL	https://check-you.vip/robots.txt
Domain IPs	104.21.61.221, 172.67.215.192, 2606:4700:3036::6815:3ddd, 2606:4700:3036::ac43:d7c0
Response IP	172.67.215.192
Found	Yes
Hash	8a36270ec5845b74188d3c0af0ae688bbe25913b02a9921ed7fa9d576ca9b2ce
SimHash	771466704c97

Groups

*

Rule	Path
Allow	/

Rule

Path

Allow

amazonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

google-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalagent

Rule	Path
Disallow	/

Rule

Path

Disallow

yandex

Rule	Path
Disallow	/404.php
Disallow	/profiles.php
Disallow	/profiles_old.php

Rule

Path

Disallow

/404.php

Disallow

/profiles.php

Disallow

/profiles_old.php

Other Records

Field	Value
crawl-delay	20

Field

Value

crawl-delay

googlebot

Rule	Path
Disallow	/404.php
Disallow	/profiles.php
Disallow	/profiles_old.php

Rule

Path

Disallow

/404.php

Disallow

/profiles.php

Disallow

/profiles_old.php

Other Records

Field	Value
crawl-delay	20

Field

Value

crawl-delay

bingbot

Rule	Path
Disallow	/404.php
Disallow	/profiles.php
Disallow	/profiles_old.php

Rule

Path

Disallow

/404.php

Disallow

/profiles.php

Disallow

/profiles_old.php

Other Records

Field	Value
crawl-delay	20

Field

Value

crawl-delay

mail.ru_bot

Rule	Path
Disallow	/404.php
Disallow	/profiles.php
Disallow	/profiles_old.php

Rule

Path

Disallow

/404.php

Disallow

/profiles.php

Disallow

/profiles_old.php

Other Records

Field	Value
crawl-delay	20

Field

Value

crawl-delay

bingpreview

Rule	Path
Disallow	/404.php
Disallow	/profiles.php
Disallow	/profiles_old.php

Rule

Path

Disallow

/404.php

Disallow

/profiles.php

Disallow

/profiles_old.php

Other Records

Field	Value
crawl-delay	20

Field

Value

crawl-delay

telegrambot

Rule	Path
Disallow	/404.php

Rule

Path

Disallow

/404.php

*

Rule	Path
Disallow	/

Rule

Path

Disallow

a6-indexer

Rule	Path
Disallow	/

Rule

Path

Disallow

aboundex

Rule	Path
Disallow	/

Rule

Path

Disallow

academicbotrtu

Rule	Path
Disallow	/

Rule

Path

Disallow

acapbot

Rule	Path
Disallow	/

Rule

Path

Disallow

acoonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

adbeat_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

addsearchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

addthis

Rule	Path
Disallow	/

Rule

Path

Disallow

adidxbot

Rule	Path
Disallow	/

Rule

Path

Disallow

admantx

Rule	Path
Disallow	/

Rule

Path

Disallow

adscanner

Rule	Path
Disallow	/

Rule

Path

Disallow

adstxtcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

advbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ahc

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefs

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

aihitbot

Rule	Path
Disallow	/

Rule

Path

Disallow

aisearchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

alphabot

Rule	Path
Disallow	/

Rule

Path

Disallow

amazon cloudfront

Rule	Path
Disallow	/

Rule

Path

Disallow

amazonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

anderspinkbot

Rule	Path
Disallow	/

Rule

Path

Disallow

antibot

Rule	Path
Disallow	/

Rule

Path

Disallow

anyevent

Rule	Path
Disallow	/

Rule

Path

Disallow

apercite

Rule	Path
Disallow	/

Rule

Path

Disallow

appinsights

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot

Rule	Path
Disallow	/

Rule

Path

Disallow

arabot

Rule

Path

Disallow

archive.org_bot

Rule

Path

Disallow

archivebot

Rule

Path

Disallow

aspiegelbot

Rule

Path

Disallow

atom feed robot

Rule

Path

Disallow

awariobot

Rule

Path

Disallow

awariorssbot

Rule

Path

Disallow

awariosmartbot

Rule

Path

Disallow

awesomecrawler

Rule

Path

Disallow

axios

Rule

Path

Disallow

b2b bot

Rule

Path

Disallow

backlinkcrawler

Rule

Path

Disallow

backupland

Rule

Path

Disallow

baiduspider

Rule

Path

Disallow

baidu-yunguance

Rule

Path

Disallow

bark[rr]owler

Rule

Path

Disallow

barkrowler

Rule

Path

Disallow

bazqux

Rule

Path

Disallow

bdcbot

Rule

Path

Disallow

behloolbot

Rule

Path

Disallow

betabot

Rule

Path

Disallow

bidswitchbot

Rule

Path

Disallow

biglotron

Rule

Path

Disallow

binlar

Rule

Path

Disallow

bitbot

Rule

Path

Disallow

bitlybot

Rule

Path

Disallow

blackboard

Rule

Path

Disallow

blexbot

Rule

Path

Disallow

blogmurabot

Rule

Path

Disallow

blogtrafficd.d+ feed-fetcher

Rule

Path

Disallow

blp_bbot

Rule

Path

Disallow

bnf.fr_bot

Rule

Path

Disallow

bomborabot

Rule

Path

Disallow

bot.araturka.com

Rule

Path

Disallow

botify

Rule

Path

Disallow

bot-pge.chlooe.com

Rule

Path

Disallow

boxcarbot

Rule

Path

Disallow

brainobot

Rule

Path

Disallow

brandonbot

Rule

Path

Disallow

brandverity

Rule

Path

Disallow

btwebclient

Rule

Path

Disallow

bubing

Rule

Path

Disallow

bublupbot

Rule

Path

Disallow

buck

Rule

Path

Disallow

builtwith

Rule

Path

Disallow

buzzbot

Rule

Path

Disallow

bytespider

Rule

Path

Disallow

caliperbot

Rule

Path

Disallow

capsulechecker

Rule

Path

Disallow

careerbot

Rule

Path

Disallow

cc metadata scaper

Rule

Path

Disallow

ccbot

Rule

Path

Disallow

changedetection

Rule

Path

Disallow

check_http

Rule

Path

Disallow

checkmarknetwork

Rule

Path

Disallow

chrome-lighthouse

Rule

Path

Disallow

cincraw

Rule

Path

Disallow

citeseerxbot

Rule

Path

Disallow

clickagy

Rule

Path

Disallow

cliqzbot

Rule

Path

Disallow

cloudflare-alwaysonline

Rule

Path

Disallow

coccoc

Rule

Path

Disallow

collection@infegy.com

Rule

Path

Disallow

companybook-crawler

Rule

Path

Disallow

content crawler spider

Rule

Path

Disallow

contextad bot

Rule

Path

Disallow

contxbot

Rule

Path

Disallow

convera

Rule

Path

Disallow

crawler4j

Rule

Path

Disallow

crunchbot

Rule

Path

Disallow

crystalsemanticsbot

Rule

Path

Disallow

curebot

Rule

Path

Disallow

cutbot

Rule

Path

Disallow

cxensebot

Rule

Path

Disallow

cyberpatrol

Rule

Path

Disallow

dareboost

Rule

Path

Disallow

datafeedwatch

Rule

Path

Disallow

dataforseobot

Rule

Path

Disallow

datagnionbot

Rule

Path

Disallow

datanyze

Rule

Path

Disallow

dataprovider.com

Rule

Path

Disallow

daum

Rule

Path

Disallow

dcrawl

Rule

Path

Disallow

deadlinkchecker

Rule

Path

Disallow

detectify

Rule

Path

Disallow

deusu

Rule

Path

Disallow

diffbot

Rule

Path

Disallow

digg deeper

Rule

Path

Disallow

digincore bot

Rule

Path

Disallow

discobot

Rule

Path

Disallow

discordbot

Rule

Path

Disallow

disqus

Rule

Path

Disallow

dnyzbot

Rule

Path

Disallow

domain re-animator bot

Rule

Path

Disallow

domaincrawler

Rule

Path

Disallow

domains project

Rule

Path

Disallow

domainstatsbot

Rule

Path

Disallow

dotbot

Rule

Path

Disallow

dragonbot

Rule

Path

Disallow

drupact

Rule

Path

Disallow

dubbotbot

Rule

Path

Disallow

duckduckbot

Rule

Path

Disallow

duckduckgo-favicons-bot

Rule

Path

Disallow

ec2linkfinder

Rule

Path

Disallow

edisterbot

Rule

Path

Disallow

electricmonk

Rule

Path

Disallow

elisabot

Rule

Path

Disallow

embedly

Rule

Path

Disallow

epicbot

Rule

Path

Disallow

eright

Rule

Path

Disallow

europarchive.org

Rule

Path

Disallow

everyonesocialbot

Rule

Path

Disallow

exabot

Rule

Path

Disallow

experibot

Rule

Path

Disallow

extlinksbot

Rule

Path

Disallow

eyeotabot

Rule

Path

Disallow

ezid

Rule

Path

Disallow

ezooms

Rule

Path

Disallow

facebookexternalhit

Rule

Path

Disallow

facebot

Rule

Path

Disallow

fast enterprise crawler

Rule

Path

Disallow

fast-webcrawler

Rule

Path

Disallow

fedoraplanet

Rule

Path

Disallow

feedbot

Rule

Path

Disallow

feedly

Rule

Path

Disallow

feedspot

Rule

Path

Disallow

feedvalidator

Rule

Path

Disallow

femtosearchbot

Rule

Path

Disallow

fetch

Rule

Path

Disallow

fever

Rule

Path

Disallow

filterdb.iss.netcrawler

Rule

Path

Disallow

finditanswersbot

Rule

Path

Disallow

findlink

Rule

Path

Disallow

findthatfile

Rule

Path

Disallow

findxbot

Rule

Path

Disallow

flamingo_searchengine

Rule

Path

Disallow

flipboardproxy

Rule

Path

Disallow

fluffy

Rule

Path

Disallow

fr-crawler

Rule

Path

Disallow

freewebmonitoring sitechecker

Rule

Path

Disallow

freshrss

Rule

Path

Disallow

friendica

Rule

Path

Disallow

fuelbot

Rule

Path

Disallow

fyrebot

Rule

Path

Disallow

g00g1e.net

Rule

Path

Disallow

g2 web services

Rule

Path

Disallow

g2reader-bot

Rule

Path

Disallow

garlikcrawler

Rule

Path

Disallow

genieo

Rule

Path

Disallow

gigablast

Rule

Path

Disallow

gigabot

Rule

Path

Disallow

gingercrawler

Rule

Path

Disallow

gluten free crawler

Rule

Path

Disallow

gnam gnam spider

Rule

Path

Disallow

gnowitnewsbot

Rule

Path

Disallow

go-http-client

Rule

Path

Disallow

go-http-client

Rule

Path

Disallow

gowikibot

Rule

Path

Disallow

gptbot

Rule

Path

Disallow

grapeshotcrawler

Rule

Path

Disallow

grobbot

Rule

Path

Disallow

grouphigh

Rule

Path

Disallow

grub.org

Rule

Path

Disallow

gslfbot

Rule

Path

Disallow

gwene

Rule

Path

Disallow

hatena

Rule

Path

Disallow

headlesschrome

Rule

Path

Disallow

hello, world

Rule

Path

Disallow

heritrix

Rule

Path

Disallow

http_get

Rule

Path

Disallow

httpunit

Rule

Path

Disallow

httpurlconnection

Rule

Path

Disallow

httrack

Rule

Path

Disallow

hubspot

Rule

Path

Disallow

ia_archiver

Rule

Path

Disallow

ias crawler

Rule

Path

Disallow

icbot

Rule

Path

Disallow

icc-crawler

Rule

Path

Disallow

ichiro

Rule

Path

Disallow

imrbot

Rule

Path

Disallow

indeedbot

Rule

Path

Disallow

infoobot

Rule

Path

Disallow

integromedb

Rule

Path

Disallow

intelium_bot

Rule

Path

Disallow

interfaxscanbot

Rule

Path

Disallow

ips-agent

Rule

Path

Disallow

ip-web-crawler.com

Rule

Path

Disallow

iskanie

Rule

Path

Disallow

istellabot

Rule

Path

Disallow

it2media-domain-crawler

Rule

Path

Disallow

james bot

Rule

Path

Disallow

jamie's spider

Rule

Path

Disallow

jetslide

Rule

Path

Disallow

jetty

Rule

Path

Disallow

jobboersebot

Rule

Path

Disallow

jooblebot

Rule

Path

Disallow

jpg-newsbot

Rule

Path

Disallow

jugendschutzprogramm-crawler

Rule

Path

Disallow

jyxobot

Rule

Path

Disallow

k7mlwcbot

Rule

Path

Disallow

kemvibot

Rule

Path

Disallow

kocmohabt

Rule

Path

Disallow

kosmiobot

Rule

Path

Disallow

landau-media-spider

Rule

Path

Disallow

laserlikebot

Rule

Path

Disallow

lb-spider

Rule

Path

Disallow

leikibot

Rule

Path

Disallow

libwww-perl

Rule

Path

Disallow

linguee bot

Rule

Path

Disallow

linkapediabot

Rule

Path

Disallow

linkarchiver

Rule

Path

Disallow

linkdex

Rule

Path

Disallow

linkedinbot

Rule

Path

Disallow

linkfluence

Rule

Path

Disallow

linkisbot

Rule

Path

Disallow

linkpadbot

Rule

Path

Disallow

lipperhey

Rule

Path

Disallow

livelap[bb]ot

Rule

Path

Disallow

lssbot

Rule

Path

Disallow

lssrocketcrawler

Rule

Path

Disallow

ltx71

Rule

Path

Disallow

luminator-robots

Rule

Path

Disallow

magpie-crawler

Rule

Path

Disallow

mappydata

Rule

Path

Disallow

mastodon

Rule

Path

Disallow

mauibot

Rule

Path

Disallow

mbcrawler

Rule

Path

Disallow

mediapartners

Rule

Path

Disallow

mediatoolkitbot

Rule

Path

Disallow

megaindex

Rule

Path

Disallow

megaindex

Rule

Path

Disallow

meltwaternews

Rule

Path

Disallow

memorybot

Rule

Path

Disallow

metajobbot

Rule

Path

Disallow

metauri

Rule

Path

Disallow

mindupbot

Rule

Path

Disallow

miniflux

Rule

Path

Disallow

mixnodecache

Rule

Path

Disallow

mj12bot

Rule

Path

Disallow

mlbot

Rule

Path

Disallow

moatbot

Rule

Path

Disallow

mojeekbot

Rule

Path

Disallow

moodlebot

Rule

Path

Disallow

moreover

Rule

Path

Disallow

moz

Rule

Path

Disallow

msnbot

Rule

Path

Disallow

msrbot

Rule

Path

Disallow

muckrack

Rule

Path

Disallow

multiviewbot

Rule

Path

Disallow

naver blog rssbot

Rule

Path

Disallow

nerdbynature.bot

Rule

Path

Disallow

nerdybot

Rule

Path

Disallow

netcraftsurveyagent

Rule

Path

Disallow

netcraftsurveyagent

Rule

Path

Disallow

netestate ne crawler

Rule

Path

Disallow

neticle crawler

Rule

Path

Disallow

netresearchserver

Rule

Path

Disallow

netvibes

Rule

Path

Disallow

newsharecounts

Rule

Path

Disallow

newspaper

Rule

Path

Disallow

nextcloud

Rule

Path

Disallow

niki-bot

Rule

Path

Disallow

nimbostratus-bot

Rule

Path

Disallow

ning

Rule

Path

Disallow

ninja bot

Rule

Path

Disallow

nixstatsbot

Rule

Path

Disallow

nmap scripting engine

Rule

Path

Disallow

ntentbot

Rule

Path

Disallow

nuclei

Rule

Path

Disallow

nutch

Rule

Path

Disallow

nuzzel

Rule

Path

Disallow

ocarinabot

Rule

Path

Disallow

officestorebot

Rule

Path

Disallow

okhttp

Rule

Path

Disallow

omgili

Rule

Path

Disallow

online-webceo-bot

Rule

Path

Disallow

openhosebot

Rule

Path

Disallow

openindexspider

Rule

Path

Disallow

orangebot

Rule

Path

Disallow

outbrain

Rule

Path

Disallow

outclicksbot

Rule

Path

Disallow

page2rss

Rule

Path

Disallow

pagepeeker

Rule

Path

Disallow

paloaltonetworks

Rule

Path

Disallow

pandalytics

Rule

Path

Disallow

panscient

Rule

Path

Disallow

paperlibot

Rule

Path

Disallow

pcore-http

Rule

Path

Disallow

phantomjs

Rule

Path

Disallow

phpcrawl

Rule

Path

Disallow

pingdom

Rule

Path

Disallow

pinterest.com.bot

Rule

Path

Disallow

piplbot

Rule

Path

Disallow

pocketparser

Rule

Path

Disallow

postrank

Rule

Path

Disallow

pr-cy.ru

Rule

Path

Disallow

pr-cy.ru

Rule

Path

Disallow

primalbot

Rule

Path

Disallow

privacyawarebot

Rule

Path

Disallow

proximic

Rule

Path

Disallow

psbot

Rule

Path

Disallow

pulsepoint

Rule

Path

Disallow

purebot

Rule

Path

Disallow

python-requests

Rule

Path

Disallow

python-urllib

Rule

Path

Disallow

qwantify

Rule

Path

Disallow

rankactivelinkbot

Rule

Path

Disallow

redditbot

Rule

Path

Disallow

refindbot

Rule

Path

Disallow

regionstuttgartbot

Rule

Path

Disallow

retrevopageanalyzer

Rule

Path

Disallow

ridderbot

Rule

Path

Disallow

riddler

Rule

Path

Disallow

rivva

Rule

Path

Disallow

rogerbot

Rule

Path

Disallow

rssbot

Rule

Path

Disallow

rssingbot

Rule

Path

Disallow

rytebot

Rule

Path

Disallow

safednsbot

Rule

Path

Disallow

safesearch microdata crawler

Rule

Path

Disallow

sbl-bot

Rule

Path

Disallow

scoutjet

Rule

Path

Disallow

scrapy

Rule

Path

Disallow

scrapy

Rule

Path

Disallow

screaming frog seo spider

Rule

Path

Disallow

screaming frog seo spider

Rule

Path

Disallow

scribdbot

Rule

Path

Disallow

searchatlas

Rule

Path

Disallow

seekbot

Rule

Path

Disallow

seekport crawler

Rule

Path

Disallow

seekportbot

Rule

Path

Disallow

seewithkids

Rule

Path

Disallow

semanticbot

Rule

Path

Disallow

semanticscholarbot

Rule

Path

Disallow

semrush

Rule

Path

Disallow

semrushbot

Rule

Path

Disallow

semrushbot-sa

Rule

Path

Disallow

sentibot

Rule

Path

Disallow

seobilitybot

Rule

Path

Disallow

seokicks

Rule

Path

Disallow

seokicks

Rule

Path

Disallow

seopultcontentanalyzer

Rule

Path

Disallow

seoscanners

Rule

Path

Disallow

serendeputy

Rule

Path

Disallow

serendeputybot

Rule

Path

Disallow

serpstatbot

Rule

Path

Disallow

serpstatbot

Rule

Path

Disallow

seznambot

Rule

Path

Disallow

simplecrawler

Rule

Path

Disallow

simplepie

Rule

Path

Disallow

simplescraper

Rule

Path

Disallow

sistrix crawler

Rule

Path

Disallow

sitebot

Rule

Path

Disallow

siteexplorer.info

Rule

Path

Disallow

siteimprove.com

Rule

Path

Disallow

skypeuripreview

Rule

Path

Disallow

slackbot

Rule

Path

Disallow

slack-imgproxy

Rule

Path

Disallow

slurp

Rule

Path

Disallow

smtbot

Rule

Path

Disallow

snacktory

Rule

Path

Disallow

socialrankiobot

Rule

Path

Disallow

sogou

Rule

Path

Disallow

sonic

Rule

Path

Disallow

spbot

Rule

Path

Disallow

speedy

Rule

Path

Disallow

startmebot

Rule

Path

Disallow

statdom

Rule

Path

Disallow

storygizebot

Rule

Path

Disallow

streamline3bot

Rule

Path

Disallow

summify

Rule

Path

Disallow

superfeedr

Rule

Path

Disallow

surdotlybot

Rule

Path

Disallow

surveybot

Rule

Path

Disallow

swimgbot

Rule

Path

Disallow

sysomos

Rule

Path

Disallow

taboolabot

Rule

Path

Disallow

tagoobot

Rule

Path

Disallow

tangibleebot

Rule

Path

Disallow

teoma

Rule

Path

Disallow

theoldreader.com

Rule

Path

Disallow

thinklab

Rule

Path

Disallow

tigerbot

Rule

Path

Disallow

tineye

Rule

Path

Disallow

tiny tiny rss

Rule

Path

Disallow

toplistbot

Rule

Path

Disallow

toutiaospider

Rule

Path

Disallow

traackr.com

Rule

Path

Disallow

tracemyfile

Rule

Path

Disallow

trendictionbot

Rule

Path

Disallow

trendsmapresolver

Rule

Path

Disallow

trove

Rule

Path

Disallow

trovitbot

Rule

Path

Disallow

turnitin

Rule

Path

Disallow

turnitinbot

Rule

Path

Disallow

tweetedtimes

Rule

Path

Disallow

tweetmemebot

Rule

Path

Disallow

twengabot

Rule

Path

Disallow

twingly

Rule

Path

Disallow

twitterbot

Rule

Path

Disallow

twurly

Rule

Path

Disallow

um-ln

Rule

Path

Disallow

upflow

Rule

Path

Disallow

uptimebot.org

Rule

Path

Disallow

uptimerobot

Rule

Path

Disallow

urlappendbot

Rule

Path

Disallow

usinenouvellecrawler

Rule

Path

Disallow

ut-dorkbot

Rule

Path

Disallow

validator.nu

Rule

Path

Disallow

vebidoobot

Rule

Path

Disallow

velenpublicwebcrawler

Rule

Path

Disallow

veoozbot

Rule

Path

Disallow

vigil

Rule

Path

Disallow

vkrobot

Rule

Path

Disallow

vkshare

Rule

Path

Disallow

voilabot

Rule

Path

Disallow

voluumdsp-content-bot

Rule

Path

Disallow

w3c_css_validator

Rule

Path

Disallow

w3c_i18n-checker

Rule

Path

Disallow

w3c_unicorn

Rule

Path

Disallow

w3c_validator

Rule

Path

Disallow

w3c-checklink

Rule

Path

Disallow

w3c-mobileok

Rule

Path

Disallow

wbsearchbot

Rule

Path

Disallow

web-archive-net.com.bot

Rule

Path

Disallow

webcompanycrawler

Rule

Path

Disallow

webdatastats

Rule

Path

Disallow

webmon

Rule

Path

Disallow

weborama

Rule

Path

Disallow

wellknownbot

Rule

Path

Disallow

wesee:search

Rule

Path

Disallow

Rule

Path

Disallow

wocbot

Rule

Path

Disallow

woobot

Rule

Path

Disallow

wordupinfosearch

Rule

Path

Disallow

woriobot

Rule

Path

Disallow

wotbox

Rule

Path

Disallow

www.uptime.com

Rule

Path

Disallow

xenu link sleuth

Rule

Path

Disallow

xovibot

Rule

Path

Disallow

y!j

Rule

Path

Disallow

yacybot

Rule

Path

Disallow

yahoo link preview

Rule

Path

Disallow

yak

Rule

Path

Disallow

yanga

Rule

Path

Disallow

yeti

Rule

Path

Disallow

yisouspider

Rule

Path

Disallow

yoozbot

Rule

Path

Disallow

zabbix

Rule

Path

Disallow

zenback bot

Rule

Path

Disallow

zgrab

Rule

Path

Disallow

zoombot

Rule

Path

Disallow

zoominfobot

Rule

Path

Disallow

zoominfobot

Rule

Path

Disallow

zumbot

Rule

Path

Disallow

zuperlistbot

Rule

Path

Disallow

Comments

As a condition of accessing this website, you agree to abide by the following
content signals:
(a) If a Content-Signal = yes, you may collect content for the corresponding
use.
(b) If a Content-Signal = no, you may not collect content for the
corresponding use.
(c) If the website operator does not include a Content-Signal for a
corresponding use, the website operator neither grants nor restricts
permission via Content-Signal with respect to the corresponding use.
The content signals and their meanings are:
search: building a search index and providing search results (e.g., returning
hyperlinks and short excerpts from your website's contents). Search does not
include providing AI-generated search summaries.
ai-input: inputting content into one or more AI models (e.g., retrieval
augmented generation, grounding, or other real-time taking of content for
generative AI search answers).
ai-train: training or fine-tuning AI models.
ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF
AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET.
BEGIN Cloudflare Managed content
END Cloudflare Managed Content

Warnings

16 invalid lines.
`content-signal` is not a known field.

check-you.viprobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

amazonbot

applebot-extended

bytespider

ccbot

claudebot

google-extended

gptbot

meta-externalagent

yandex

Other Records

googlebot

Other Records

bingbot

Other Records

mail.ru_bot

Other Records

bingpreview

Other Records

telegrambot

*

a6-indexer

aboundex

academicbotrtu

acapbot

acoonbot

adbeat_bot

addsearchbot

addthis

adidxbot

admantx

adscanner

adstxtcrawler

advbot

ahc

ahrefs

ahrefsbot

aihitbot

aisearchbot

alphabot

amazon cloudfront

amazonbot

anderspinkbot

antibot

anyevent

apercite

appinsights

applebot

arabot

archive.org_bot

archivebot

aspiegelbot

atom feed robot

awariobot

awariorssbot

awariosmartbot

awesomecrawler

axios

b2b bot

backlinkcrawler

backupland

baiduspider

baidu-yunguance

bark[rr]owler

barkrowler

bazqux

bdcbot

behloolbot

betabot

bidswitchbot

biglotron

binlar

bitbot

bitlybot

blackboard

check-you.vip
robots.txt