Log In Sign Up

unaique.net
robots.txt

Robots Exclusion Standard data for unaique.net

Resource Scan

Scan Details

Site Domain unaique.net
Base Domain unaique.net
Scan Status Ok
Last Scan2026-01-24T03:09:18+00:00
Next Scan 2026-01-31T03:09:18+00:00

Last Scan

Scanned2026-01-24T03:09:18+00:00
URL https://unaique.net/robots.txt
Redirect https://www.unaique.net/robots.txt
Redirect Domain www.unaique.net
Redirect Base unaique.net
Domain IPs 45.136.28.169
Redirect IPs 45.136.28.169
Response IP 45.136.28.169
Found Yes
Hash b608f26dbee1d66a659e00db47bf4a06174d3e2702f2666094cb69a6866151fd
SimHash 7d0567720287

Groups

semrushbot-ft

Rule Path
Allow /

semrushbot-ocob

Rule Path
Allow /

seocherrybot

Rule Path
Allow /

splitsignalbot

Rule Path
Allow /

everyfeed-spider

Rule Path
Allow /

meta-externalfetcher

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

bytespider

Rule Path
Allow /

ccbot

Rule Path
Allow /

diffbot

Rule Path
Allow /

facebookbot

Rule Path
Allow /

google-extended

Rule Path
Allow /

meta-externalagent

Rule Path
Allow /

omgili

Rule Path
Allow /

timpibot

Rule Path
Allow /

webzio-extended

Rule Path
Allow /

amazonbot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

ahrefssiteaudit

Rule Path
Allow /

dark visitor

Rule Path
Allow /

deadlinkchecker

Rule Path
Allow /

rogerbot

Rule Path
Allow /

siteauditbot

Rule Path
Allow /

t3versionsbot

Rule Path
Allow /

wellknownbot

Rule Path
Allow /

adbeat_bot

Rule Path
Allow /

aihitbot

Rule Path
Allow /

anderspinkbot

Rule Path
Allow /

awariobot

Rule Path
Allow /

awariosmartbot

Rule Path
Allow /

bitsightbot

Rule Path
Allow /

blackboard

Rule Path
Allow /

brandverity

Rule Path
Allow /

cincraw

Rule Path
Allow /

ev-crawler

Rule Path
Allow /

hubspot

Rule Path
Allow /

imagesiftbot

Rule Path
Allow /

ioncrawl

Rule Path
Allow /

jugendschutzprogramm-crawler

Rule Path
Allow /

kstandbot

Rule Path
Allow /

lightspeedsystemscrawler

Rule Path
Allow /

linkfluence

Rule Path
Allow /

linkwalker

Rule Path
Allow /

magpie-crawler

Rule Path
Allow /

mediatoolkitbot

Rule Path
Allow /

muckrack

Rule Path
Allow /

netcraftsurveyagent

Rule Path
Allow /

netvibes

Rule Path
Allow /

pandalytics

Rule Path
Allow /

panscient.com

Rule Path
Allow /

proximic

Rule Path
Allow /

seekportbot

Rule Path
Allow /

smtbot

Rule Path
Allow /

trendictionbot

Rule Path
Allow /

trendsmapresolver

Rule Path
Allow /

turnitin

Rule Path
Allow /

turnitinbot

Rule Path
Allow /

tweetmemebot

Rule Path
Allow /

twingly

Rule Path
Allow /

um-ln

Rule Path
Allow /

velenpublicwebcrawler

Rule Path
Allow /

virustotal

Rule Path
Allow /

webzio

Rule Path
Allow /

zoominfobot

Rule Path
Allow /

dataprovider.com

Rule Path
Allow /

dcrawl

Rule Path
Allow /

httrack

Rule Path
Allow /

httrack 3.0

Rule Path
Allow /

metainspector

Rule Path
Allow /

newspaper

Rule Path
Allow /

nutch

Rule Path
Allow /

offline explorer

Rule Path
Allow /

openindexspider

Rule Path
Allow /

scrapy

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

barkrowler

Rule Path
Allow /

blexbot

Rule Path
Allow /

brightedge crawler

Rule Path
Allow /

cocolyzebot

Rule Path
Allow /

dataforseobot

Rule Path
Allow /

domainstatsbot

Rule Path
Allow /

dotbot

Rule Path
Allow /

hypestat

Rule Path
Allow /

linkdexbot

Rule Path
Allow /

mj12bot

Rule Path
Allow /

online-webceo-bot

Rule Path
Allow /

screaming frog seo spider

Rule Path
Allow /

semrushbot

Rule Path
Allow /

semrushbot-ba

Rule Path
Allow /

semrushbot-ct

Rule Path
Allow /

semrushbot-si

Rule Path
Allow /

semrushbot-swa

Rule Path
Allow /

senutobot

Rule Path
Allow /

seobilitybot

Rule Path
Allow /

seokicks

Rule Path
Allow /

seolizer

Rule Path
Allow /

serpstatbot

Rule Path
Allow /

sitecheckerbotcrawler

Rule Path
Allow /

zoombot

Rule Path
Allow /

a patent crawler

Rule Path
Allow /

a6-indexer

Rule Path
Allow /

aboundex

Rule Path
Allow /

academicbotrtu

Rule Path
Allow /

acapbot

Rule Path
Allow /

acoonbot

Rule Path
Allow /

acunetix security scanner

Rule Path
Allow /

acunetix web vulnerability scanner

Rule Path
Allow /

addsearchbot

Rule Path
Allow /

addthis

Rule Path
Allow /

adequat

Rule Path
Allow /

adequat-systems

Rule Path
Allow /

adidxbot

Rule Path
Allow /

admantx

Rule Path
Allow /

adscanner

Rule Path
Allow /

adstxtcrawler

Rule Path
Allow /

advbot

Rule Path
Allow /

aisearchbot

Rule Path
Allow /

alexabot

Rule Path
Allow /

alexibot

Rule Path
Allow /

alphabot

Rule Path
Allow /

amisoftware

Rule Path
Allow /

antibot

Rule Path
Allow /

anyevent

Rule Path
Allow /

apercite

Rule Path
Allow /

appinsights

Rule Path
Allow /

aqua_products

Rule Path
Allow /

arabot

Rule Path
Allow /

ask n read

Rule Path
Allow /

asknread.com

Rule Path
Allow /

aspiegelbot

Rule Path
Allow /

asterias

Rule Path
Allow /

augure

Rule Path
Allow /

auramundi

Rule Path
Allow /

awariorssbot

Rule Path
Allow /

awesomecrawler

Rule Path
Allow /

b2b bot

Rule Path
Allow /

b2w

Rule Path
Allow /

backdoorbot

Rule Path
Allow /

backlinkcrawler

Rule Path
Allow /

bdcbot

Rule Path
Allow /

behloolbot

Rule Path
Allow /

betabot

Rule Path
Allow /

better uptime bot

Rule Path
Allow /

bidswitchbot

Rule Path
Allow /

biglotron

Rule Path
Allow /

binlar

Rule Path
Allow /

birdcrawlerbot

Rule Path
Allow /

bitbot

Rule Path
Allow /

black hole

Rule Path
Allow /

blekkobot

Rule Path
Allow /

blogmurabot

Rule Path
Allow /

blowfish

Rule Path
Allow /

blp_bbot

Rule Path
Allow /

bnf.fr_bot

Rule Path
Allow /

bomborabot

Rule Path
Allow /

bookmark search tool

Rule Path
Allow /

bot-pge.chlooe.com

Rule Path
Allow /

bot.araturka.com

Rule Path
Allow /

botalot

Rule Path
Allow /

botify

Rule Path
Allow /

botrighthere

Rule Path
Allow /

boxcarbot

Rule Path
Allow /

brainobot

Rule Path
Allow /

brandonbot

Rule Path
Allow /

btwebclient

Rule Path
Allow /

bubing

Rule Path
Allow /

buck

Rule Path
Allow /

builtbottough

Rule Path
Allow /

bullseye

Rule Path
Allow /

bunnyslippers

Rule Path
Allow /

buzzbot

Rule Path
Allow /

caliperbot

Rule Path
Allow /

capsulechecker

Rule Path
Allow /

careerbot

Rule Path
Allow /

cc metadata scaper

Rule Path
Allow /

cegbfeieh

Rule Path
Allow /

centurybot

Rule Path
Allow /

changedetection

Rule Path
Allow /

checkmarknetwork

Rule Path
Allow /

cheesebot

Rule Path
Allow /

cherrypicker

Rule Path
Allow /

cherrypickerelite

Rule Path
Allow /

cherrypickerse

Rule Path
Allow /

cision

Rule Path
Allow /

cispa webcrawler

Rule Path
Allow /

citeseerxbot

Rule Path
Allow /

citoid

Rule Path
Allow /

claritybot

Rule Path
Allow /

clickagy

Rule Path
Allow /

cliqzbot

Rule Path
Allow /

cloudflare-alwaysonline

Rule Path
Allow /

coexel

Rule Path
Allow /

companybook-crawler

Rule Path
Allow /

content crawler spider

Rule Path
Allow /

contextad bot

Rule Path
Allow /

contxbot

Rule Path
Allow /

convera

Rule Path
Allow /

converacrawler

Rule Path
Allow /

cookiebot

Rule Path
Allow /

copernic

Rule Path
Allow /

copyrightcheck

Rule Path
Allow /

corporama

Rule Path
Allow /

cosmos

Rule Path
Allow /

crawler4j

Rule Path
Allow /

crawlyprojectcrawler

Rule Path
Allow /

crescent

Rule Path
Allow /

crescent internet toolpak http ole control v.1.0

Rule Path
Allow /

criteobot

Rule Path
Allow /

crunchbot

Rule Path
Allow /

crystalsemanticsbot

Rule Path
Allow /

curebot

Rule Path
Allow /

cutbot

Rule Path
Allow /

cxensebot

Rule Path
Allow /

cyberpatrol

Rule Path
Allow /

dareboost

Rule Path
Allow /

datafeedwatch

Rule Path
Allow /

datagnionbot

Rule Path
Allow /

datanyze

Rule Path
Allow /

daumoa

Rule Path
Allow /

deepcrawl

Rule Path
Allow /

deepnoc

Rule Path
Allow /

deusu

Rule Path
Allow /

digg deeper

Rule Path
Allow /

digimind

Rule Path
Allow /

digincore bot

Rule Path
Allow /

discobot

Rule Path
Allow /

disqus

Rule Path
Allow /

dittospyder

Rule Path
Allow /

dnyzbot

Rule Path
Allow /

domain re-animator bot

Rule Path
Allow /

domaincrawler

Rule Path
Allow /

dow jones searchbot

Rule Path
Allow /

download ninja

Rule Path
Allow /

dragonbot

Rule Path
Allow /

drupact

Rule Path
Allow /

dubbotbot

Rule Path
Allow /

e.ventures investment crawler

Rule Path
Allow /

easybib autocite

Rule Path
Allow /

ec2linkfinder

Rule Path
Allow /

edisterbot

Rule Path
Allow /

electricmonk

Rule Path
Allow /

elisabot

Rule Path
Allow /

ellisphere

Rule Path
Allow /

emailcollector

Rule Path
Allow /

emailsiphon

Rule Path
Allow /

emailwolf

Rule Path
Allow /

epicbot

Rule Path
Allow /

eright

Rule Path
Allow /

erocrawler

Rule Path
Allow /

etaospider

Rule Path
Allow /

europarchive.org

Rule Path
Allow /

evc-batch

Rule Path
Allow /

everyonesocialbot

Rule Path
Allow /

exabot

Rule Path
Allow /

experibot

Rule Path
Allow /

extlinksbot

Rule Path
Allow /

extractorpro

Rule Path
Allow /

eyeotabot

Rule Path
Allow /

ezid

Rule Path
Allow /

ezooms

Rule Path
Allow /

facebot

Rule Path
Allow /

fairad client

Rule Path
Allow /

fast enterprise crawler

Rule Path
Allow /

fast-webcrawler

Rule Path
Allow /

fedidb

Rule Path
Allow /

fedoraplanet

Rule Path
Allow /

feedbin

Rule Path
Allow /

feedbot

Rule Path
Allow /

feedspot

Rule Path
Allow /

femtosearchbot

Rule Path
Allow /

fever

Rule Path
Allow /

finditanswersbot

Rule Path
Allow /

findlink

Rule Path
Allow /

findthatfile

Rule Path
Allow /

findxbot

Rule Path
Allow /

flaming attackbot

Rule Path
Allow /

flamingo_searchengine

Rule Path
Allow /

fluffy

Rule Path
Allow /

foobot

Rule Path
Allow /

fr-crawler

Rule Path
Allow /

freewebmonitoring sitechecker

Rule Path
Allow /

freshpingbot

Rule Path
Allow /

fuelbot

Rule Path
Allow /

fyrebot

Rule Path
Allow /

g00g1e.net

Rule Path
Allow /

g2 web services

Rule Path
Allow /

g2reader-bot

Rule Path
Allow /

gaisbot

Rule Path
Allow /

garlikcrawler

Rule Path
Allow /

genieo

Rule Path
Allow /

genomecrawlerd

Rule Path
Allow /

getright

Rule Path
Allow /

gingercrawler

Rule Path
Allow /

gluten free crawler

Rule Path
Allow /

gnam gnam spider

Rule Path
Allow /

gnowitnewsbot

Rule Path
Allow /

google-xrawler

Rule Path
Allow /

gowikibot

Rule Path
Allow /

grapeshot

Rule Path
Allow /

grapeshotcrawler

Rule Path
Allow /

grobbot

Rule Path
Allow /

grouphigh

Rule Path
Allow /

grub-client

Rule Path
Allow /

grub.org

Rule Path
Allow /

gsa-crawler

Rule Path
Allow /

gslfbot

Rule Path
Allow /

gwene

Rule Path
Allow /

harvest

Rule Path
Allow /

hawaiibot

Rule Path
Allow /

humanlinks

Rule Path
Allow /

hyscore.io

Rule Path
Allow /

ias crawler

Rule Path
Allow /

icbot

Rule Path
Allow /

icc-crawler

Rule Path
Allow /

ichiro

Rule Path
Allow /

imrbot

Rule Path
Allow /

indeedbot

Rule Path
Allow /

inetdex-bot

Rule Path
Allow /

infonavirobot

Rule Path
Allow /

infoobot

Rule Path
Allow /

infoseek

Rule Path
Allow /

integromedb

Rule Path
Allow /

intelium_bot

Rule Path
Allow /

interfaxscanbot

Rule Path
Allow /

ip-web-crawler.com

Rule Path
Allow /

irlbot

Rule Path
Allow /

iron33

Rule Path
Allow /

iskanie

Rule Path
Allow /

israbot

Rule Path
Allow /

istellabot

Rule Path
Allow /

it2media-domain-crawler

Rule Path
Allow /

james bot

Rule Path
Allow /

jamesbot

Rule Path
Allow /

jamie's spider

Rule Path
Allow /

jenkersbot

Rule Path
Allow /

jennybot

Rule Path
Allow /

jetbot

Rule Path
Allow /

jetty

Rule Path
Allow /

jikespider

Rule Path
Allow /

jobboersebot

Rule Path
Allow /

jooblebot

Rule Path
Allow /

jpg-newsbot

Rule Path
Allow /

jyxobot

Rule Path
Allow /

k2spider

Rule Path
Allow /

k7mlwcbot

Rule Path
Allow /

kbcrawl

Rule Path
Allow /

kemvibot

Rule Path
Allow /

kenjin spider

Rule Path
Allow /

keys-so-bot

Rule Path
Allow /

keyword density

Rule Path
Allow /

knowings

Rule Path
Allow /

komodiabot

Rule Path
Allow /

kosmiobot

Rule Path
Allow /

landau-media-spider

Rule Path
Allow /

larbin

Rule Path
Allow /

laserlikebot

Rule Path
Allow /

lb-spider

Rule Path
Allow /

leadbox

Rule Path
Allow /

leikibot

Rule Path
Allow /

lexibot

Rule Path
Allow /

libweb

Rule Path
Allow /

linespider

Rule Path
Allow /

linguee bot

Rule Path
Allow /

linkapediabot

Rule Path
Allow /

linkarchiver

Rule Path
Allow /

linkcheck by siteimprove.com

Rule Path
Allow /

linkdex

Rule Path
Allow /

linkextractorpro

Rule Path
Allow /

linkisbot

Rule Path
Allow /

linko

Rule Path
Allow /

linkpadbot

Rule Path
Allow /

linkscan

Rule Path
Allow /

lipperhey

Rule Path
Allow /

livelapbot

Rule Path
Allow /

lkxscan

Rule Path
Allow /

lnspiderguy

Rule Path
Allow /

lssbot

Rule Path
Allow /

lssrocketcrawler

Rule Path
Allow /

ltx71

Rule Path
Allow /

luminator-robots

Rule Path
Allow /

lwp-trivial

Rule Path
Allow /

macocu

Rule Path
Allow /

mappydata

Rule Path
Allow /

mata hari

Rule Path
Allow /

mauibot

Rule Path
Allow /

mbcrawler

Rule Path
Allow /

meltawer

Rule Path
Allow /

meltwater

Rule Path
Allow /

meltwaternews

Rule Path
Allow /

memorybot

Rule Path
Allow /

mention

Rule Path
Allow /

metajobbot

Rule Path
Allow /

metauri

Rule Path
Allow /

miixpc

Rule Path
Allow /

mindupbot

Rule Path
Allow /

minicrawler

Rule Path
Allow /

mister pix

Rule Path
Allow /

mixnodecache

Rule Path
Allow /

mlbot

Rule Path
Allow /

moatbot

Rule Path
Allow /

moget

Rule Path
Allow /

moodlebot

Rule Path
Allow /

moreover

Rule Path
Allow /

ms search 4.0 robot

Rule Path
Allow /

ms search 6.0 robot

Rule Path
Allow /

msiecrawler

Rule Path
Allow /

msrbot

Rule Path
Allow /

mtrobot

Rule Path
Allow /

multiviewbot

Rule Path
Allow /

mytwip

Rule Path
Allow /

naver blog rssbot

Rule Path
Allow /

naverbot

Rule Path
Allow /

neevabot

Rule Path
Allow /

nerdbynature.bot

Rule Path
Allow /

nerdybot

Rule Path
Allow /

netants

Rule Path
Allow /

netestate ne crawler

Rule Path
Allow /

neticle crawler

Rule Path
Allow /

netmechanic

Rule Path
Allow /

netresearchserver

Rule Path
Allow /

netsystemsresearch

Rule Path
Allow /

newsharecounts

Rule Path
Allow /

newsnow

Rule Path
Allow /

newzbin

Rule Path
Allow /

nextgensearchbot

Rule Path
Allow /

nicerspro

Rule Path
Allow /

niki-bot

Rule Path
Allow /

nimblecrawler

Rule Path
Allow /

nimbostratus-bot

Rule Path
Allow /

ninja bot

Rule Path
Allow /

nixstatsbot

Rule Path
Allow /

nlux_iaharvester

Rule Path
Allow /

nmap scripting engine

Rule Path
Allow /

npbot

Rule Path
Allow /

ntentbot

Rule Path
Allow /

nuzzel

Rule Path
Allow /

odklbot

Rule Path
Allow /

officestorebot

Rule Path
Allow /

omgilibot

Rule Path
Allow /

openbot

Rule Path
Allow /

openfind

Rule Path
Allow /

openfind data gatherer

Rule Path
Allow /

opengraphcheck

Rule Path
Allow /

openhosebot

Rule Path
Allow /

opinion-tracker

Rule Path
Allow /

oracle ultra search

Rule Path
Allow /

orangebot

Rule Path
Allow /

orthogaffe

Rule Path
Allow /

outbrain

Rule Path
Allow /

outclicksbot

Rule Path
Allow /

page2rss

Rule Path
Allow /

pagepeeker

Rule Path
Allow /

pagething

Rule Path
Allow /

peer39_crawler

Rule Path
Allow /

perman

Rule Path
Allow /

pingdom

Rule Path
Allow /

piplbot

Rule Path
Allow /

postrank

Rule Path
Allow /

pr-cy.ru

Rule Path
Allow /

primalbot

Rule Path
Allow /

privacyawarebot

Rule Path
Allow /

propowerbot

Rule Path
Allow /

prowebwalker

Rule Path
Allow /

proxem

Rule Path
Allow /

psbot

Rule Path
Allow /

pulsepoint

Rule Path
Allow /

purebot

Rule Path
Allow /

queryn metasearch

Rule Path
Allow /

qwam content intelligence

Rule Path
Allow /

radiation retriever 1.1

Rule Path
Allow /

rankactivelinkbot

Rule Path
Allow /

rankflex

Rule Path
Allow /

refindbot

Rule Path
Allow /

regionstuttgartbot

Rule Path
Allow /

repomonkey

Rule Path
Allow /

repomonkey bait & tackle

Rule Path
Allow /

retrevopageanalyzer

Rule Path
Allow /

reverseengineeringbot

Rule Path
Allow /

ridderbot

Rule Path
Allow /

riddler

Rule Path
Allow /

rivva

Rule Path
Allow /

robozilla

Rule Path
Allow /

rssingbot

Rule Path
Allow /

rukicrawler

Rule Path
Allow /

ruxitsynthetic

Rule Path
Allow /

rytebot

Rule Path
Allow /

safednsbot

Rule Path
Allow /

safesearch microdata crawler

Rule Path
Allow /

sbl-bot

Rule Path
Allow /

score3

Rule Path
Allow /

scoutjet

Rule Path
Allow /

scribdbot

Rule Path
Allow /

scrubby

Rule Path
Allow /

search.marginalia.nu

Rule Path
Allow /

searchatlas

Rule Path
Allow /

searchmetricsbot

Rule Path
Allow /

searchpreview

Rule Path
Allow /

seekbot

Rule Path
Allow /

seekport crawler

Rule Path
Allow /

seekr

Rule Path
Allow /

seewithkids

Rule Path
Allow /

semanticbot

Rule Path
Allow /

sempi.tech

Rule Path
Allow /

semrushbot-bm

Rule Path
Allow /

semrushbot-sa

Rule Path
Allow /

sentibot

Rule Path
Allow /

seokicks-robot

Rule Path
Allow /

seoscanners

Rule Path
Allow /

seostar.co

Rule Path
Allow /

seostats

Rule Path
Allow /

simplecrawler

Rule Path
Allow /

simplescraper

Rule Path
Allow /

sindup

Rule Path
Allow /

sistrix crawler

Rule Path
Allow /

sitebot

Rule Path
Allow /

sitecheck.internetseer.com

Rule Path
Allow /

siteexplorer.info

Rule Path
Allow /

siteimprove

Rule Path
Allow /

siteimprove.com

Rule Path
Allow /

sitesnagger

Rule Path
Allow /

sitesucker

Rule Path
Allow /

slack-imgproxy

Rule Path
Allow /

slackbot

Rule Path
Allow /

socialrankiobot

Rule Path
Allow /

sonic

Rule Path
Allow /

sosospider

Rule Path
Allow /

spankbot

Rule Path
Allow /

spanner

Rule Path
Allow /

spbot

Rule Path
Allow /

spinn3r

Rule Path
Allow /

spotter

Rule Path
Allow /

sputnikbot

Rule Path
Allow /

storebot-google

Rule Path
Allow /

storygizebot

Rule Path
Allow /

stractbot

Rule Path
Allow /

streamline3bot

Rule Path
Allow /

summalybot

Rule Path
Allow /

summify

Rule Path
Allow /

superbot

Rule Path
Allow /

surveybot

Rule Path
Allow /

suzuran

Rule Path
Allow /

swiftbot

Rule Path
Allow /

swimgbot

Rule Path
Allow /

synthesio

Rule Path
Allow /

sysomos

Rule Path
Allow /

szukacz

Rule Path
Allow /

taboolabot

Rule Path
Allow /

tagoobot

Rule Path
Allow /

talkwater

Rule Path
Allow /

tangibleebot

Rule Path
Allow /

teleport

Rule Path
Allow /

teleportpro

Rule Path
Allow /

telesoft

Rule Path
Allow /

the intraformant

Rule Path
Allow /

thenomad

Rule Path
Allow /

theoldreader.com

Rule Path
Allow /

thinklab

Rule Path
Allow /

tigerbot

Rule Path
Allow /

titan

Rule Path
Allow /

tocrawl

Rule Path
Allow /

tombapublicwebcrawler

Rule Path
Allow /

toplistbot

Rule Path
Allow /

toutiaospider

Rule Path
Allow /

traackr.com

Rule Path
Allow /

tracemyfile

Rule Path
Allow /

trafilatura

Rule Path
Allow /

trendeo

Rule Path
Allow /

trendkite-akashic-crawler

Rule Path
Allow /

trendybuzz

Rule Path
Allow /

trovitbot

Rule Path
Allow /

true_robot

Rule Path
Allow /

truliabot

Rule Path
Allow /

turingos

Rule Path
Allow /

tweetedtimes

Rule Path
Allow /

twengabot

Rule Path
Allow /

twurly

Rule Path
Allow /

ubicrawler

Rule Path
Allow /

um-ic

Rule Path
Allow /

updownerbot

Rule Path
Allow /

upflow

Rule Path
Allow /

uptime-kuma

Rule Path
Allow /

uptimebot.org

Rule Path
Allow /

uptimerobot

Rule Path
Allow /

url control

Rule Path
Allow /

url_spider_pro

Rule Path
Allow /

urlappendbot

Rule Path
Allow /

urly warning

Rule Path
Allow /

usasearch

Rule Path
Allow /

usinenouvellecrawler

Rule Path
Allow /

ut-dorkbot

Rule Path
Allow /

validator.nu

Rule Path
Allow /

vci

Rule Path
Allow /

vci webviewer vci webviewer win32

Rule Path
Allow /

vebidoobot

Rule Path
Allow /

vecteurplus

Rule Path
Allow /

veoozbot

Rule Path
Allow /

verticalsearch

Rule Path
Allow /

vigil

Rule Path
Allow /

vkrobot

Rule Path
Allow /

voilabot

Rule Path
Allow /

voltron

Rule Path
Allow /

voluumdsp-content-bot

Rule Path
Allow /

vsw

Rule Path
Allow /

vuhuvbot

Rule Path
Allow /

w3c_i18n-checker

Rule Path
Allow /

w3c_unicorn

Rule Path
Allow /

w3c-checklink

Rule Path
Allow /

w3c-mobileok

Rule Path
Allow /

wasalive-bot

Rule Path
Allow /

wbsearchbot

Rule Path
Allow /

web image collector

Rule Path
Allow /

web-archive-net.com.bot

Rule Path
Allow /

webauto

Rule Path
Allow /

webbandit

Rule Path
Allow /

webcapture 2.0

Rule Path
Allow /

webcompanycrawler

Rule Path
Allow /

webcopier

Rule Path
Allow /

webcopier v.2.2

Rule Path
Allow /

webcopier v3.2a

Rule Path
Allow /

webdatastats

Rule Path
Allow /

webenhancer

Rule Path
Allow /

webmasterworldforumbot

Rule Path
Allow /

webmon

Rule Path
Allow /

webreaper

Rule Path
Allow /

websauger

Rule Path
Allow /

website quester

Rule Path
Allow /

webstripper

Rule Path
Allow /

webzip

Rule Path
Allow /

winello

Rule Path
Allow /

winhttrack

Rule Path
Allow /

wiseguys robot

Rule Path
Allow /

wocbot

Rule Path
Allow /

woobot

Rule Path
Allow /

woorankreview

Rule Path
Allow /

wordupinfosearch

Rule Path
Allow /

woriobot

Rule Path
Allow /

wotbox

Rule Path
Allow /

www-collector-e

Rule Path
Allow /

www-mechanize

Rule Path
Allow /

www.uptime.com

Rule Path
Allow /

xenu

Rule Path
Allow /

xenu link sleuth

Rule Path
Allow /

xenu's

Rule Path
Allow /

xenu's link sleuth 1.1c

Rule Path
Allow /

xovibot

Rule Path
Allow /

yahoo pipes 1.0

Rule Path
Allow /

yak

Rule Path
Allow /

yandexmobilebot

Rule Path
Allow /

yandexvideo

Rule Path
Allow /

yanga

Rule Path
Allow /

yellowbrandprotectionbot

Rule Path
Allow /

yoozbot

Rule Path
Allow /

youdaobot

Rule Path
Allow /

youmag

Rule Path
Allow /

zabbix

Rule Path
Allow /

zao

Rule Path
Allow /

zealbot

Rule Path
Allow /

zenback bot

Rule Path
Allow /

zeus

Rule Path
Allow /

zeus link scout

Rule Path
Allow /

zgrab

Rule Path
Allow /

zite

Rule Path
Allow /

zuperlistbot

Rule Path
Allow /

zyborg

Rule Path
Allow /

cohere-ai

Rule Path
Allow /

barkrowler

Rule Path
Allow /

dotbot

Rule Path
Allow /

seoscanners.net

Rule Path
Allow /

yeti

Rule Path
Allow /

docoloc

Rule Path
Allow /

aibot

Rule Path
Allow /

alittle client

Rule Path
Allow /

aspseek

Rule Path
Allow /

abonti

Rule Path
Allow /

aboundexbot

Rule Path
Allow /

acunetix

Rule Path
Allow /

adstxtcrawlertp

Rule Path
Allow /

afd-verbotsverfahren

Rule Path
Allow /

aihitbot

Rule Path
Allow /

aipbot

Rule Path
Allow /

allsubmitter

Rule Path
Allow /

alligator

Rule Path
Allow /

anarchie

Rule Path
Allow /

anarchy

Rule Path
Allow /

anarchy99

Rule Path
Allow /

ankit

Rule Path
Allow /

anthill

Rule Path
Allow /

apexoo

Rule Path
Allow /

aspiegel

Rule Path
Allow /

asterias

Rule Path
Allow /

atomseobot

Rule Path
Allow /

attach

Rule Path
Allow /

bbbike

Rule Path
Allow /

bdfetch

Rule Path
Allow /

backstreet

Rule Path
Allow /

backweb

Rule Path
Allow /

backlink-ceck

Rule Path
Allow /

backlinksextendedbot

Rule Path
Allow /

badass

Rule Path
Allow /

bandit

Rule Path
Allow /

batchftp

Rule Path
Allow /

battleztar bazinga

Rule Path
Allow /

betabot

Rule Path
Allow /

bigfoot

Rule Path
Allow /

bitacle

Rule Path
Allow /

blackwidow

Rule Path
Allow /

blow

Rule Path
Allow /

boardreader

Rule Path
Allow /

bolt

Rule Path
Allow /

brandprotect

Rule Path
Allow /

brandwatch

Rule Path
Allow /

buddy

Rule Path
Allow /

builtwith

Rule Path
Allow /

buzzsumo

Rule Path
Allow /

catexplorador

Rule Path
Allow /

code87

Rule Path
Allow /

cshttp

Rule Path
Allow /

calculon

Rule Path
Allow /

cazoodlebot

Rule Path
Allow /

censysinspect

Rule Path
Allow /

cheteam

Rule Path
Allow /

chinaclaw

Rule Path
Allow /

chlooe

Rule Path
Allow /

cloud mapping

Rule Path
Allow /

cogentbot

Rule Path
Allow /

collector

Rule Path
Allow /

copier

Rule Path
Allow /

copyscape

Rule Path
Allow /

cosmos

Rule Path
Allow /

craftbot

Rule Path
Allow /

crawling at home project

Rule Path
Allow /

crazywebcrawler

Rule Path
Allow /

curious

Rule Path
Allow /

custo

Rule Path
Allow /

cyotekwebcopy

Rule Path
Allow /

dblbot

Rule Path
Allow /

diibot

Rule Path
Allow /

dsearch

Rule Path
Allow /

dts agent

Rule Path
Allow /

datacha0s

Rule Path
Allow /

databasedrivermysqli

Rule Path
Allow /

demon

Rule Path
Allow /

deusu

Rule Path
Allow /

devil

Rule Path
Allow /

digincore

Rule Path
Allow /

digitalpebble

Rule Path
Allow /

dirbuster

Rule Path
Allow /

disco

Rule Path
Allow /

discobot

Rule Path
Allow /

discoverybot

Rule Path
Allow /

dispatch

Rule Path
Allow /

dnbcrawler-analytics

Rule Path
Allow /

domcopbot

Rule Path
Allow /

domainappender

Rule Path
Allow /

domainsigmacrawler

Rule Path
Allow /

domains project

Rule Path
Allow /

dotbot

Rule Path
Allow /

download wonder

Rule Path
Allow /

dragonfly

Rule Path
Allow /

drip

Rule Path
Allow /

eccp/1.0

Rule Path
Allow /

email siphon

Rule Path
Allow /

email wolf

Rule Path
Allow /

easydl

Rule Path
Allow /

ebingbong

Rule Path
Allow /

ecxi

Rule Path
Allow /

eirgrabber

Rule Path
Allow /

evil

Rule Path
Allow /

express webpictures

Rule Path
Allow /

extractor

Rule Path
Allow /

extreme picture finder

Rule Path
Allow /

eyenetie

Rule Path
Allow /

fdm

Rule Path
Allow /

fhscan

Rule Path
Allow /

fimap

Rule Path
Allow /

firefox/7.0

Rule Path
Allow /

flashget

Rule Path
Allow /

flunky

Rule Path
Allow /

freeuploader

Rule Path
Allow /

frontpage

Rule Path
Allow /

fuzz

Rule Path
Allow /

fyberspider

Rule Path
Allow /

g-i-g-a-b-o-t

Rule Path
Allow /

gt

Rule Path
Allow /

galaxybot

Rule Path
Allow /

germcrawler

Rule Path
Allow /

getweb

Rule Path
Allow /

getintent

Rule Path
Allow /

go!zilla

Rule Path
Allow /

go-ahead-got-it

Rule Path
Allow /

gozilla

Rule Path
Allow /

gotit

Rule Path
Allow /

grabnet

Rule Path
Allow /

grabber

Rule Path
Allow /

grafula

Rule Path
Allow /

grapefx

Rule Path
Allow /

gridbot

Rule Path
Allow /

headmasterseo

Rule Path
Allow /

hmview

Rule Path
Allow /

htmlparser

Rule Path
Allow /

http

Rule Path
Allow /

haansoft

Rule Path
Allow /

haosouspider

Rule Path
Allow /

havij

Rule Path
Allow /

heritrix

Rule Path
Allow /

hloader

Rule Path
Allow /

honolulubot

Rule Path
Allow /

humanlinks

Rule Path
Allow /

hybridbot

Rule Path
Allow /

idbte4m

Rule Path
Allow /

idbot

Rule Path
Allow /

iblog

Rule Path
Allow /

id-search

Rule Path
Allow /

ilsebot

Rule Path
Allow /

image fetch

Rule Path
Allow /

image sucker

Rule Path
Allow /

indy library

Rule Path
Allow /

infotekies

Rule Path
Allow /

information security team infrasec scanner

Rule Path
Allow /

infrasec scanner

Rule Path
Allow /

intelliseek

Rule Path
Allow /

interget

Rule Path
Allow /

internetmeasurement

Rule Path
Allow /

internetseer

Rule Path
Allow /

internet ninja

Rule Path
Allow /

iria

Rule Path
Allow /

iskanie

Rule Path
Allow /

istellabot

Rule Path
Allow /

joc web spider

Rule Path
Allow /

jbrofuzz

Rule Path
Allow /

jetcar

Rule Path
Allow /

joomla

Rule Path
Allow /

jorgee

Rule Path
Allow /

justview

Rule Path
Allow /

jyxobot

Rule Path
Allow /

keybot translation-search-machine

Rule Path
Allow /

kinza

Rule Path
Allow /

kozmosbot

Rule Path
Allow /

lwp

Rule Path
Allow /

lanshanbot

Rule Path
Allow /

larbin

Rule Path
Allow /

leap

Rule Path
Allow /

leechftp

Rule Path
Allow /

leechget

Rule Path
Allow /

lftp

Rule Path
Allow /

libweb

Rule Path
Allow /

libwhisker

Rule Path
Allow /

liebaofast

Rule Path
Allow /

lightspeedsystems

Rule Path
Allow /

likse

Rule Path
Allow /

linkbot

Rule Path
Allow /

linksmanager

Rule Path
Allow /

linqiametadatadownloaderbot

Rule Path
Allow /

linqiarssbot

Rule Path
Allow /

linqiascrapebot

Rule Path
Allow /

lipperhey

Rule Path
Allow /

lipperhey spider

Rule Path
Allow /

litemage_walker

Rule Path
Allow /

lmspider

Rule Path
Allow /

ltx71

Rule Path
Allow /

mfc_tear_sample

Rule Path
Allow /

midown tool

Rule Path
Allow /

mqqbrowser

Rule Path
Allow /

msfrontpage

Rule Path
Allow /

mag-net

Rule Path
Allow /

magnet

Rule Path
Allow /

mail.ru_bot

Rule Path
Allow /

majestic-seo

Rule Path
Allow /

majestic12

Rule Path
Allow /

majestic seo

Rule Path
Allow /

markmonitor

Rule Path
Allow /

markwatch

Rule Path
Allow /

mass downloader

Rule Path
Allow /

masscan

Rule Path
Allow /

mb2345browser

Rule Path
Allow /

meanpath bot

Rule Path
Allow /

meanpathbot

Rule Path
Allow /

metauri

Rule Path
Allow /

micromessenger

Rule Path
Allow /

microsoft data access

Rule Path
Allow /

microsoft url control

Rule Path
Allow /

minefield

Rule Path
Allow /

moblie safari

Rule Path
Allow /

mojolicious

Rule Path
Allow /

molokaibot

Rule Path
Allow /

morfeus fucking scanner

Rule Path
Allow /

mozlila

Rule Path
Allow /

mr.4x3

Rule Path
Allow /

msrabot

Rule Path
Allow /

musobot

Rule Path
Allow /

npbot

Rule Path
Allow /

name intelligence

Rule Path
Allow /

nameprotect

Rule Path
Allow /

navroad

Rule Path
Allow /

nearsite

Rule Path
Allow /

needle

Rule Path
Allow /

nessus

Rule Path
Allow /

netlyzer

Rule Path
Allow /

netspider

Rule Path
Allow /

netzip

Rule Path
Allow /

net vampire

Rule Path
Allow /

netcraft

Rule Path
Allow /

nettrack

Rule Path
Allow /

nibbler

Rule Path
Allow /

niki-bot

Rule Path
Allow /

nikto

Rule Path
Allow /

nimbostratus

Rule Path
Allow /

ninja

Rule Path
Allow /

nmap

Rule Path
Allow /

nuclei

Rule Path
Allow /

octopus

Rule Path
Allow /

offline navigator

Rule Path
Allow /

oncrawl

Rule Path
Allow /

openlinkprofiler

Rule Path
Allow /

openvas

Rule Path
Allow /

openvas

Rule Path
Allow /

orangespider

Rule Path
Allow /

outfoxbot

Rule Path
Allow /

pecl

Rule Path
Allow /

phpcrawl

Rule Path
Allow /

poe-component-client-http

Rule Path
Allow /

pageanalyzer

Rule Path
Allow /

pagegrabber

Rule Path
Allow /

pagescorer

Rule Path
Allow /

pagething.com

Rule Path
Allow /

page analyzer

Rule Path
Allow /

panscient

Rule Path
Allow /

papa foto

Rule Path
Allow /

pavuk

Rule Path
Allow /

peoplepal

Rule Path
Allow /

pi-monster

Rule Path
Allow /

picscout

Rule Path
Allow /

picsearch

Rule Path
Allow /

picturefinder

Rule Path
Allow /

piepmatz

Rule Path
Allow /

pimonster

Rule Path
Allow /

pixray

Rule Path
Allow /

pleasecrawl

Rule Path
Allow /

pockey

Rule Path
Allow /

probethenet

Rule Path
Allow /

proximic

Rule Path
Allow /

psbot

Rule Path
Allow /

pu_in

Rule Path
Allow /

pump

Rule Path
Allow /

pxbroker

Rule Path
Allow /

pycurl

Rule Path
Allow /

quick-crawler

Rule Path
Allow /

rainbot

Rule Path
Allow /

rankactive

Rule Path
Allow /

rankingbot

Rule Path
Allow /

rankingbot2

Rule Path
Allow /

rankivabot

Rule Path
Allow /

rankurbot

Rule Path
Allow /

re-re

Rule Path
Allow /

reget

Rule Path
Allow /

realdownload

Rule Path
Allow /

reaper

Rule Path
Allow /

rebelmouse

Rule Path
Allow /

recorder

Rule Path
Allow /

redesscrapy

Rule Path
Allow /

ripper

Rule Path
Allow /

rocketcrawler

Rule Path
Allow /

rogerbot

Rule Path
Allow /

sbider

Rule Path
Allow /

seolyt

Rule Path
Allow /

seolyticscrawler

Rule Path
Allow /

seoprofiler

Rule Path
Allow /

sistrix

Rule Path
Allow /

salesintelligent

Rule Path
Allow /

scanalert

Rule Path
Allow /

scanbot

Rule Path
Allow /

screaming

Rule Path
Allow /

screenerbot

Rule Path
Allow /

screpybot

Rule Path
Allow /

searchestate

Rule Path
Allow /

seekport

Rule Path
Allow /

semanticjuice

Rule Path
Allow /

semrush

Rule Path
Allow /

sentibot

Rule Path
Allow /

seositecheckup

Rule Path
Allow /

seomoz

Rule Path
Allow /

shodan

Rule Path
Allow /

siphon

Rule Path
Allow /

siteexplorer

Rule Path
Allow /

sitelockspider

Rule Path
Allow /

site sucker

Rule Path
Allow /

sitebeam

Rule Path
Allow /

sitevigil

Rule Path
Allow /

slysearch

Rule Path
Allow /

smartdownload

Rule Path
Allow /

snake

Rule Path
Allow /

snoopy

Rule Path
Allow /

sociscraper

Rule Path
Allow /

sottopop

Rule Path
Allow /

spacebison

Rule Path
Allow /

spammen

Rule Path
Allow /

spanner

Rule Path
Allow /

spbot

Rule Path
Allow /

spider_bot

Rule Path
Allow /

spider_bot/3.0

Rule Path
Allow /

sqlmap

Rule Path
Allow /

sqlworm

Rule Path
Allow /

sqworm

Rule Path
Allow /

steeler

Rule Path
Allow /

stripper

Rule Path
Allow /

sucker

Rule Path
Allow /

sucuri

Rule Path
Allow /

superhttp

Rule Path
Allow /

surfbot

Rule Path
Allow /

suzuran

Rule Path
Allow /

t0phackteam

Rule Path
Allow /

t8abot

Rule Path
Allow /

telesphoreo

Rule Path
Allow /

telesphorep

Rule Path
Allow /

thumbor

Rule Path
Allow /

tighttwatbot

Rule Path
Allow /

tinytestbot

Rule Path
Allow /

toata

Rule Path
Allow /

toweyabot

Rule Path
Allow /

tracemyfile

Rule Path
Allow /

trendiction

Rule Path
Allow /

trendictionbot

Rule Path
Allow /

turingos

Rule Path
Allow /

twengabot

Rule Path
Allow /

twice

Rule Path
Allow /

typhoeus

Rule Path
Allow /

urly.warning

Rule Path
Allow /

unisterbot

Rule Path
Allow /

v-bot

Rule Path
Allow /

vb project

Rule Path
Allow /

vacuum

Rule Path
Allow /

vagabondo

Rule Path
Allow /

vericitecrawler

Rule Path
Allow /

vidiblescraper

Rule Path
Allow /

virusdie

Rule Path
Allow /

voideye

Rule Path
Allow /

voil

Rule Path
Allow /

voltron

Rule Path
Allow /

wbsearchbot

Rule Path
Allow /

webdav

Rule Path
Allow /

wisenutbot

Rule Path
Allow /

wpscan

Rule Path
Allow /

www

Rule Path
Allow /

wwwoffle

Rule Path
Allow /

wallpapers

Rule Path
Allow /

wallpapers/3.0

Rule Path
Allow /

wallpapershd

Rule Path
Allow /

wesee

Rule Path
Allow /

webcollage

Rule Path
Allow /

webfetch

Rule Path
Allow /

webfuck

Rule Path
Allow /

webgo is

Rule Path
Allow /

webimagecollector

Rule Path
Allow /

webleacher

Rule Path
Allow /

webpix

Rule Path
Allow /

websucker

Rule Path
Allow /

webwhacker

Rule Path
Allow /

web auto

Rule Path
Allow /

web collage

Rule Path
Allow /

web enhancer

Rule Path
Allow /

web fetch

Rule Path
Allow /

web fuck

Rule Path
Allow /

web pix

Rule Path
Allow /

web sauger

Rule Path
Allow /

web sucker

Rule Path
Allow /

webalta

Rule Path
Allow /

webshag

Rule Path
Allow /

websiteextractor

Rule Path
Allow /

websitequester

Rule Path
Allow /

webster

Rule Path
Allow /

whack

Rule Path
Allow /

whacker

Rule Path
Allow /

whatweb

Rule Path
Allow /

who.is bot

Rule Path
Allow /

widow

Rule Path
Allow /

wonderbot

Rule Path
Allow /

woobot

Rule Path
Allow /

wotbox

Rule Path
Allow /

wprecon

Rule Path
Allow /

xaldon webspider

Rule Path
Allow /

xaldon_webspider

Rule Path
Allow /

zade

Rule Path
Allow /

zauba

Rule Path
Allow /

zermelo

Rule Path
Allow /

zitebot

Rule Path
Allow /

zmeu

Rule Path
Allow /

zumbot

Rule Path
Allow /

zyborg

Rule Path
Allow /

arquivo-web-crawler

Rule Path
Allow /

arquivo.pt

Rule Path
Allow /

autoemailspider

Rule Path
Allow /

awario.com

Rule Path
Allow /

backlink-check

Rule Path
Allow /

cah.io.community

Rule Path
Allow /

check1.exe

Rule Path
Allow /

clark-crawler

Rule Path
Allow /

cognitiveseo

Rule Path
Allow /

com.plumanalytics

Rule Path
Allow /

crawler.feedback

Rule Path
Allow /

dataforseo.com

Rule Path
Allow /

dataforseobot

Rule Path
Allow /

demandbase-bot

Rule Path
Allow /

domainsproject.org

Rule Path
Allow /

ecatch

Rule Path
Allow /

facebookscraper

Rule Path
Allow /

gopher

Rule Path
Allow /

heritrix

Rule Path
Allow /

imagesift.com

Rule Path
Allow /

instabid

Rule Path
Allow /

internetvista monitor

Rule Path
Allow /

ips-agent

Rule Path
Allow /

isitwp.com

Rule Path
Allow /

iubenda-radar

Rule Path
Allow /

lwp-request

Rule Path
Allow /

meanpathbot

Rule Path
Allow /

mediawords

Rule Path
Allow /

muhstik-scan

Rule Path
Allow /

obot

Rule Path
Allow /

page scorer

Rule Path
Allow /

pcbrowser

Rule Path
Allow /

plumanalytics

Rule Path
Allow /

polaris version

Rule Path
Allow /

probe-image-size

Rule Path
Allow /

ripz

Rule Path
Allow /

s1z.ru

Rule Path
Allow /

satoristudio.net

Rule Path
Allow /

scalaj-http

Rule Path
Allow /

scan.lol

Rule Path
Allow /

seobility

Rule Path
Allow /

seocompany.store

Rule Path
Allow /

seostar

Rule Path
Allow /

sexsearcher

Rule Path
Allow /

sitechecker.pro

Rule Path
Allow /

siteripz

Rule Path
Allow /

sp_auditbot

Rule Path
Allow /

spyfu

Rule Path
Allow /

sysscan

Rule Path
Allow /

takeout

Rule Path
Allow /

trendiction.com

Rule Path
Allow /

trendiction.de

Rule Path
Allow /

ubermetrics-technologies.com

Rule Path
Allow /

voyagerx.com

Rule Path
Allow /

webgains-bot

Rule Path
Allow /

webmeup-crawler

Rule Path
Allow /

webpros.com

Rule Path
Allow /

webprosbot

Rule Path
Allow /

x09mozilla

Rule Path
Allow /

x22mozilla

Rule Path
Allow /

xpymep1.exe

Rule Path
Allow /

zauba.io

Rule Path
Allow /

*

Rule Path
Allow /
Allow /blog/
Allow /blog/feed/
Allow /text/pdf/
Allow /text/
Allow /blog/wp-admin/admin-ajax.php
Allow /en/
Allow /was-ist-unaique.net.html
Allow /textgenerator-fuer-copywriting-mit-ki.html
Allow /en/copywriter-powered-by-chatgpt-gpt4.html
Allow /blog/wp-admin/admin-ajax.php
Allow /ads.txt
Allow /llms.txt
Allow /llms.xml
Allow /llms.md
Allow /sitemap_text_all.php
Allow /sitemap.php
Allow /sitemap_b.xml
Allow /sitemap_pdf.php
Allow /blog/sitemap_index.xml
Allow /blog/page-sitemap.xml
Allow /blog/category-sitemap.xml
Allow /blog/post-sitemap.xml
Disallow /blog/wp-admin/
Disallow /aitextgenerator.php
Disallow /redirect.php
Disallow /copywriting.php
Disallow /copywriter.php
Disallow /agb.html
Disallow /ajax/*
Disallow /api/*
Disallow /authentification/*
Disallow /de/
Disallow /de/*
Disallow /redirect.php*
Disallow /blog/wp-admin/
Disallow /blog/xmlrpc.php
Disallow /blog/wp-login.php
Disallow /blog/search/*
Disallow /blog/?s=*
Disallow /a*_*1/*
Disallow /w*_*1/*
Disallow /ajax/api/*
Disallow /marketing/*
Disallow /login.php
Disallow /jobs/
Disallow /en/api/
Disallow /en/team/
Disallow /t/*
Disallow /js/s.min.js
Disallow /js/lt.min.js
Disallow /t/t.php
Disallow /t/s.php
Back to top

Other Records

Field Value
sitemap https://www.unaique.net/sitemap.php
sitemap https://www.unaique.net/blog/sitemap_index.xml
sitemap https://www.unaique.net/blog/page-sitemap.xml
sitemap https://www.unaique.net/blog/category-sitemap.xml
sitemap https://www.unaique.net/blog/post-sitemap.xml
sitemap https://www.artikelschreiben.com/sitemap_b.xml
sitemap https://www.artikelschreiben.com/sitemap_text_all.php
sitemap https://www.unaique.net/sitemap_pdf.php
Back to top

Comments

  • ArtikelSchreiber.com, ArtikelSchreiben.com, UNAIQUE.NET, UNAIQUE.COM, SebastianEnger.com
  • Topic: Dynamic Robots.txt Generator - Version: 13.03.2025 - 1.0.0.0.4
  • Copyright: M.Sc. Sebastian Enger
  • Contact: https://www.artikelschreiber.com/contact/
  • Updated: 2025-09-05T19:50:26Z
  • Server: www.unaique.net
  • ARTIKELSCHREIBER/UNAIQUE/REVIEWS
  • by Sebastian Enger, M.Sc.
  • Version vom: 10.01.2026 - 8.7.3.4.4
  • RESULTS:START
  • : [www.unaique.net]
Back to top

Warnings

  • 20 invalid lines.
Back to top