Log In Sign Up

empresaexterior.com
robots.txt

Robots Exclusion Standard data for empresaexterior.com

Resource Scan

Scan Details

Site Domain empresaexterior.com
Base Domain empresaexterior.com
Scan Status Ok
Last Scan2024-06-03T02:24:54+00:00
Next Scan 2024-06-10T02:24:54+00:00

Last Scan

Scanned2024-06-03T02:24:54+00:00
URL https://empresaexterior.com/robots.txt
Domain IPs 104.21.37.123, 172.67.208.27, 2606:4700:3034::6815:257b, 2606:4700:3037::ac43:d01b
Response IP 104.21.37.123
Found Yes
Hash b28994277a36b2175f6485d54cdd00fdca8c2557cb3d5ebc2249da1bfb1ce7be
SimHash 1306514184e8

Groups

neticle crawler

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

telegrambot

Rule Path
Disallow /

amazonadbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

sirdatabot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

buck

Rule Path
Disallow /

twitterbot

Rule Path
Disallow *
Allow /upload

moreover

Rule Path
Disallow /

test-bot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

linkdexbot/2.1

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

moodlebot/1.0

Rule Path
Disallow /

moodlebot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

adbeat_bot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot/0.1

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

nutch

Rule Path
Disallow /

dow jones searchbot

Rule Path
Disallow /

linkdex

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

linkdex.com/v2.0

Rule Path
Disallow /

flamingo_searchengine+(+http://www.flamingosearch.com/bot)

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

owlin bot v. 3.0

Rule Path
Disallow /

owlin bot v3

Rule Path
Disallow /

owlin bot

Rule Path
Disallow /

owlin

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

domainappender

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

orangebot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

fyberspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

niki-bot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

jobs.de-robot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

seodiver

Rule Path
Disallow /

spbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

obot

Rule Path
Disallow /

fr-crawler

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

cloudservermarketspider

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

lipperhey-kaus-australis

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

spiderbot

Rule Path
Disallow /

linkstats

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

iccrawler

Rule Path
Disallow /

plista

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

um-ic

Rule Path
Disallow /

mindupbot

Rule Path
Disallow /

sg-orbiter

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

kraken

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

openhosebot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

thumbsniper

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

implisensebot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

indeedbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

*

Rule Path
Disallow /app
Disallow /timeout

yoozbot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

*

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120
Back to top

Warnings

  • 10 invalid lines.
Back to top