Log In Sign Up

bijutotal.com.br
robots.txt

Robots Exclusion Standard data for bijutotal.com.br

Resource Scan

Scan Details

Site Domain bijutotal.com.br
Base Domain bijutotal.com.br
Scan Status Ok
Last Scan2024-09-20T05:37:11+00:00
Next Scan 2024-10-20T05:37:11+00:00

Last Scan

Scanned2024-09-20T05:37:11+00:00
URL https://www.bijutotal.com.br/robots.txt
Domain IPs 108.157.254.39, 108.157.254.47, 108.157.254.48, 108.157.254.53, 2600:9000:2753:1c00:14:9907:3b40:93a1, 2600:9000:2753:2400:14:9907:3b40:93a1, 2600:9000:2753:4c00:14:9907:3b40:93a1, 2600:9000:2753:6c00:14:9907:3b40:93a1, 2600:9000:2753:7c00:14:9907:3b40:93a1, 2600:9000:2753:a600:14:9907:3b40:93a1, 2600:9000:2753:ae00:14:9907:3b40:93a1, 2600:9000:2753:e400:14:9907:3b40:93a1
Response IP 108.157.254.39
Found Yes
Hash 63643f94001cedf2b959290d6ef5ee085eea9f2aec06b2d350db70beda0f0b19
SimHash f030e94a68f0

Groups

baiduspider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

obot

Rule Path
Disallow /

addthis

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

embedly

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

bdcbot/1.0

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

sunrise

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

twengabot-2.0

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

amznkassocbot/4.0

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mixrankbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

riddler

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

psbot

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

hypercrawl

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

netseer

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

alexabot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

deusu

Rule Path
Disallow /

tdjbot

Rule Path
Disallow /

findxbot/1.0

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

y! j-asr

Rule Path
Disallow /

infoweb

Rule Path
Disallow /

nutch bots

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

unwindfetchor

Rule Path
Disallow /

flipboard

Rule Path
Disallow /

voltron

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

silk

Rule Path
Disallow /

*

Rule Path
Disallow /img/*
Disallow /account/*
Disallow /login/*
Disallow /checkout/*
Disallow /busca/*
Disallow /quick-view/*
Disallow /espiar/*
Back to top

Other Records

Field Value
sitemap https://www.bijutotal.com.br/sitemap.xml
Back to top

Warnings

  • 12 invalid lines.
  • `noindex` is not a known field.
Back to top