/.well-known/

Log In Sign Up

cursos.tiagomiarelli.com.br
robots.txt

Robots Exclusion Standard data for cursos.tiagomiarelli.com.br

Archived Snapshots

Resource Scan

Scan Details

Site Domain	cursos.tiagomiarelli.com.br
Base Domain	tiagomiarelli.com.br
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Server returned a client error.
Last Scan	2024-06-06T13:38:52+00:00
Next Scan	2024-09-04T13:38:52+00:00

Last Successful Scan

Scanned	2023-11-03T13:37:15+00:00
URL	https://cursos.tiagomiarelli.com.br/robots.txt
Redirect	https://publishers.academy/robots.txt
Redirect Domain	publishers.academy
Redirect Base	publishers.academy
Domain IPs	104.21.86.26, 172.67.214.60, 2606:4700:3032::6815:561a, 2606:4700:3032::ac43:d63c
Redirect IPs	31.220.97.71
Response IP	31.220.97.71
Found	Yes
Hash	bab9698eff81ae58457d604a7898824d3334137c949f6ea393615710ae89052c
SimHash	dc9570d16674

Groups

mauibot

Rule

Path

Disallow

/

blexbot

Rule

Path

Disallow

/

seo spider

Rule

Path

Disallow

/

yandexbot

Rule

Path

Disallow

/

alexawebsearchplatform

Rule

Path

Disallow

/

baiduspider

Rule

Path

Disallow

/

betabot

Rule

Path

Disallow

/

petalbot

Rule

Path

Disallow

/

dotbot

Rule

Path

Disallow

/

screaming frog seo spider

Rule

Path

Disallow

/

crawl

Rule

Path

Disallow

/

exabot

Rule

Path

Disallow

/

gigabot

Rule

Path

Disallow

/

infoseek sidewinder

Rule

Path

Disallow

/

linkchecker

Rule

Path

Disallow

/

netsongbot

Rule

Path

Disallow

/

dataforseobot

Rule

Path

Disallow

/

dataforseo-bot

Rule

Path

Disallow

/

mj12bot/

Rule

Path

Disallow

/

linkdexbot

Rule

Path

Disallow

/

adsbot

Rule

Path

Disallow

/

awariosmartbot

Rule

Path

Disallow

/

feedspot

Rule

Path

Disallow

/

sogou web spider

Rule

Path

Disallow

/

feedspot/1.0

Rule

Path

Disallow

/

keybot

Rule

Path

Disallow

/

zoominfobot

Rule

Path

Disallow

/

feedspot

Rule

Path

Disallow

/

checkhost

Rule

Path

Disallow

/

orbbot

Rule

Path

Disallow

/

yandeximages

Rule

Path

Disallow

/

wellknownbot

Rule

Path

Disallow

/

surdotlybot

Rule

Path

Disallow

/

gdnplus.com

Rule

Path

Disallow

/

online-webceo-bot

Rule

Path

Disallow

/

scrapy

Rule

Path

Disallow

/

rogerbot

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

dotbot

Rule

Path

Disallow

/

alexibot

Rule

Path

Disallow

/

surveybot

Rule

Path

Disallow

/

xenu

Rule

Path

Disallow

/

exabot

Rule

Path

Disallow

/

gigabot

Rule

Path

Disallow

/

blekkobot

Rule

Path

Disallow

/

mecrawler

Rule

Path

Disallow

/

turnitinbot

Rule

Path

Disallow

/

python-requests

Rule

Path

Disallow

/

python-urllib/2.7

Rule

Path

Disallow

/

curl/7.35.0

Rule

Path

Disallow

/

wp_is_mobile

Rule

Path

Disallow

/

java/11.0.10

Rule

Path

Disallow

/

photon

Rule

Path

Disallow

/

photon/1.0

Rule

Path

Disallow

/

serendeputybot

Rule

Path

Disallow

/

webpage-inspector.com

Rule

Path

Disallow

/

twingly recon

Rule

Path

Disallow

/

bidtellect/0.0.958.0

Rule

Path

Disallow

/

slackbot-linkexpanding 1.0

Rule

Path

Disallow

/

df bot 1.0

Rule

Path

Disallow

/

who.is bot

Rule

Path

Disallow

/

sem rush bot

Rule

Path

Disallow

/

yandex bot

Rule

Path

Disallow

/

sogou bot

Rule

Path

Disallow

/

majestic bot

Rule

Path

Disallow

/

exalead bot

Rule

Path

Disallow

/

baidu bot

Rule

Path

Disallow

/

mediamathbot/1.0

Rule

Path

Disallow

/

trendictionbot0

Rule

Path

Disallow

/

omgili/0.5

Rule

Path

Disallow

/

iframely

Rule

Path

Disallow

/

cortex/1.0

Rule

Path

Disallow

/

trendictionbot0.5.0

Rule

Path

Disallow

/

domains project/1.3.7

Rule

Path

Disallow

/

trendictionbot

Rule

Path

Disallow

/

mediamathbot

Rule

Path

Disallow

/

omgili

Rule

Path

Disallow

/

universalfeedparser/5.2.1

Rule

Path

Disallow

/

woorankreview/2.0

Rule

Path

Disallow

/

feedbot/1.0

Rule

Path

Disallow

/

seekportbot

Rule

Path

Disallow

/

siteauditbot

Rule

Path

Disallow

/

daum/4.1

Rule

Path

Disallow

/

paperlibot/2.1

Rule

Path

Disallow

/

aboutusbot

Rule

Path

Disallow

/

neevabot/1.0

Rule

Path

Disallow

/

feedly/1.0

Rule

Path

Disallow

/

heritrix/3.3.0

Rule

Path

Disallow

/

wget

Rule

Path

Disallow

/

microsoft.url.control

Rule

Path

Disallow

/

ubicrawler

Rule

Path

Disallow

/

icc-crawler

Rule

Path

Disallow

/

sitecheck.internetseer.com

Rule

Path

Disallow

/

zealbot

Rule

Path

Disallow

/

webstripper

Rule

Path

Disallow

/

webcopier

Rule

Path

Disallow

/

httrack

Rule

Path

Disallow

/

libwww

Rule

Path

Disallow

/

baiduspider-image

Rule

Path

Disallow

/

k2spider

Rule

Path

Disallow

/

megaindex.ru

Rule

Path

Disallow

/

megaindex.com

Rule

Path

Disallow

/

linguee bot

Rule

Path

Disallow

/

mappy

Rule

Path

Disallow

/

garlikcrawler

Rule

Path

Disallow

/

feedbot

Rule

Path

Disallow

/

feedlyapp

Rule

Path

Disallow

/

feedly

Rule

Path

Disallow

/

wordpress

Rule

Path

Disallow

/

heritrix

Rule

Path

Disallow

/

fullstorybot

Rule

Path

Disallow

/

ias-va/3.1

Rule

Path

Disallow

/

xpanse-bot

Rule

Path

Disallow

/

phxbot/0.1

Rule

Path

Disallow

/

http://2ip.io

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

mojeebot

Rule

Path

Disallow

/

siteexplorer

Rule

Path

Disallow

/

uptimebot

Rule

Path

Disallow

/

screaming frog seo spider

Rule

Path

Disallow

/

phxbot

Rule

Path

Disallow

/

zgrab/0.x

Rule

Path

Disallow

/

wappalyzer

Rule

Path

Disallow

/

checkmarknetwork/1.0

Rule

Path

Disallow

/

duckduckgo-favicons-bot/1.0;

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

gptbot

Rule

Path

Disallow

/

chatgptbot

Rule

Path

Disallow

/

researchscan.comsys.rwth-aachen.de

Rule

Path

Disallow

/

semrushbot/7

Rule

Path

Disallow

/

awariobot/1.0

Rule

Path

Disallow

/

bitsightbot/1.0

Rule

Path

Disallow

/

internetmeasurement/1.0

Rule

Path

Disallow

/

duckduckgo-favicons-bot/1.0

Rule

Path

Disallow

/

yak/1.0

Rule

Path

Disallow

/

uptimerobot/2.0

Rule

Path

Disallow

/

awariorssbot/1.0

Rule

Path

Disallow

/

awariosmartbot/1.0

Rule

Path

Disallow

/

repolookoutbot/v1

Rule

Path

Disallow

/

netcraftsurveyagent/1.0

Rule

Path

Disallow

/

stractbot/0.1

Rule

Path

Disallow

/

yeti/1.1

Rule

Path

Disallow

/

mediatoolkitbot

Rule

Path

Disallow

/

serpstatbot/2.1

Rule

Path

Disallow

/

webwikibot/2.1

Rule

Path

Disallow

/

anderspinkbot/1.1

Rule

Path

Disallow

/

pandalytics/1.0

Rule

Path

Disallow

/

anthropic-ai

Rule

Path

Disallow

/

researchscan

Rule

Path

Disallow

/

hstspreload-bot

Rule

Path

Disallow

/

censysinspect/1.1

Rule

Path

Disallow

/

apache-httpclient/4.5.13

Rule

Path

Disallow

/

python/3.10 aiohttp/3.8.5

Rule

Path

Disallow

/

alittle client

Rule

Path

Disallow

/

dalvik/2.1.0

Rule

Path

Disallow

/

python/3.10

Rule

Path

Disallow

/

curl/7.29.0

Rule

Path

Disallow

/

crystal

Rule

Path

Disallow

/

schema-markup-validator

Rule

Path

Disallow

/

ccbot

Rule

Path

Disallow

/

test-bot

Rule

Path

Disallow

/

startmebot/1.0

Rule

Path

Disallow

/

fidget-spinner-bot

Rule

Path

Disallow

/

guzzlehttp/7

Rule

Path

Disallow

/

claritybot

Rule

Path

Disallow

/

chatgpt-user

Rule

Path

Disallow

/

gptbot

Rule

Path

Disallow

/

google-extended

Rule

Path

Disallow

/

spinn3r

Rule

Path

Disallow

/

yahoo pipes 1.0

Rule

Path

Disallow

/

kscrawler

Rule

Path

Disallow

/

bytespider

Rule

Path

Disallow

/

amazonbot

Rule

Path

Disallow

/

newsnow

Rule

Path

Disallow

/

moodlebot

Rule

Path

Disallow

/

*

Rule

Path

Disallow

/admin/

Disallow

/auth/

Disallow

/assets/browser-update*.js

Disallow

/email/

Disallow

/session

Disallow

/user-api-key

Disallow

/*?api_key*

Disallow

/*?*api_key*

Disallow

/badges

Disallow

/u/

Disallow

/my

Disallow

/search

Disallow

/tag/*/l

Disallow

/g

Disallow

/t/*/*.rss

Disallow

/c/*.rss

googlebot

Rule

Path

Disallow

/admin/

Disallow

/auth/

Disallow

/assets/browser-update*.js

Disallow

/email/

Disallow

/session

Disallow

/user-api-key

Disallow

/*?api_key*

Disallow

/*?*api_key*

Back to top

Other Records

Field

Value

sitemap

https://publishers.academy/sitemap.xml

Back to top

Comments

See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file

Back to top

Warnings

4 invalid lines.

Back to top