Log In Sign Up

blog.renatolucena.net
robots.txt

Robots Exclusion Standard data for blog.renatolucena.net

Resource Scan

Scan Details

Site Domain blog.renatolucena.net
Base Domain renatolucena.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-07-25T13:23:20+00:00
Next Scan 2025-10-23T13:23:20+00:00

Last Successful Scan

Scanned2024-09-05T20:17:45+00:00
URL https://blog.renatolucena.net/robots.txt
Domain IPs 104.21.2.90, 172.67.129.1, 2606:4700:3032::ac43:8101, 2606:4700:3033::6815:25a
Response IP 104.21.2.90
Found Yes
Hash a583caed89c319f4ae046be7ef4b0bc9d5b694195209b95676f64be4b532e51c
SimHash 4242fa99c475

Groups

applebot
baiduspider
bingbot
discordbot
facebookexternalhit
googlebot
googlebot-image
ia_archiver
linkedinbot
msnbot
naverbot
pinterestbot
screaming frog seo spider
seznambot
slurp
teoma
telegrambot
twitterbot
yandex
yeti

No rules defined. All paths allowed.

*

Rule Path
Disallow /login
Disallow /*.env$
Allow /*?lang=
Allow /hashtag/*?src=
Allow /search?q=%23

mediapartners-google

Rule Path
Allow /

rogerbot

Rule Path
Disallow /i/u

Other Records

Field Value
crawl-delay 10
crawl-delay 1

yandex
yandexblogs

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60
Back to top

Other Records

Field Value
sitemap https://blog.renatolucena.net/xml
Back to top

Comments

  • Adsense
  • Moz
  • WHAT-4882 - Block indexing of links in notification emails. This applies to all bots.
  • =====================================================================================
  • Wait 1 second between successive requests. See ONBOARD-2698 for details.
  • Independent of user agent. Links in the sitemap are full URLs using https:// and need to match
  • the protocol of the sitemap.
  • Yandex
  • Bingbot
  • .__________________________.
  • | .___________________. |==|
  • | | ................. | | |
  • | | ::[ Dear robot ]: | | |
  • | | ::::[ be nice ]:: | | |
  • | | ::::::::::::::::: | | |
  • | | ::::::::::::::::: | | |
  • | | ::::::::::::::::: | | |
  • | | ::::::::::::::::: | | ,|
  • | !___________________! |(c|
  • !_______________________!__!
  • / \
  • / [][][][][][][][][][][][][] \
  • / [][][][][][][][][][][][][][] \
  • ( [][][][][____________][][][][] )
  • \ ------------------------------ /
  • \______________________________/
Back to top

Warnings

  • 2 invalid lines.
  • `noindex` is not a known field.
Back to top