bitcoin-newsletter.blogspot.com
robots.txt

Robots Exclusion Standard data for bitcoin-newsletter.blogspot.com

Resource Scan

Scan Details

Site Domain bitcoin-newsletter.blogspot.com
Base Domain bitcoin-newsletter.blogspot.com
Scan Status Ok
Last Scan2024-09-25T23:31:29+00:00
Next Scan 2024-10-02T23:31:29+00:00

Last Scan

Scanned2024-09-25T23:31:29+00:00
URL https://bitcoin-newsletter.blogspot.com/robots.txt
Domain IPs 142.251.175.132, 2404:6800:4003:c0f::84
Response IP 74.125.130.132
Found Yes
Hash 865ef86e8958175a2fbc5843447558dc8f1bd18ace6e7ee567346e0bba280dd2
SimHash 2c501c86c0e1

Groups

*

Rule Path
Disallow search
Allow

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow search
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

ahrefssiteaudit

Rule Path
Allow /

googlebot-image

Rule Path
Disallow

*

Rule Path
Disallow

feedburner/1.0

Rule Path
Disallow

parse

Rule Path
Disallow

a6-indexer 1.0

Rule Path
Disallow

twitterbot

Rule Path
Disallow

facebookexternalhit

Rule Path
Disallow

a6 corp user agents

Rule Path
Disallow

python

Rule Path
Disallow

streaming

Rule Path
Disallow

protocol

Rule Path
Disallow

adsbot-google
adsbot-google-mobile
adsbot-google-mobile-apps
adidxbot
applebot
applenewsbot
baiduspider
baiduspider-image
baiduspider-news
baiduspider-video
bingbot
bingpreview
bublupbot
ccbot
cliqzbot
coccoc
coccocbot-image
coccocbot-web
daumoa
dazoobot
deusu
duckduckbot
duckduckgo-favicons-bot
euripbot
exploratodo
facebookcatalog
facebookexternalhit
facebot
feedly
findxbot
googlebot
googlebot-image
googlebot-mobile
googlebot-news
googlebot-video
haosouspider
ichiro
istellabot
jikespider
lycos
mail.ru
mediapartners-google
microsoftpreview
mojeekbot
msnbot
msnbot-media
orangebot
pinterest
plukkie
qwantify
rambler
seznambot
sosospider
slurp
sogou blog
sogou inst spider
sogou news spider
sogou orion spider
sogou spider2
sogou web spider
sputnikbot
teoma
twitterbot
whatsapp
wotbox
yacybot
yandex
yandexmobilebot
yeti
yioopbot
yoozbot
youdaobot

Rule Path
Disallow

*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://bitcoin-newsletter.blogspot.com/sitemap.xml
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=501&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=1001&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=1501&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=2001&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=2501&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=3001&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=3501&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=4001&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=4501&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=5001&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=5501&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=6001&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=6501&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/atom.xml?redirect=false&start-index=7001&max-results=500
sitemap https://bitcoin-newsletter.blogspot.com/sitemap.xml

Comments

  • Crawlers of certain posts media sites are allowed to access page markup when google.com/imgres* links are shared.

Warnings

  • 3 invalid lines.