Log In Sign Up

bymalina.com
robots.txt

Robots Exclusion Standard data for bymalina.com

Resource Scan

Scan Details

Site Domain bymalina.com
Base Domain bymalina.com
Scan Status Ok
Last Scan5/6/2025, 10:10:48 AM
Next Scan 6/5/2025, 10:10:48 AM

Last Scan

Scanned5/6/2025, 10:10:48 AM
URL https://bymalina.com/robots.txt
Domain IPs 104.22.30.123, 104.22.31.123, 172.67.7.161, 2606:4700:10::6816:1e7b, 2606:4700:10::6816:1f7b, 2606:4700:10::ac43:7a1
Response IP 104.22.31.123
Found Yes
Hash a2d85524944f82f327913a86a997d4859d1309043f7eafcd9028eca4928d4088
SimHash 865cfcd25037

Groups

*

Rule Path
Disallow /selection
Disallow /selection/*

adsbot-google

Rule Path
Disallow /selection
Disallow /selection/*

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefssiteaudit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

nutch

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

lexxebot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

alligator

Rule Path
Disallow /

sq webscanner

Rule Path
Disallow /

allsubmitter

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

anonymous

Rule Path
Disallow /

asterias

Rule Path
Disallow /

autoemailspider

Rule Path
Disallow /

badass

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

bitacle

Rule Path
Disallow /

dragonfly

Rule Path
Disallow /

dreampassport

Rule Path
Disallow /

dsurf

Rule Path
Disallow /

ebrowse

Rule Path
Disallow /

ecatch

Rule Path
Disallow /

edgeio

Rule Path
Disallow /

dreampassport

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

fsurf

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

gamespy_arcade

Rule Path
Disallow /

geniebot

Rule Path
Disallow /

getbot

Rule Path
Disallow /

getright

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

go!zilla

Rule Path
Disallow /

go-ahead-got-it

Rule Path
Disallow /

goforitbot

Rule Path
Disallow /

email extractor

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

jeteye

Rule Path
Disallow /

jeteyebot

Rule Path
Disallow /

jobo

Rule Path
Disallow /

joc web spider

Rule Path
Disallow /

kapere

Rule Path
Disallow /

larbin

Rule Path
Disallow /

leechget

Rule Path
Disallow /

lightningdownload

Rule Path
Disallow /

linkie

Rule Path
Disallow /

mac finder

Rule Path
Disallow /

mail sweeper

Rule Path
Disallow /

outfoxbot

Rule Path
Disallow /

omniexplorer

Rule Path
Disallow /

p3p

Rule Path
Disallow /

psycheclone

Rule Path
Disallow /

puf

Rule Path
Disallow /

pussycat

Rule Path
Disallow /

puxarapido

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

realdownload

Rule Path
Disallow /

redkernel

Rule Path
Disallow /

relevantnoise

Rule Path
Disallow /

y!tunnelpro

Rule Path
Disallow /

yahooysmcm

Rule Path
Disallow /

zade

Rule Path
Disallow /

zbot

Rule Path
Disallow /

zerxbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://bymalina.com/sitemap.xml
Back to top

Comments

  • Google Adsbot ignores robots.txt unless specifically named
  • Slow down some bots
  • Disallow bad bots
  • Fix for googlebot
Back to top