Log In Sign Up

rainbow-blog.com
robots.txt

Robots Exclusion Standard data for rainbow-blog.com

Resource Scan

Scan Details

Site Domain rainbow-blog.com
Base Domain rainbow-blog.com
Scan Status Ok
Last Scan2025-11-14T03:52:29+00:00
Next Scan 2025-11-21T03:52:29+00:00

Last Scan

Scanned2025-11-14T03:52:29+00:00
URL https://rainbow-blog.com/robots.txt
Domain IPs 213.186.33.3
Response IP 213.186.33.3
Found Yes
Hash 255337c668e05db5a2a22df2c3a113c4e2ae3782f3d34d76546322a65bc18c07
SimHash 5015d2e3c743

Groups

*

Rule Path
Allow /

Other Records

Field Value Comment
crawl-delay 10 Attendre 10 secondes entre les demandes

meta-externalagent

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

atomic_email_hunter/4.0

Rule Path
Disallow /

atspider/1.0

Rule Path
Disallow /

autoemailspider

Rule Path
Disallow /

bwh3_user_agent

Rule Path
Disallow /

china local browse 2.6

Rule Path
Disallow /

contactbot/0.2

Rule Path
Disallow /

contentsmartz

Rule Path
Disallow /

datacha0s/2.0

Rule Path
Disallow /

dbrowse 1.4b

Rule Path
Disallow /

dbrowse 1.4d

Rule Path
Disallow /

demo bot dot 16b

Rule Path
Disallow /

demo bot z 16b

Rule Path
Disallow /

dsurf15a 01

Rule Path
Disallow /

dsurf15a 71

Rule Path
Disallow /

dsurf15a 81

Rule Path
Disallow /

dsurf15a va

Rule Path
Disallow /

ebrowse 1.4b

Rule Path
Disallow /

educate search vxb

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailspider

Rule Path
Disallow /

emailwolf 1.00

Rule Path
Disallow /

esurf15a 15

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

franklin locator 1.8

Rule Path
Disallow /

fsurf15a 01

Rule Path
Disallow /

full web bot 0416b

Rule Path
Disallow /

full web bot 0516b

Rule Path
Disallow /

full web bot 2816b

Rule Path
Disallow /

guestbook auto submitter

Rule Path
Disallow /

industry program 1.0.x

Rule Path
Disallow /

infoconveracrawler/0.8 ( http://www.authoritativeweb.com/crawl)

Rule Path
Disallow /

isc systems irc search 2.1

Rule Path
Disallow /

iupui research bot v 1.9a

Rule Path
Disallow /

larbin-experimental (efp@gmx.net)

Rule Path
Disallow /

letscrawl.com/1.0 +http://letscrawl.com/

Rule Path
Disallow /

lincoln state web browser

Rule Path
Disallow /

lmqueuebot/0.2

Rule Path
Disallow /

lwp::simple/5.803

Rule Path
Disallow /

mac finder 1.0.xx

Rule Path
Disallow /

mfc foundation class library 4.0

Rule Path
Disallow /

microsoft url control – 6.00.8xxx

Rule Path
Disallow /

missauga locate 1.0.0

Rule Path
Disallow /

missigua locator 1.9

Rule Path
Disallow /

missouri college browse

Rule Path
Disallow /

mizzu labs 2.2

Rule Path
Disallow /

mo college 1.9

Rule Path
Disallow /

mozilla/2.0 (compatible; newt activex; win32)

Rule Path
Disallow /

mozilla/3.0 (compatible)

Rule Path
Disallow /

mozilla/3.0 (compatible; indy library)

Rule Path
Disallow /

mozilla/3.0 (compatible; scan4mail (advanced version) http://www.peterspages.net/?scan4mail)

Rule Path
Disallow /

mozilla/4.0 (compatible; advanced email extractor v2.xx)

Rule Path
Disallow /

mozilla/4.0 (compatible; iplexx spider/1.0 http://www.iplexx.at)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 5.0; windows nt; digext; dts agent

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 5.5; windows nt 5.0) fetch api request

Rule Path
Disallow /

mozilla/4.0 efp@gmx.net

Rule Path
Disallow /

mozilla/4.08 [en] (win98; u ;nav)

Rule Path
Disallow /

mozilla/5.0 (version: xxxx type:xx)

Rule Path
Disallow /

mvaclient

Rule Path
Disallow /

nameofagent (cms spider)

Rule Path
Disallow /

nasa search 1.0

Rule Path
Disallow /

nsauditor/1.x

Rule Path
Disallow /

pbrowse 1.4b

Rule Path
Disallow /

peval 1.4b

Rule Path
Disallow /

poirot

Rule Path
Disallow /

port huron labs

Rule Path
Disallow /

production bot 0116b

Rule Path
Disallow /

production bot 2016b

Rule Path
Disallow /

production bot dot 3016b

Rule Path
Disallow /

program shareware 1.0.2

Rule Path
Disallow /

psurf15a 11

Rule Path
Disallow /

psurf15a 51

Rule Path
Disallow /

psurf15a va

Rule Path
Disallow /

psycheclone

Rule Path
Disallow /

rsurf15a 41

Rule Path
Disallow /

rsurf15a 51

Rule Path
Disallow /

rsurf15a 81

Rule Path
Disallow /

searchbot admin@google.com

Rule Path
Disallow /

shablastbot 1.0

Rule Path
Disallow /

snap.com beta crawler v0

Rule Path
Disallow /

snapbot/1.0

Rule Path
Disallow /

snapbot/1.0 (snap shots, +http://www.snap.com)

Rule Path
Disallow /

sogou develop spider

Rule Path
Disallow /

sogou orion spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm

Product Comment
sogou orion spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm 07)
Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm

Product Comment
sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm 07)
Rule Path
Disallow /

sohu agent

Rule Path
Disallow /

ssurf15a 11

Rule Path
Disallow /

tsurf15a 11

Rule Path
Disallow /

under the rainbow 2.2

Rule Path
Disallow /

user-agent: mozilla/4.0 (compatible; msie 6.0; windows nt 5.1)

Rule Path
Disallow /

vadixbot

Rule Path
Disallow /

webvulncrawl.unknown/1.0 libwww-perl/5.803

Rule Path
Disallow /

wells search ii

Rule Path
Disallow /

wep search 00

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://rainbow-blog.com/sitemap_index.xml
Back to top

Warnings

  • 2 invalid lines.
Back to top