ww38.mindillumination.com
robots.txt

Robots Exclusion Standard data for ww38.mindillumination.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	ww38.mindillumination.com
Base Domain	mindillumination.com
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Couldn't connect to server.
Last Scan	2025-10-30T23:55:54+00:00
Next Scan	2026-01-28T23:55:54+00:00

Last Successful Scan

Scanned	2023-12-18T03:03:20+00:00
URL	https://ww38.mindillumination.com/robots.txt
Domain IPs	104.21.37.251, 172.67.216.173, 2606:4700:3032::ac43:d8ad, 2606:4700:3037::6815:25fb
Response IP	104.21.37.251
Found	Yes
Hash	c2638717a929cf78ac4cb4b536c12da3a64e9eb858f1f72f2b34329165eab214
SimHash	3ad4a04248ca

Groups

lycos_spider_

Rule	Path
Disallow	/

Rule

Path

Disallow

fast-webcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

youdaobot

Rule	Path
Disallow	/

Rule

Path

Disallow

jikespider

Rule	Path
Disallow	/

Rule

Path

Disallow

sosospider

Rule	Path
Disallow	/

Rule

Path

Disallow

pangusospider

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider

Rule	Path
Disallow	/

Rule

Path

Disallow

sogou

Rule	Path
Disallow	/

Rule

Path

Disallow

jikespider

Rule	Path
Disallow	/

Rule

Path

Disallow

easouspider

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-sa

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

netestate ne crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

seznambot

Rule	Path
Disallow	/

Rule

Path

Disallow

exabot

Rule	Path
Disallow	/

Rule

Path

Disallow

seokicks-robot

Rule	Path
Disallow	/

Rule

Path

Disallow

archive.org_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

linkpadbot

Rule	Path
Disallow	/

Rule

Path

Disallow

aihitbot

Rule	Path
Disallow	/

Rule

Path

Disallow

safednsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

rogerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

youdaofeedfetcher

Rule	Path
Disallow	/

Rule

Path

Disallow

yyspider

Rule	Path
Disallow	/

Rule

Path

Disallow

wbsrch

Rule	Path
Disallow	/

Rule

Path

Disallow

wbsearchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

toutiaospider

Rule	Path
Disallow	/

Rule

Path

Disallow

surdotlybot

Rule	Path
Disallow	/

Rule

Path

Disallow

special_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

spbot

Rule	Path
Disallow	/

Rule

Path

Disallow

gigabot

Rule	Path
Disallow	/

Rule

Path

Disallow

siteexplorer

Rule	Path
Disallow	/

Rule

Path

Disallow

obot

Rule	Path
Disallow	/

Rule

Path

Disallow

mixrankbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mail.ru_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

gluten free crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

gimmeusabot

Rule	Path
Disallow	/

Rule

Path

Disallow

etaospider

Rule	Path
Disallow	/

Rule

Path

Disallow

deusu

Rule	Path
Disallow	/

Rule

Path

Disallow

dataprovider

Rule	Path
Disallow	/

Rule

Path

Disallow

dataforseobot

Rule	Path
Disallow	/

Rule

Path

Disallow

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Disallow	/*.js$
Disallow	/*.css$
Disallow	/wp-content/
Disallow	/wp-includes/

Rule

Path

Disallow

/*.js$

Disallow

/*.css$

Disallow

/wp-content/

Disallow

/wp-includes/

Warnings

2 invalid lines.

ww38.mindillumination.comrobots.txt

Resource Scan

Scan Details

Last Successful Scan

Groups

lycos_spider_

fast-webcrawler

youdaobot

jikespider

sosospider

pangusospider

baiduspider

sogou

jikespider

easouspider

semrushbot

semrushbot-sa

ahrefsbot

mj12bot

netestate ne crawler

seznambot

exabot

seokicks-robot

archive.org_bot

linkpadbot

aihitbot

safednsbot

rogerbot

youdaofeedfetcher

yyspider

wbsrch

wbsearchbot

toutiaospider

surdotlybot

special_archiver

spbot

gigabot

siteexplorer

obot

mixrankbot

mail.ru_bot

gluten free crawler

gimmeusabot

etaospider

deusu

dataprovider

dataforseobot

ia_archiver

*

Warnings

ww38.mindillumination.com
robots.txt