dev.spotation.com
robots.txt

Robots Exclusion Standard data for dev.spotation.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	dev.spotation.com
Base Domain	spotation.com
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Couldn't connect to server.
Last Scan	2025-06-07T20:17:23+00:00
Next Scan	2025-09-05T20:17:23+00:00

Last Successful Scan

Scanned	2022-01-17T21:17:10+00:00
URL	http://dev.spotation.com/robots.txt
Response IP	154.204.210.16
Found	Yes
Hash	577952457594e79d4810556c1b23686274eec6f30fc6a534ed1418687646c77b
SimHash	7e87c471e988

Groups

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex.ru

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

barkrowler

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

googlebot

Rule	Path
Disallow	/

Rule

Path

Disallow

bingbot

Rule	Path
Disallow	/

Rule

Path

Disallow

seokicks

Rule	Path
Disallow	/

Rule

Path

Disallow

serpstatbot

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

seekport

Rule	Path
Disallow	/

Rule

Path

Disallow

trendictionbot

Rule	Path
Disallow	/

Rule

Path

Disallow

aihitbot

Rule	Path
Disallow	/

Rule

Path

Disallow

yahoo

Rule	Path
Disallow	/

Rule

Path

Disallow

zoominfobot

Rule	Path
Disallow	/

Rule

Path

Disallow

facebookexternalhit

Rule	Path
Disallow	/

Rule

Path

Disallow

coccocbot

Rule	Path
Disallow	/

Rule

Path

Disallow

seznambot

Rule	Path
Disallow	/

Rule

Path

Disallow

pinterestbot

Rule	Path
Disallow	/

Rule

Path

Disallow

vuhuvbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mojeekbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mail.ru

Rule	Path
Disallow	/

Rule

Path

Disallow

slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/

Rule

Path

Disallow

yisouspider

Rule	Path
Disallow	/

Rule

Path

Disallow

msnbot

Rule	Path
Disallow	/

Rule

Path

Disallow

toutiaospider

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

feeddemon

Rule	Path
Disallow	/

Rule

Path

Disallow

bot/0.1 (bot for jce)

Rule	Path
Disallow	/

Rule

Path

Disallow

crawldaddy

Rule	Path
Disallow	/

Rule

Path

Disallow

java

Rule	Path
Disallow	/

Rule

Path

Disallow

jullo

Rule	Path
Disallow	/

Rule

Path

Disallow

feedly

Rule	Path
Disallow	/

Rule

Path

Disallow

universalfeedparser

Rule	Path
Disallow	/

Rule

Path

Disallow

apachebench

Rule	Path
Disallow	/

Rule

Path

Disallow

swiftbot

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Disallow	/h_m_w/
Disallow

Rule

Path

Disallow

/h_m_w/

Disallow

Other Records

Field	Value
sitemap	http://dev.spotation.com/sitemap.xml

Field

Value

sitemap

http://dev.spotation.com/sitemap.xml

dev.spotation.comrobots.txt

Resource Scan

Scan Details

Last Successful Scan

Groups

ahrefsbot

semrushbot

dotbot

mj12bot

megaindex.ru

yandexbot

barkrowler

blexbot

googlebot

bingbot

seokicks

serpstatbot

petalbot

seekport

trendictionbot

aihitbot

yahoo

zoominfobot

facebookexternalhit

coccocbot

seznambot

pinterestbot

vuhuvbot

mojeekbot

mail.ru

slurp

bytespider

yisouspider

msnbot

toutiaospider

petalbot

feeddemon

bot/0.1 (bot for jce)

crawldaddy

java

jullo

feedly

universalfeedparser

apachebench

swiftbot

*

Other Records

dev.spotation.com
robots.txt