fivestarguitars.com
robots.txt

Robots Exclusion Standard data for fivestarguitars.com

Resource Scan

Scan Details

Site Domain fivestarguitars.com
Base Domain fivestarguitars.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-16T14:36:48+00:00
Next Scan 2025-02-14T14:36:48+00:00

Last Successful Scan

Scanned2024-04-27T12:26:24+00:00
URL http://fivestarguitars.com/robots.txt
Domain IPs 3.209.14.119, 54.88.82.3
Response IP 3.209.14.119
Found Yes
Hash c47c7c58b8c1b0f2b6a305711646fd5910432d02cc612f5fd210c08d6553df70
SimHash d31c55fa7fb9

Groups

*

Rule Path
Disallow

Other Records

Field Value
crawl-delay 4

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

goodzer

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

dotbot
dotbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

checkmarknetwork/1.0 (+http://www.checkmarknetwork.com/spider.html)

Rule Path
Disallow /

ever accountable

Rule Path
Disallow /

seekportbot
claudbot
mauibot
houzzbot
baiduspider
baiduspider-image
serpstatbot
sogou blog
sogou inst spider
sogou news spider
sogou orion spider
sogou spider2
sogou web spider
uptimebot
yandex
yandexmobilebot
zoominfobot
megaindex.ru
alphaseobot-sa
proximic
amazonbot
petalbot
re-re studio
barkrowler
siteauditbot
bytespider

Rule Path
Disallow /

semrushbot-ba
semrushbot

Rule Path
Disallow /

Comments

  • See if Semrush can behave - March 2023.
  • Update Nov 2023, no, they can't behave.

Warnings

  • 2 invalid lines.