store.cityhill.co.jp
robots.txt
Robots Exclusion Standard data for store.cityhill.co.jp
Resource Scan
Scan Details
| Site Domain | store.cityhill.co.jp |
| Base Domain | cityhill.co.jp |
| Scan Status | Ok |
| Last Scan | 2024-11-03T09:44:56+00:00 |
| Next Scan | 2024-12-03T09:44:56+00:00 |
Last Scan
| Scanned | 2024-11-03T09:44:56+00:00 |
| URL | https://store.cityhill.co.jp/robots.txt |
| Domain IPs | 151.101.131.52, 151.101.195.52, 151.101.3.52, 151.101.67.52 |
| Response IP | 199.232.47.52 |
| Found | Yes |
| Hash | ce3290ce0a67b522fa49491fd49a10dc861dfc6dbd336133fa85d117a243be82 |
| SimHash | 501e55d2c483 |
Groups
a6-indexer
aboundex
aboundexbot
adsbot
advbot
ahrefsbot
aihitbot
alphabot
alphaseobot
alphaseobot-sa
amazonbot
applebot
archive.org_bot
awariobot
awariorssbot
awariosmartbot
baiduspider
baiduspider-image
baiduspider-video
barkrowler
bbot
betabot
bingbot
blexbot
brands-bot-logo
brightedge crawler
bubing
bytespider
ccbot
chatgpt-user
checkmarknetwork
cincraw
clarabot
clark-crawler
claude
claudebot
cliqzbot
coccocbot
coccocbot-image
coccocbot-web
contacts-crawler
crawler4j
dataforseobot
datanyze
dataprovider
daum
diffbot
dnyzbot
domaincrawler
domainstatsbot
dotbot
easouspider
ec2linkfinder
etaospider
exabot
experibot
extlinksbot
ezooms
ezooms robot
femtosearchbot
fidget-spinner-bot
findxbot
flfbaldrbot
funnelback
garlikcrawler
geedobot
geedoproductsearch
gocrawl
go-http-client
gptbot
gsa-crawler
haosouspider
hubspot
ia_archiver
icc-crawler
idbot
iframely
iframely/1.3.1
indeedbot
inetdex-bot
infopath
infopath.2
infotiger
infotigerbot
ioncrawl
ip-web-crawler.com
jambot
jamesbot
jikespider
jobboersebot
keybot translation-search-machine
linespider
linguee
linkpadbot
lipperhey
loadtimebot
lssrocketcrawler
ltx71
macocu
magpie-crawler
mail.ru
mappy
mauibot
meanpathbot
megaindex
megaindex.ru
megaindex.ru/2.0
memorybot
mixrankbot
mj12bot
mojeekbot
msnbot
naverbot
neevabot
netestate ne crawler
netestate ne crawler (+http://www.website-datenbank.de/)
nextgensearchbot
niocbot
nutch
oai-searchbot
obot
okhttp
owler
panscient.com
parsijoo-bot
perl lwp
petalbot
pimonster
piplbot
plukkie
psbot
qwantify
rankactivelinkbot
ranksonicbot
riddler
rogerbot
rukicrawler
safednsbot
scrapy
searchmetericsbot
seekbot
seeker
seekport
seekportbot
semanticscholarbot
semrushbot
senutobot
seokicks
seokicks-robot
serpstatbot
serptimizerbot
seznambot
sidetrade indexer bot
sistrix
sistrix crawler
sitebot
siteexplorer
slack-imgproxy
slurp
smtbot
sogou blog
sogou inst spider
sogou news spider
sogou orion spider
sogou spider2
sogou web spider
sosospider
spbot
stractbot
stractbot/0.1
surdotlybot
swebot
taptubot
tbot-nutch
the knowledge ai
thunderstone
tinytestbot
turnitin robot
turnitinbot
twengabot
twiceler
twitterbot
uptimebot
uptimerobot/2.0
velenpublicwebcrawler
vscooter
vuhuvbot
wbsearchbot
webprosbot
wiseguys robot
woorankreview
worldwebheritage.org
worldwebheritage.org/1.0
wotbox
xovibot
yacybot
yak
yandex
yeti
yeti/1.1
yisouspider
yoozbot
youdaobot
zoominfobot
facebookexternalhit
google-extended
meta-externalagent
meta-externalagent
facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
facebookexternalhit/1.1
facebot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /mypage/ |
| Disallow | /cart/ |
| Disallow | /register/ |
| Disallow | /coordinate/ |
| Disallow | /sp/mypage/ |
| Disallow | /sp/cart/ |
| Disallow | /sp/register/ |
| Disallow | /sp/coordinate/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 120 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://store.cityhill.co.jp/sitemap.xml |
Warnings
- 5 invalid lines.