olympus-ims.com
robots.txt
Robots Exclusion Standard data for olympus-ims.com
Resource Scan
Scan Details
| Site Domain | olympus-ims.com |
| Base Domain | olympus-ims.com |
| Scan Status | Ok |
| Last Scan | 2024-10-18T06:28:07+00:00 |
| Next Scan | 2024-11-17T06:28:07+00:00 |
Last Scan
| Scanned | 2024-10-18T06:28:07+00:00 |
| URL | https://olympus-ims.com/robots.txt |
| Redirect | https://www.olympus-ims.com/robots.txt |
| Redirect Domain | www.olympus-ims.com |
| Redirect Base | olympus-ims.com |
| Domain IPs | 139.144.22.12, 143.42.23.103, 173.255.252.125 |
| Redirect IPs | 139.144.22.12, 143.42.23.103, 173.255.252.125 |
| Response IP | 143.42.23.103 |
| Found | Yes |
| Hash | b865fab63a33408bab96d718b3c118886384368b42f6b17a1d46e1aa1a0f366f |
| SimHash | 46e228c08c6a |
Groups
mozilla/5.0 (compatible; seznam screenshot-generator 2.1; +http://fulltext.sblog.cz/screenshot/)
addthis.com robot tech.support@clearspring.com
msrbot (http://research.microsoft.com/research/sv/msrbot/)
msrbot
sandcrawler
shim-crawler
scoutjet
mozilla/5.0 (compatible; ahrefsbot/5.0; +http://ahrefs.com/robot/)
mozilla/5.0 (compatible; dotbot/1.1; http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org)
dotbot/1.0.1 (http://www.dotnetdotcom.org/
mozilla/5.0 (compatible; dotbot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)
findlinks
findlinks/2.0.1 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.6-beta6 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.6-beta4 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.6-beta1 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.5-beta7 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.4-beta1 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.3-beta9 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.3-beta8 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.3-beta6 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.3-beta4 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.3-beta2 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.3-beta1 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.2-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.1-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.1-a1 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1.1 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1-a9 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1-a8 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1-a8 ( http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1-a7 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1-a4 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1-a3 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.1 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.06 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.0.9 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.0.8 (+http://wortschatz.uni-leipzig.de/findlinks/)
findlinks/1.0 (+http://wortschatz.uni-leipzig.de/findlinks/)
mozilla/5.0 (compatible; heritrix/1.14.3 +http://www.searchtechnologies.com)
sputnikbot
accoona
addsugarspiderbot
anyapexbot
becomebot
beslistbot
billybobbot
bimbot
boitho
btbot
catchbot
cosmos
covario
cerberian
diamondbot
discobot
emeraldshield.com
envolk[its]spider
esperanzabot
exabot
fast enterprise crawler
fast-webcrawler
fdse robot
furlbot
fyberspider
g2crawler
gaisbot
galaxybot
geniebot
gigabot
girafabot
happyfunbot
holmes
htdig
iaskspider
ia_archiver
irlbot
issuecrawler
jaxified bot
jyxobot
koepabot
l.webis
lapozzbot
larbin
ldspider
linguee bot
linkwalker
lmspider
lwp-trivial
mabontland
magpie-crawler
mj12bot
mogimogi
mojeekbot
moreoverbot
morning paper
mvaclient
mxbot
netresearchserver
netseer crawler
newsgator
ng-search
nicebot
nusearch
nutchcvs
nymesis
obot
oegp
omgilibot
omniexplorer_bot
oozbot
orbiter
pagebiteshyperbot
peew
polybot
postpost
psbot - image search
pycurl
qseero
radian6
rampybot
rufusbot
sbider
scrubby
searchsight
seekbot
semanticdiscovery
sensis web crawler
seochat::bot
shopwiki
shoula robot
silk
sitebot
snappy
speedy spider
sqworm
suggybot
surveybot
terrawizbot
thesubot
tineye/1.1 (http://tineye.com/crawler.html)
tineye
truwogps
turnitinbot
tweetedtimes bot
twengabot
updated
urlfilebot
vagabondo
vortex
voyager
vyu2
webcollage
websquash.com
wf84
womlpefactory
xaldon_webspider
yacy
yasaklibot
yooglifetchagent
zao/0.1 (http://www.kototoi.org/zao/)
mozilla/4.0 (compatible; zealbot 1.0)
zspider/0.9-dev http://feedback.redkolibri.com/
mozilla/4.0 compatible zyborg/1.0 dlc (wn.zyborg@looksmart.net; http://www.wisenutbot.com)
mozilla/4.0 compatible zyborg/1.0 dead link checker (wn.zyborg@looksmart.net; http://www.wisenutbot.com)
mozilla/4.0 compatible zyborg/1.0 dead link checker (wn.dlc@looksmart.net; http://www.wisenutbot.com)
mozilla/4.0 compatible zyborg/1.0 (wn.zyborg@looksmart.net; http://www.wisenutbot.com)
mozilla/4.0 compatible zyborg/1.0 (wn-16.zyborg@looksmart.net; http://www.wisenutbot.com)
mozilla/4.0 compatible zyborg/1.0 (wn-14.zyborg@looksmart.net; http://www.wisenutbot.com)
| Rule | Path |
|---|---|
| Disallow | / |
ultraseek
| Rule | Path |
|---|---|
| Disallow | /*video/ |
| Disallow | /en/video/ |
| Disallow | /ja/video/ |
| Disallow | /fr/video/ |
| Disallow | /zh/video/ |
| Disallow | /de/video/ |
| Disallow | /it/video/ |
| Disallow | /cs/video/ |
| Disallow | /hu/video/ |
| Disallow | /es/video/ |
| Disallow | /ru/video/ |
| Disallow | /pl/video/ |
| Disallow | /pt/video/ |
| Disallow | /ko/video/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
baiduspider+(+http://www.baidu.com/search/spider_jp.html)
baiduspider+(+http://www.baidu.com/search/spider.htm)
baiduspider
baiduspider
baiduspider-video
baiduspider-image
sosospider+(+http://help.soso.com/webspider.htm)
sogou spider
sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm
mozilla/5.0 (compatible; easouspider; +http://www.easou.com/search/spider.html)
mozilla/5.0 (compatible; yodaobot/1.0; http://www.yodao.com/help/webmaster/spider/; )
| Product | Comment |
|---|---|
| sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm | 07) |
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; yandexbot/3.0; +http://yandex.com/bots)
igdespyder (compatible; igde.ru; +http://igde.ru/doc/tech.html)
stackrambler/2.0 (msie incompatible
stackrambler/2.0
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/4.0 (compatible; arachmo)
ichiro/4.0 (http://help.goo.ne.jp/door/crawler.html)
ichiro/3.0 (http://help.goo.ne.jp/door/crawler.html)
ichiro/2.0+(http://help.goo.ne.jp/door/crawler.html)
ichiro/2.0 (ichiro@nttr.co.jp)
ichiro/2.0 (http://help.goo.ne.jp/door/crawler.html)
| Rule | Path |
|---|---|
| Disallow | /fr/ |
| Disallow | /zh/ |
| Disallow | /de/ |
| Disallow | /it/ |
| Disallow | /cs/ |
| Disallow | /hu/ |
| Disallow | /es/ |
| Disallow | /ru/ |
| Disallow | /pl/ |
| Disallow | /pt/ |
| Disallow | /ko/ |
| Disallow | /vi/ |
| Disallow | /th/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |
abachobot
mozilla/4.0 (compatible; b-l-i-t-z-b-o-t)
iccrawler (http://www.iccenter.net/bot.htm)
synoobot/0.7.1 (synoobot; http://www.synoo.de/bot.html; webmaster@synoo.com)
fastbot.de crawler 2.0 beta (http://www.fastbot.de)
| Product | Comment |
|---|---|
| iccrawler (http://www.iccenter.net/bot.htm) | German job listing site |
| Rule | Path |
|---|---|
| Disallow | /ja/ |
| Disallow | /fr/ |
| Disallow | /zh/ |
| Disallow | /it/ |
| Disallow | /cs/ |
| Disallow | /hu/ |
| Disallow | /es/ |
| Disallow | /ru/ |
| Disallow | /pl/ |
| Disallow | /pt/ |
| Disallow | /ko/ |
| Disallow | /vi/ |
| Disallow | /th/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |
pompos/1.3 http://dir.com/pompos.html
pompos/1.2 http://pompos.iliad.fr
pompos/1.1 http://pompos.iliad.fr
mozilla/4.0 (compatible; msie 5.0; windows 95) voilabot beta 1.2 (http://www.voila.com/)
| Rule | Path |
|---|---|
| Disallow | /ja/ |
| Disallow | /zh/ |
| Disallow | /de/ |
| Disallow | /it/ |
| Disallow | /cs/ |
| Disallow | /hu/ |
| Disallow | /es/ |
| Disallow | /ru/ |
| Disallow | /pl/ |
| Disallow | /pt/ |
| Disallow | /ko/ |
| Disallow | /vi/ |
| Disallow | /th/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
seznambot/2.0 (+http://fulltext.seznam.cz/)
seznambot/2.0 (+http://fulltext.sblog.cz/robot/)
| Rule | Path |
|---|---|
| Disallow | /ja/ |
| Disallow | /fr/ |
| Disallow | /zh/ |
| Disallow | /de/ |
| Disallow | /it/ |
| Disallow | /hu/ |
| Disallow | /es/ |
| Disallow | /ru/ |
| Disallow | /pl/ |
| Disallow | /pt/ |
| Disallow | /ko/ |
| Disallow | /vi/ |
| Disallow | /th/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
yeti/1.0 (nhn corp.; http://help.naver.com/robots/)
yeti/1.0 (+http://help.naver.com/robots/)
mozilla/5.0 (compatible; msie or firefox mutant; not on windows server; + http://tab.search.daum.net/aboutwebsearch.html) daumoa/3.0
| Rule | Path |
|---|---|
| Disallow | /ja/ |
| Disallow | /fr/ |
| Disallow | /zh/ |
| Disallow | /de/ |
| Disallow | /it/ |
| Disallow | /cs/ |
| Disallow | /hu/ |
| Disallow | /es/ |
| Disallow | /ru/ |
| Disallow | /pl/ |
| Disallow | /pt/ |
| Disallow | /vi/ |
| Disallow | /th/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | /*/quote-request | google supports wildcards |
| Disallow | /en/quote-request/ | - |
| Disallow | /ja/quote-request/ | - |
| Disallow | /fr/quote-request/ | - |
| Disallow | /zh/quote-request/ | - |
| Disallow | /de/quote-request/ | - |
| Disallow | /it/quote-request/ | - |
| Disallow | /cs/quote-request/ | - |
| Disallow | /hu/quote-request/ | - |
| Disallow | /es/quote-request/ | - |
| Disallow | /ru/quote-request/ | - |
| Disallow | /pl/quote-request/ | - |
| Disallow | /pt/quote-request/ | - |
| Disallow | /ko/quote-request/ | - |
| Disallow | /vi/quote-request/ | - |
| Disallow | /th/quote-request/ | - |
| Disallow | /*/contact-us | google supports wildcards |
| Disallow | /en/products/contact-us/ | - |
| Disallow | /ja/products/contact-us/ | - |
| Disallow | /fr/products/contact-us/ | - |
| Disallow | /zh/products/contact-us/ | - |
| Disallow | /de/products/contact-us/ | - |
| Disallow | /it/products/contact-us/ | - |
| Disallow | /cs/products/contact-us/ | - |
| Disallow | /hu/products/contact-us/ | - |
| Disallow | /es/products/contact-us/ | - |
| Disallow | /ru/products/contact-us/ | - |
| Disallow | /pl/products/contact-us/ | - |
| Disallow | /pt/products/contact-us/ | - |
| Disallow | /ko/products/contact-us/ | - |
| Disallow | /vi/products/contact-us/ | - |
| Disallow | /th/products/contact-us/ | - |
| Disallow | /*/video-gallery/flaw-detection/ | - |
| Disallow | /*/video-gallery/thickness-gaging/ | - |
| Disallow | /*/video-gallery/corrosion-testing/ | - |
| Disallow | /*/video-gallery/remote-visual-inspection/ | - |
| Disallow | /*/video-gallery/high-speed-video/ | - |
| Disallow | /*/video-gallery/industrial-scanner/ | - |
| Disallow | /*/video-gallery/microscope-solutions/ | - |
| Disallow | /*/video-gallery/optical-metrology/ | - |
| Disallow | /*/video-gallery/integrated-inspection-systems/ | - |
| Disallow | /*/video-gallery/xrf-xrd-analyzers/ | - |
| Disallow | /en/404/ | - |
| Disallow | /ja/404/ | - |
| Disallow | /zh/404/ | - |
| Disallow | /es/404/ | - |
| Disallow | /de/404/ | - |
| Disallow | /ru/404/ | - |
| Disallow | /cs/404/ | - |
| Disallow | /it/404/ | - |
| Disallow | /hu/404/ | - |
| Disallow | /pl/404/ | - |
| Disallow | /pt/404/ | - |
| Disallow | /ko/404/ | - |
| Disallow | */?logout | - |
| Disallow | */.downloads/download/* | do not crawl files itself |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Warnings
- 1 invalid line.
Comments