Log In Sign Up

health-more.jp
robots.txt

Robots Exclusion Standard data for health-more.jp

Resource Scan

Scan Details

Site Domain health-more.jp
Base Domain health-more.jp
Scan Status Ok
Last Scan2024-09-14T21:10:19+00:00
Next Scan 2024-10-14T21:10:19+00:00

Last Scan

Scanned2024-09-14T21:10:19+00:00
URL https://health-more.jp/robots.txt
Domain IPs 13.35.18.111, 13.35.18.128, 13.35.18.38, 13.35.18.48
Response IP 13.35.18.38
Found Yes
Hash 439507beaa86b16aa55e538507b4fb45c915b20c943007b13c928beafe4f96c9
SimHash 98405cb042c1

Groups

mozilla/5.0 (compatible; cliqzbot/2.0; +http://cliqz.com/company/cliqzbot)

Rule Path
Disallow /

mozilla/5.0 (compatible; semrushbot/1.2~bl; +http://www.semrush.com/bot.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; dotbot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; mj12bot/v1.4.7; http://mj12bot.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/5.2; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; extlinksbot/1.5 +https://extlinks.com/bot.html)

Rule Path
Disallow /

barkrowler/0.5.1 (experimenting / debugging - sorry for your logs ) http://www.exensa.com/crawl - admin@exensa.com -- based on bubing

Rule Path
Disallow /

barkrowler/0.7 (+http://www.exensa.com/crawl)

Rule Path
Disallow /

mozilla/5.0 (compatible; yak/1.0; http://linkfluence.com/; bot@linkfluence.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; blexbot/1.0; +http://webmeup-crawler.com/)

Rule Path
Disallow /

icc-crawler/2.0 (mozilla-compatible; ; http://kc.nict.go.jp/project1/crawl.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; spbot/5.0.3; +http://openlinkprofiler.org/bot )

Rule Path
Disallow /

linguee bot (http://www.linguee.com/bot; bot@linguee.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; siteexplorer/1.1b; +http://siteexplorer.info/backlink-checker-spider/)

Rule Path
Disallow /

mozilla/5.0 (compatible; seokicks-robot; +http://www.seokicks.de/robot.html)

Rule Path
Disallow /

backlinkcrawler (http://www.backlinktest.com/crawler.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; linkdexbot/2.2; +http://www.linkdex.com/bots/)

Rule Path
Disallow /

mozilla/5.0 (compatible; yak/1.0; http://linkfluence.com/; bot@linkfluence.com)

Rule Path
Disallow /

mozilla/5.0 (windows nt 6.1) applewebkit/537.1 (khtml, like gecko) chrome/21.0.1180.89 safari/537.1; http://spyonweb.com spyonweb@gmail.com

Rule Path
Disallow /

mozilla/5.0 (compatible; smtbot/1.0; +http://www.similartech.com/smtbot)

Rule Path
Disallow /

mozilla/5.0 (compatible; alphabot/3.2; +http://alphaseobot.com/bot.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; gluten free crawler/1.0; +http://glutenfreepleasure.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; mappy/1.0; +http://mappydata.net/bot/)

Rule Path
Disallow /

bubing (+http://law.di.unimi.it/bubing.html)

Rule Path
Disallow /

sottopop/1.0(+https://upcontent.com/robots)

Rule Path
Disallow /

mozilla/5.0 (compatible; ucrawler/1.0 ; +https://blog.ucoz.ru/upolicy)

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

No rules defined. All paths allowed.

Back to top