medical.nikkeibp.co.jp
robots.txt

Robots Exclusion Standard data for medical.nikkeibp.co.jp

Resource Scan

Scan Details

Site Domain medical.nikkeibp.co.jp
Base Domain nikkeibp.co.jp
Scan Status Ok
Last Scan2024-06-03T13:04:04+00:00
Next Scan 2024-06-17T13:04:04+00:00

Last Scan

Scanned2024-06-03T13:04:04+00:00
URL https://medical.nikkeibp.co.jp/robots.txt
Domain IPs 43.206.55.160, 52.194.15.62
Response IP 43.206.55.160
Found Yes
Hash 5297953ba4e8c9a6dd8f216fe0ce644703d2ac3927926cc1aee14b52c23ddbad
SimHash 1a15d13010a7

Groups

googlebot-news

Rule Path
Disallow /leaf/mem/doctors/
Disallow /leaf/mem/*/cadetto/
Disallow */cancernavi/
Disallow */jitabata/
Disallow */dmensem/
Disallow */jwhospital/
Disallow */di/column/tool/
Disallow */special/forgp/
Disallow /images/other/static/
Disallow /all/parts2014/pc/static/

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3600

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3600

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3600

seekportbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3600

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3600

megalodon

Rule Path
Disallow /

sbooksnet

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

blexbot crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

woorank

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

*

Rule Path
Disallow /inc/$
Disallow /leaf/$
Disallow /*?pr=1
Disallow /*%26pr%3D1
Disallow /*.pdf$
Disallow */site_parts/
Disallow /all/parts/
Disallow /all/cms_parts/
Disallow /all/recommend/
Disallow */search.html?words=
Disallow /images/other/static/
Disallow /forceMailActivation.action
Disallow /member/force_mail_activation/*
Disallow /*.action
Disallow /auth/login_link
Disallow /all/parts2014/pc/static/
Disallow */cancernavi/breast/

Other Records

Field Value
sitemap https://medical.nikkeibp.co.jp/sitemap_index.xml
sitemap https://medical.nikkeibp.co.jp/all/sitemap/parts/auto/autositemap_cms_index_pages.xml
sitemap https://medical.nikkeibp.co.jp/all/sitemap/parts/auto/autositemap_column_index_pages.xml
sitemap https://medical.nikkeibp.co.jp/all/sitemap/parts/auto/autositemap_drugdic_index_pages.xml
sitemap https://medical.nikkeibp.co.jp/all/sitemap/parts/auto/autositemap_genre_index_pages.xml
sitemap https://medical.nikkeibp.co.jp/all/sitemap/parts/auto/autositemap_keyword_index_pages.xml

Comments

  • Crawl delay
  • Disallow ALL
  • Common