contents.oricon.co.jp
robots.txt

Robots Exclusion Standard data for contents.oricon.co.jp

Resource Scan

Scan Details

Site Domain contents.oricon.co.jp
Base Domain oricon.co.jp
Scan Status Ok
Last Scan2024-06-04T21:10:10+00:00
Next Scan 2024-06-18T21:10:10+00:00

Last Scan

Scanned2024-06-04T21:10:10+00:00
URL https://contents.oricon.co.jp/robots.txt
Domain IPs 104.18.250.41, 104.18.251.41
Response IP 104.18.250.41
Found Yes
Hash 5e6f8d45a51bcecaad2985971b9c3cba591528351a050bab58f075122060fc33
SimHash e14653342f13

Groups

googlebot-news
*

Rule Path
Disallow /dealer/index.php*
Disallow /dealer/amazon*
Disallow /_cdsales/redirect.php*
Disallow /clickheat/
Disallow /pj/
Allow /pj/img/
Allow /pj/ln/cs/ft_cs.css
Allow /pj/ln/cs/ft_csnews.css
Allow /pj/ln/cs/hd_cs.css
Allow /pj/ln/cs/hd_csnews.css
Allow /pj/ln/cs/hd_englishcsnews.css
Allow /pj/ln/cs/hd_jukencsnews.css
Allow /pj/ln/cs/hd_life.css
Allow /pj/ln/cs/method_cs.css
Disallow /dealer/rakutenbooks*
Disallow /common/
Allow /common/js/

yahoo-mmcrawler

Rule Path
Disallow /

psbot

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

baidumobaider

Rule Path
Disallow /

taggy

Rule Path
Disallow /

*

Rule Path
Disallow /maintenance/

mail.ru_bot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow

Other Records

Field Value
sitemap http://www.oricon.co.jp/sitemap/sitemap_news_all_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_news_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_special_all_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_special_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_photo_all_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_photo_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_tag_all_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_prof_all_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_special_girls_all_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_special_girls_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_special_mens_all_index.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_special_mens_index.xml
sitemap http://www.oricon.co.jp/sitemap/rank/sitemap.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_other.xml
sitemap http://www.oricon.co.jp/sitemap/sitemap_amp.xml
sitemap http://www.oricon.co.jp/gglnews3sd5f4sda65f464.php

Comments

  • Baidu chinese search engine
  • RUSSIA search engine
  • sogou.com chinese search engine
  • User-agent: Sogou web spider
  • Disallow: /
  • Grapeshot Allow