jiji.com
robots.txt

Robots Exclusion Standard data for jiji.com

Resource Scan

Scan Details

Site Domain jiji.com
Base Domain jiji.com
Scan Status Ok
Last Scan2024-06-07T03:17:40+00:00
Next Scan 2024-06-14T03:17:40+00:00

Last Scan

Scanned2024-06-07T03:17:40+00:00
URL https://jiji.com/robots.txt
Redirect https://www.jiji.com/robots.txt
Redirect Domain www.jiji.com
Redirect Base jiji.com
Domain IPs 210.152.253.83
Redirect IPs 23.48.107.26, 23.48.107.50, 2600:1413:a000::1730:6b1a, 2600:1413:a000::1730:6b32
Response IP 23.44.5.106
Found Yes
Hash 32b3a0bfe7094907e620a2a80a882fd5a9c0cafd7aa73089eb461c9366f551e3
SimHash 22545705e171

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /*.imageparts$
Disallow /jc/m_
Allow /hall/
Allow /service/
Allow /c_profile/
Allow /jinji/

mediapartners-google

Rule Path
Allow /
Disallow /jc/m_

mediapartners

Rule Path
Allow /
Disallow /jc/m_

googlebot-image

Rule Path
Disallow /cgi-bin/
Disallow /jc/m_
Disallow /*imageparts$
Disallow /*.gif$
Allow /*.png$
Allow /*.jpg$
Allow /

googlebot

Rule Path
Disallow /cgi-bin/
Disallow /jc/m_
Disallow /jc/zc_forward
Disallow /jc/sake
Disallow /jc/search
Disallow /*.gif$
Disallow /*imageparts$
Allow /

msnbot

Rule Path
Disallow /cgi-bin/
Disallow /jc/m_
Disallow /jc/zc_forward
Disallow /jc/sake
Disallow /jc/search
Disallow /*.gif$
Disallow /*.png$
Disallow /*imageparts$
Allow /

bingbot

Rule Path
Disallow /cgi-bin/
Disallow /jc/m_
Disallow /jc/zc_forward
Disallow /jc/sake
Disallow /jc/search
Disallow /*.gif$
Disallow /*.png$
Disallow /*imageparts$
Allow /

baiduspider+(+http://www.baidu.jp/spider/)

Rule Path
Allow /
Disallow /jc/m_

baiduimagespider(+http://www.baidu.jp/spider/)

Rule Path
Allow /
Disallow /jc/m_

mozilla/5.0 (compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Rule Path
Allow /
Disallow /jc/m_

mozilla/5.0 (compatible; baiduspider/3.0; +http://www.baidu.com/search/spider.html)

Rule Path
Allow /
Disallow /jc/m_

mozilla/5.0 (compatible; baiduspider/4.0; +http://www.baidu.com/search/spider.html)

Rule Path
Allow /
Disallow /jc/m_

mozilla/5.0 (compatible; baiduspider/5.0; +http://www.baidu.com/search/spider.html)

Rule Path
Allow /
Disallow /jc/m_

baiduspider/2.0

Rule Path
Allow /
Disallow /jc/m_

baiduspider/3.0

Rule Path
Allow /
Disallow /jc/m_

baiduspider/4.0

Rule Path
Allow /
Disallow /jc/m_

baiduspider/5.0

Rule Path
Allow /
Disallow /jc/m_

baiduspider/2.0+(+http://www.baidu.com/search/spider.htm)

Rule Path
Allow /
Disallow /jc/m_

slurp

Rule Path
Allow /
Disallow /jc/m_

yahoo! slurp

Rule Path
Allow /
Disallow /jc/m_

yeti/1.0 (nhn corp.; http://help.naver.com/robots/)

Rule Path
Allow /
Disallow /jc/m_

y!j

Rule Path
Allow /
Disallow /jc/m_

popin_agent

Rule Path
Allow /
Disallow /jc/m_

applebot

Rule Path
Allow /
Disallow /jc/m_

twitterbot

Rule Path
Disallow /cgi-bin/
Disallow /jc/m_
Disallow /jc/zc_forward
Disallow /jc/sake
Disallow /jc/search
Disallow /*.gif$
Disallow /*imageparts$
Allow /

crowsnest

Rule Path
Allow /
Disallow /jc/m_

logly

Rule Path
Allow /
Disallow /jc/m_

clipper

Rule Path
Allow /
Disallow /jc/m_

grapeshotcrawler

Rule Path
Disallow /jc/m_

grapeshot

Rule Path
Disallow /jc/m_

zucks recommend engine crawler bot

Rule Path
Disallow /jc/m_

opebot-v2-beta (http://www.1plusx.com)

Rule Path
Allow /
Disallow /jc/m_

cxensebot

Rule Path
Allow /
Disallow /jc/m_

webtru_crawler

Rule Path
Allow /
Disallow /jc/m_

y!j-brj/yats crawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.jiji.com/sitemap.xml