Log In Sign Up

job-calorie.com
robots.txt

Robots Exclusion Standard data for job-calorie.com

Resource Scan

Scan Details

Site Domain job-calorie.com
Base Domain job-calorie.com
Scan Status Ok
Last Scan2025-04-04T02:10:34+00:00
Next Scan 2025-04-18T02:10:34+00:00

Last Scan

Scanned2025-04-04T02:10:34+00:00
URL http://job-calorie.com/robots.txt
Redirect http://www.job-calorie.com/robots.txt
Redirect Domain www.job-calorie.com
Redirect Base job-calorie.com
Domain IPs 153.122.144.78
Redirect IPs 153.122.144.78
Response IP 153.122.144.78
Found Yes
Hash b1c6b56ac8ce7435f9ff770ea245e0e901b253c296015c01cfab44f35e40e98c
SimHash 60961941dfd0

Groups

yodaobot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mappy

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yahoomailproxy

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

openhosebot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

slackbot-linkexpanding

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

woobot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

telegrambot

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

bingbot

Rule Path
Disallow /

amazonbot

No rules defined. All paths allowed.

applebot

No rules defined. All paths allowed.

*

Rule Path
Disallow /.net/
Disallow /listing/
Disallow /js/
Disallow /manage/
Disallow /include/
Disallow /feed/
Disallow /indeed_xml/
Disallow /times_mail_xml/
Disallow /check_server/
Disallow /feed/
Disallow /works/_kengai/
Disallow /works/js/
Disallow /works/include/
Back to top

Other Records

Field Value
sitemap https://www.job-calorie.com/sitemap-index.xml
sitemap https://www.job-calorie.com/kagoshima/sitemap-index.xml
sitemap https://www.job-calorie.com/miyazaki/sitemap-index.xml
Back to top

Comments

  • China Yodao : www.yodao.com
  • China Baidu : www.baidu.com, www.baidu.jp
  • China Baidu : www.baidu.com, www.baidu.jp
  • Briten ahrefs : http://ahrefs.com/robot/
  • Sitemap: https://www.job-calorie.com/works/sitemap.xml
Back to top

Warnings

  • 3 invalid lines.
Back to top