joyfulhonda.jp
robots.txt

Robots Exclusion Standard data for joyfulhonda.jp

Resource Scan

Scan Details

Site Domain joyfulhonda.jp
Base Domain joyfulhonda.jp
Scan Status Ok
Last Scan2024-05-29T07:51:09+00:00
Next Scan 2024-06-28T07:51:09+00:00

Last Scan

Scanned2024-05-29T07:51:09+00:00
URL https://joyfulhonda.jp/robots.txt
Domain IPs 13.227.254.33, 13.227.254.47, 13.227.254.59, 13.227.254.95
Response IP 13.227.254.33
Found Yes
Hash 799237e3f12dcfbc54b36698a5dc35e6c82633550429cfca9271fa82d3aed1fa
SimHash 4b70dc618b5b

Groups

*

Rule Path
Disallow /admin.html
Disallow /item_download.html
Disallow /item_download_real.html
Disallow /cart_index.html
Disallow /cart_login.html
Disallow /cart_seisan.html
Disallow /cart_other_seisan.html
Disallow /cart_confirm.html
Disallow /apply.html?id=APPLY1
Disallow /category/00210008/
Allow /category/00210008/item/
Disallow /category/002100080001/
Allow /category/002100080001//item/
Disallow /category/000100160005/
Allow /category/000100160005/item/
Disallow /category/004300010001/
Disallow /category/004300020001/
Disallow /category/004300030001/
Disallow /category/004300040001/
Disallow /category/004300050001/
Disallow /category/004300060001/
Disallow /category/004300070001/
Disallow /category/004300080001/
Disallow /category/004300090001/
Disallow /category/004300100001/
Disallow /category/004300110001/
Disallow /category/004300120001/
Disallow /category/004300130001/

baiduspider

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

yetbot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

ahrefsbot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

mail.ru_bot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

ltx71

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

superfeedr bot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

feedly

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

psbot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

pocky

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

ccbot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

yak

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

icc-crawler

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

proximic

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

turnitinbot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

safednsbot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

admantx-apacas

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

paperlibot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

tweetmemebot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

archive.org_bot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

megaindex.ru

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

yandexbot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

smtbot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

ahrefsbot

Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 12