Log In Sign Up

sharedresearch.jp
robots.txt

Robots Exclusion Standard data for sharedresearch.jp

Resource Scan

Scan Details

Site Domain sharedresearch.jp
Base Domain sharedresearch.jp
Scan Status Ok
Last Scan2025-09-23T09:42:04+00:00
Next Scan 2025-10-23T09:42:04+00:00

Last Scan

Scanned2025-09-23T09:42:04+00:00
URL https://sharedresearch.jp/robots.txt
Domain IPs 104.26.14.143, 104.26.15.143, 172.67.75.106, 2606:4700:20::681a:e8f, 2606:4700:20::681a:f8f, 2606:4700:20::ac43:4b6a
Response IP 104.26.15.143
Found Yes
Hash 5dbb7ae101bfbcab59da7488cdc4a66a555d19c1f0a8eceb6559a03f773d9140
SimHash 50b49902ce52

Groups

*

Rule Path
Disallow

blexbot

Rule Path
Disallow /

chatgpt

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

aibot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

aipbot

Rule Path
Disallow /

gpt-3

Rule Path
Disallow /

ai-bot

Rule Path
Disallow /
Back to top

Comments

  • https://www.robotstxt.org/robotstxt.html
  • Let all crawlers to enter
  • identified from logs
  • Disallow list of AI crawlers
Back to top