Log In Sign Up

1kuji.com
robots.txt

Robots Exclusion Standard data for 1kuji.com

Resource Scan

Scan Details

Site Domain 1kuji.com
Base Domain 1kuji.com
Scan Status Ok
Last Scan2024-10-31T13:26:03+00:00
Next Scan 2024-11-30T13:26:03+00:00

Last Scan

Scanned2024-10-31T13:26:03+00:00
URL https://1kuji.com/robots.txt
Domain IPs 18.180.100.16, 35.72.87.26
Response IP 35.72.87.26
Found Yes
Hash e4e113112981a05e0b7503f7453058f98175e2e8fbb91f4029ab7a2c8a969d9e
SimHash 5514f7032b02

Groups

gptbot
oai-searchbot
chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

yandexbot
yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

redekenbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

*

Rule Path
Disallow /mypage$
Disallow /mypage/
Disallow /login$
Disallow /logout$
Disallow /serials/
Back to top

Comments

  • 2023-08-10 initialize
  • 2024-10-18 add crawlers
  • OpenAI add 2023-08-10 mod 2024-10-18
  • ClaudeBot add 2024-10-18
  • DataForSEO add 2024-10-18
  • Yandex add 2024-10-18
  • Semrush add 2024-10-18
  • SERPSTAT add 2024-10-18
  • Awario add 2024-10-18
  • Redeken add 2024-10-18
  • Perplexity add 2024-10-18
  • Pinterest add 2024-10-18
  • YaCy add 2024-10-18
  • Configured for ALB
  • Meltwater add 2024-10-18
  • Configured for ALB
  • Cincraw add 2024-10-18
  • Configured for ALB
  • ALL
Back to top