kenbiya.com
robots.txt

Robots Exclusion Standard data for kenbiya.com

Resource Scan

Scan Details

Site Domain kenbiya.com
Base Domain kenbiya.com
Scan Status Ok
Last Scan2024-09-07T13:22:20+00:00
Next Scan 2024-10-07T13:22:20+00:00

Last Scan

Scanned2024-09-07T13:22:20+00:00
URL https://www.kenbiya.com/robots.txt
Domain IPs 13.230.235.32, 18.182.119.183
Response IP 13.230.235.32
Found Yes
Hash 35f76ede90bfc2c3967934f519c0a9e19c3727db64ac9078135b05fee15fee16
SimHash 6a7cf96285b5

Groups

velenpublicwebcrawler

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

lc-crawler

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /blog/c1%3D
Disallow /realestate/
Disallow /seminar/
Disallow /kaisha/*/pp*
Disallow /*.rdf$
Allow /pp0/saikenfuka%3Dy/
Disallow /pp0/saikenfuka%3Dy/*%3D
Disallow /pp*_*/eki-*/p2%3D*
Disallow /pp*_*/eki-*/r1%3D10/
Allow /pp*eki-*/p2%3D*
Allow /pp*eki-*/r1%3D10/
Disallow /pp*%3D
Disallow /pp*_
Allow /pp*/re_
Disallow /sm/*%3D
Disallow /sm/*_
Disallow /sm/*/t-*/pt-*/
Allow /sm/*/t-*/pt-*/dt_
Allow /sm/pay%3Dy/
Disallow /ar/*ac%3D
Disallow /news/p%3D
Disallow /app/exe/
Disallow /upload/
Disallow /map/
Disallow /realestate2/
Disallow /list/kw%3D
Disallow */NOCR%3DY*
Disallow /clm/*.html
Disallow /mr/*
Disallow /mb/*
Disallow /mc/*
Disallow /mt/*
Disallow /ms/*
Disallow /kaisha/*%3D
Disallow /kaishasm/*%3D
Disallow /column/
Allow /list/r1%3D15/z%3Dr2/

Other Records

Field Value
crawl-delay 60

*

Rule Path
Disallow /blog/c1%3D
Disallow /realestate/
Disallow /seminar/
Disallow /kaisha/*/pp*
Disallow /*.rdf$
Allow /pp0/saikenfuka%3Dy/
Disallow /pp0/saikenfuka%3Dy/*%3D
Disallow /pp*_*/eki-*/p2%3D*
Disallow /pp*_*/eki-*/r1%3D10/
Allow /pp*eki-*/p2%3D*
Allow /pp*eki-*/r1%3D10/
Disallow /pp*%3D
Disallow /pp*_
Allow /pp*/re_
Disallow /sm/*%3D
Disallow /sm/*_
Disallow /sm/*/t-*/pt-*/
Allow /sm/*/t-*/pt-*/dt_
Allow /sm/pay%3Dy/
Disallow /ar/*ac%3D
Disallow /news/p%3D
Disallow /app/exe/
Disallow /upload/
Disallow /map/
Disallow /realestate2/
Disallow /list/kw%3D
Disallow */NOCR%3DY*
Disallow /clm/*.html
Disallow /mr/*
Disallow /mb/*
Disallow /mc/*
Disallow /mt/*
Disallow /ms/*
Disallow /kaisha/*%3D
Disallow /kaishasm/*%3D
Disallow /column/
Allow /list/r1%3D15/z%3Dr2/

googlebot

Rule Path
Disallow /blog/c1%3D
Disallow /realestate/
Disallow /seminar/
Disallow /kaisha/*/pp*
Disallow /*.rdf$
Allow /pp0/saikenfuka%3Dy/
Disallow /pp0/saikenfuka%3Dy/*%3D
Disallow /pp*_*/eki-*/p2%3D*
Disallow /pp*_*/eki-*/r1%3D10/
Allow /pp*eki-*/p2%3D*
Allow /pp*eki-*/r1%3D10/
Disallow /pp*%3D
Disallow /pp*_
Allow /pp*/re_
Disallow /sm/*%3D
Disallow /sm/*_
Disallow /sm/*/t-*/pt-*/
Allow /sm/*/t-*/pt-*/dt_
Allow /sm/pay%3Dy/
Disallow /ar/*ac%3D
Disallow /news/p%3D
Disallow /app/exe/
Disallow /upload/
Disallow /map/
Disallow /realestate2/
Disallow /list/kw%3D
Disallow */NOCR%3DY*
Disallow /clm/*.html
Disallow /mr/*
Disallow /mb/*
Disallow /mc/*
Disallow /mt/*
Disallow /ms/*
Disallow */tl%3DY/
Disallow /column/
Allow /pp0/r1%3D15/z%3Dr2/
Disallow /kaisha/*%3D
Disallow /kaishasm/*%3D

Other Records

Field Value
sitemap https://www.kenbiya.com/sitemap_idx202210.xml

Warnings

  • 2 invalid lines.