Log In Sign Up

topbox.co.kr
robots.txt

Robots Exclusion Standard data for topbox.co.kr

Resource Scan

Scan Details

Site Domain topbox.co.kr
Base Domain topbox.co.kr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-12T04:25:42+00:00
Next Scan 2025-09-26T04:25:42+00:00

Last Successful Scan

Scanned2025-08-05T04:24:35+00:00
URL https://topbox.co.kr/robots.txt
Domain IPs 133.186.218.48
Response IP 133.186.218.48
Found Yes
Hash 11d84d855ce49ef06d9f5e161a212cece95b546fbe3d2052eb4342828badee5c
SimHash b0541003c782

Groups

*

Rule Path
Disallow /

googlebot
cowbot
naverbot
yeti
daumoa

Rule Path
Disallow /admin/
Disallow /config/
Disallow /module/
Disallow /tmp/
Disallow /data/

mj12bot
semrushbot
claudebot
gptbot

Rule Path
Disallow /

facebookexternalhit
dotbot
screaming frog seo spide
heritrix
bingbot
owasp
geofeed
dirbuster-1.0-rc1
pfinapp
droid build
googlebot
cowbot
yeti
ads-naver
blueno
daumoa

Rule Path
Disallow /admin/
Disallow /config/
Disallow /data/
Disallow /module/
Disallow /tmp/

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • Default Bot Policy List provided by GODOMALL
Back to top