worker.co.kr
robots.txt
Robots Exclusion Standard data for worker.co.kr
Resource Scan
Scan Details
| Site Domain | worker.co.kr |
| Base Domain | worker.co.kr |
| Scan Status | Ok |
| Last Scan | 2024-06-26T11:29:54+00:00 |
| Next Scan | 2024-07-03T11:29:54+00:00 |
Last Scan
| Scanned | 2024-06-26T11:29:54+00:00 |
| URL | https://worker.co.kr/robots.txt |
| Domain IPs | 210.122.7.129 |
| Response IP | 210.122.7.129 |
| Found | Yes |
| Hash | 387768e7b0ca4be5c3e9eaedf5e36ccba24c16c3dd31713355f81b5b9d765b69 |
| SimHash | 6b5c5750c61c |
Groups
ahrefsbot
amazonbot
arachni
blexbot
dotbot
exabot
eyeotabot
megaindex
mj12bot
petalbot
semrushbot
wordpress
bbot
brands-bot
dataforseo-bot
domaincrawler
alphabot
ltx71
ccbot
sogou
piplbot
mauibot
ezooms
sistrix
yandex
bytespider
acunetix
fhscan
baiduspider
java/
coccoc
criteo
mail.ru_bot
comscore
grapeshotcrawler
peer39
ias_crawler
criteobot
go-http-client/1.1
wirereader
compress-api
pinterest
magpie
lwp::simple
hatenablog
simplepie
spider
validator.nu
pocketparser
node-fetch
python-urllib/
python-requests/
malitanyo|yarnd
ias-va
ias-ie
ias-or
ias-sg
ias-jp
ias-au
censysinspect
bitlybot
duckduckgo
curl/
paloaltonetworks
awariobot
wpbot
yahoo ad monitoring
iframely/
imagesiftbot
y!j-wsc/
internetmeasurement/
seokicks
dataprovider
alittle client
barkrowler
scmguard
zgrab
wappalyzer
ev-crawler
surdotlybot
ia_archiver
zoominfobot
http banner detection
claudebot
applebot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /inc/ |
| Disallow | /findjob/ |
| Disallow | /findjob/mod.asp |
| Disallow | /findjob/mod2.asp |
| Disallow | /findjob/modw.asp |
| Disallow | /findjob/infomodw.asp |
| Disallow | /findjob/infomodw2.asp |
| Disallow | /findjob/input.asp |
| Disallow | /findjob/input2.asp |
| Disallow | /findjob/inputw.asp |
| Disallow | /findjob/list.asp |
| Disallow | /findjob/list_2.asp |
| Disallow | /findjob/list_3.asp |
| Disallow | /findjob/view.asp |
| Disallow | /findjob/view_3.asp |
| Disallow | /findjob/print_3.asp |
| Disallow | /findjob/list_up.asp |
| Disallow | /findjob/list_upgo.asp |
| Disallow | /join/ |
| Disallow | /join/pmy_info.asp |
| Disallow | /join/pmy_edit.asp |
| Disallow | /join/pmy_editw.asp |
| Disallow | /join/pmy_infomod.asp |
| Disallow | /join/pmy_infomodw.asp |
| Disallow | /join/pmy_infomodw2.asp |
| Disallow | /join/cmy_info.asp |
| Disallow | /join/cmy_edit.asp |
| Disallow | /join/cmy_editw.asp |
| Disallow | /join/cmy_infomod.asp |
| Disallow | /join/cmy_infomodw.asp |
| Disallow | /join/cmy_infomodw2.asp |
| Disallow | /join/cmy_findman.asp |
| Disallow | /admin/ |
| Disallow | /findman/ |
| Disallow | /board2004/ |
| Disallow | /pds/photo/ |
| Disallow | /pds/nomember/ |
| Disallow | /headhunting/ |
| Disallow | /pdsbackup/ |
| Disallow | /newback/ |
| Disallow | /newback2/ |
| Disallow | /online/online_worker.asp |
| Disallow | /online/online_jiwon.asp |
| Disallow | /online/online_email.asp |
| Disallow | /online/online_email2.asp |
| Disallow | /online/online_email3.asp |
| Disallow | /online/online_nomemberw.asp |
| Disallow | /online/online_simplenomemberw.asp |
| Disallow | /online/online_simplew.asp |
| Disallow | /online/online_emailw.asp |
| Disallow | /online/online_jiwonw.asp |
| Disallow | /online/online_simple.asp |
| Disallow | /online/qmailw.asp |
| Disallow | /t_oauth/ |
| Disallow | /njoin/ |
| Disallow | /newjoin/ |
| Disallow | /job/wod2.asp |
| Disallow | /job/mod.asp |
| Disallow | /job/metoday.asp |
| Disallow | /job/yozm.asp |
| Disallow | /newfreeboard/board/pin.asp |
| Disallow | /newfreeboard/board/form.asp |
| Disallow | /newfreeboard/board/copy_ok.asp |
| Disallow | /newfreeboard/board/del_ok.asp |
| Disallow | /batchupload/ |
| Disallow | /url_re.asp |
| Disallow | /script/jquery.js |
| Disallow | /corprank/script/jquery.js |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.worker.co.kr/sitemap.xml |
Warnings
- 2 invalid lines.