en.snu.ac.kr
robots.txt

Robots Exclusion Standard data for en.snu.ac.kr

Archived Snapshots

Resource Scan

Scan Details

Site Domain	en.snu.ac.kr
Base Domain	snu.ac.kr
Scan Status	Ok
Last Scan	2025-08-27T02:09:45+00:00
Next Scan	2025-09-26T02:09:45+00:00

Last Scan

Scanned	2025-08-27T02:09:45+00:00
URL	https://en.snu.ac.kr/robots.txt
Domain IPs	147.46.10.129
Response IP	147.46.10.129
Found	Yes
Hash	c558e8e358a44efd1ec9a9103a3b0dc7259848b1c82c762c3e9d7cd707deae07
SimHash	d3565150670b

Groups

femtosearchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

turnitinbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

amazonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexcalendar

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexmobilebot

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

awariorssbot
awariosmartbot

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bomborabot

Rule	Path
Disallow	/

Rule

Path

Disallow

buck

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

seekportbot

Rule	Path
Disallow	/

Rule

Path

Disallow

turnitinbot

Rule	Path
Disallow	/

Rule

Path

Disallow

paqlebot

Rule	Path
Disallow	/

Rule

Path

Disallow

grapeshot

Rule	Path
Disallow	/

Rule

Path

Disallow

mail.ru_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

geedobot

Rule	Path
Disallow	/

Rule

Path

Disallow

femtosearchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

serpstatbot

Rule	Path
Disallow	/

Rule

Path

Disallow

amazonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

criteobot/0.1

Rule	Path
Disallow	/

Rule

Path

Disallow

dataforseobot

Rule	Path
Disallow	/

Rule

Path

Disallow

openindexspider

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider
yisouspider
petalbot
bytespider
sogou web spider
sogou inst spider

Rule	Path
Disallow	/

Rule

Path

Disallow

proximic

Rule	Path
Disallow	/

Rule

Path

Disallow

ias_crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

awariorssbot
awariosmartbot

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bingbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	30

Field

Value

crawl-delay

amazonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Allow	/

Rule

Path

Allow

Warnings

1 invalid line.

en.snu.ac.krrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

femtosearchbot

ahrefsbot

turnitinbot

blexbot

mj12bot

petalbot

amazonbot

applebot

yandexcalendar

yandexmobilebot

dotbot

awariorssbotawariosmartbot

baiduspider

semrushbot

petalbot

bomborabot

buck

blexbot

seekportbot

turnitinbot

paqlebot

grapeshot

mail.ru_bot

geedobot

femtosearchbot

serpstatbot

amazonbot

criteobot/0.1

dataforseobot

openindexspider

gptbot

baiduspideryisouspiderpetalbotbytespidersogou web spidersogou inst spider

proximic

ias_crawler

awariorssbotawariosmartbot

claudebot

ccbot

bingbot

Other Records

amazonbot

claudebot

*

Warnings

en.snu.ac.kr
robots.txt

awariorssbot
awariosmartbot

baiduspider
yisouspider
petalbot
bytespider
sogou web spider
sogou inst spider

awariorssbot
awariosmartbot