tuck.dartmouth.edu
robots.txt

Robots Exclusion Standard data for tuck.dartmouth.edu

Resource Scan

Scan Details

Site Domain tuck.dartmouth.edu
Base Domain dartmouth.edu
Scan Status Ok
Last Scan2025-07-25T10:02:51+00:00
Next Scan 2025-08-24T10:02:51+00:00

Last Scan

Scanned2025-07-25T10:02:51+00:00
URL https://tuck.dartmouth.edu/robots.txt
Domain IPs 129.170.226.227, 129.170.226.9
Response IP 129.170.226.9
Found Yes
Hash f4de68d97465c7b56a0ad43957d71949e2e884806b89b0992059201c3dc1432e
SimHash 4f838981c210

Groups

*

Rule Path
Disallow /home-2017/
Disallow /tuck/
Disallow /mba/elective-curriculum/elective-courses-1
Disallow /content/
Disallow /cbgs/
Disallow /cgbg/
Disallow /recruitingOLD/
Disallow /2014/
Disallow /2015/
Disallow /2016/
Disallow /2017/
Disallow /2018/
Disallow /2019/
Disallow /2020/
Disallow /2021/
Disallow /2022/
Disallow /2023/
Disallow /2024/
Disallow /2025/
Disallow /2026/
Disallow /2027/
Disallow /2028/
Disallow /2029/
Disallow /2030/
Disallow /tuck-emails/
Disallow /payflow/
Disallow /buchanan-hall-room-reservations
Disallow /tuck-minority-programs
Disallow /about/pilot-programs/tech

Other Records

Field Value
crawl-delay 3

bytespider

Rule Path
Disallow /

Comments

  • Set a crawl delay for all bots
  • Disallow Bytespider completely