micromasters.mit.edu
robots.txt
Robots Exclusion Standard data for micromasters.mit.edu
Resource Scan
Scan Details
| Site Domain | micromasters.mit.edu |
| Base Domain | mit.edu |
| Scan Status | Ok |
| Last Scan | 2025-07-01T19:11:49+00:00 |
| Next Scan | 2025-07-31T19:11:49+00:00 |
Last Scan
| Scanned | 2025-07-01T19:11:49+00:00 |
| URL | https://micromasters.mit.edu/robots.txt |
| Domain IPs | 15.197.246.237, 3.33.193.101, 52.223.46.195, 99.83.183.127 |
| Response IP | 52.223.46.195 |
| Found | Yes |
| Hash | 5733d5b42b45adc7b257ff4f977e75c392851646ef320856721b81f023fe1c9c |
| SimHash | 4905648eee13 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /dashboard/ |
| Disallow | /learner/ |
| Disallow | /learners/ |
| Disallow | /cms/ |
| Disallow | /profile/ |
| Disallow | /financial_aid/ |
| Disallow | /admin/ |
| Disallow | /logout/ |
| Disallow | /pearson/ |
| Disallow | /api/ |
| Disallow | /certificate/ |
| Disallow | /records/ |
| Disallow | /letter/ |