www.csc.kth.se
robots.txt
Robots Exclusion Standard data for www.csc.kth.se
Resource Scan
Scan Details
| Site Domain | www.csc.kth.se |
| Base Domain | kth.se |
| Scan Status | Ok |
| Last Scan | 2025-12-20T16:46:53+00:00 |
| Next Scan | 2026-01-03T16:46:53+00:00 |
Last Scan
| Scanned | 2025-12-20T16:46:53+00:00 |
| URL | https://www.csc.kth.se/robots.txt |
| Domain IPs | 130.237.28.57, 2001:6b0:1:11c2::82ed:1c39 |
| Response IP | 130.237.28.57 |
| Found | Yes |
| Hash | 1140fb45418ef27401b9d06ad74200787b55de1364996c32103fb0b83d7a1ab5 |
| SimHash | b551899745b5 |
Groups
*
| Product | Comment |
|---|---|
| * | directed to all spiders, not just Scooter |
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /internt/ |
| Disallow | /larare/internt/ |
| Disallow | /test/ |
| Disallow | /RCS/ |
| Disallow | /kth/ |
| Disallow | /bin/ |
| Disallow | /forskning/cb/publist/divapublications.php |
| Disallow | /bokning |
| Disallow | /bokning-matematik |
| Disallow | /bokning-student |
| Disallow | /~ann/exjobb/ |