vt.edu
robots.txt
Robots Exclusion Standard data for vt.edu
Resource Scan
Scan Details
| Site Domain | vt.edu |
| Base Domain | vt.edu |
| Scan Status | Ok |
| Last Scan | 2024-11-08T23:02:10+00:00 |
| Next Scan | 2024-12-08T23:02:10+00:00 |
Last Scan
| Scanned | 2024-11-08T23:02:10+00:00 |
| URL | https://vt.edu/robots.txt |
| Redirect | https://www.vt.edu/content/dam/vt_edu/robots.txt |
| Redirect Domain | www.vt.edu |
| Redirect Base | vt.edu |
| Domain IPs | 198.82.215.14, 2607:b400:92:26:0:97:1e7:3947 |
| Redirect IPs | 13.33.88.20, 13.33.88.42, 13.33.88.54, 13.33.88.75, 2600:9000:223b:1200:11:6561:2d80:93a1, 2600:9000:223b:2000:11:6561:2d80:93a1, 2600:9000:223b:3000:11:6561:2d80:93a1, 2600:9000:223b:3c00:11:6561:2d80:93a1, 2600:9000:223b:7400:11:6561:2d80:93a1, 2600:9000:223b:b400:11:6561:2d80:93a1, 2600:9000:223b:bc00:11:6561:2d80:93a1, 2600:9000:223b:d800:11:6561:2d80:93a1 |
| Response IP | 13.33.88.20 |
| Found | Yes |
| Hash | 6698fd11aa14e125b3199d512037a0d0167f1e6376006927eca25c7582436aab |
| SimHash | 6940ec22e7ed |
Groups
bingbot
| Rule | Path |
|---|---|
| Disallow | /test/* |
| Disallow | /test.html |
| Disallow | */admissions/*do-not-crawl |
| Disallow | /admissions/*_old |
| Disallow | /admissions/*_ref_ |
| Disallow | /admissions/*_remove_ |
| Disallow | /admissions/info |
| Disallow | /admissions/error |
| Disallow | /admissions/undergraduate/redirect* |
| Disallow | /admissions/undergraduate/sandbox* |
| Disallow | /admissions/undergraduate/visit/*update.html |
| Disallow | /admissions/undergraduate/visit/*cancel.html |
| Disallow | /admissions/undergraduate/visit/*reprint.html |
| Disallow | /admissions/undergraduate/visit/hokiefocus/closed.html |
| Disallow | /admissions/undergraduate/visit/hokiepreview/closed.html |
| Disallow | /admissions/undergraduate/visit/gateway/closed.html |
| Disallow | /admissions/undergraduate/visit/fallvisitation/closed.html |
| Disallow | /admissions/undergraduate/visit/openhouse/closed.html |
| Disallow | /admissions/undergraduate/apply/freshmen/secondchoicemajor-us.html |
| Disallow | /innovationcampus/Ref.html |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
semrushbot
| Rule | Path |
|---|---|
| Disallow | /test/* |
| Disallow | /test.html |
| Disallow | */admissions/*do-not-crawl |
| Disallow | /admissions/*_old |
| Disallow | /admissions/*_ref_ |
| Disallow | /admissions/*_remove_ |
| Disallow | /admissions/info |
| Disallow | /admissions/error |
| Disallow | /admissions/undergraduate/redirect* |
| Disallow | /admissions/undergraduate/sandbox* |
| Disallow | /admissions/undergraduate/visit/*update.html |
| Disallow | /admissions/undergraduate/visit/*cancel.html |
| Disallow | /admissions/undergraduate/visit/*reprint.html |
| Disallow | /admissions/undergraduate/visit/hokiefocus/closed.html |
| Disallow | /admissions/undergraduate/visit/hokiepreview/closed.html |
| Disallow | /admissions/undergraduate/visit/gateway/closed.html |
| Disallow | /admissions/undergraduate/visit/fallvisitation/closed.html |
| Disallow | /admissions/undergraduate/visit/openhouse/closed.html |
| Disallow | /admissions/undergraduate/apply/freshmen/secondchoicemajor-us.html |
| Disallow | /innovationcampus/Ref.html |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
*
| Rule | Path |
|---|---|
| Disallow | /test/* |
| Disallow | /test.html |
| Disallow | */admissions/*do-not-crawl |
| Disallow | /admissions/*_old |
| Disallow | /admissions/*_ref_ |
| Disallow | /admissions/*_remove_ |
| Disallow | /admissions/info |
| Disallow | /admissions/error |
| Disallow | /admissions/undergraduate/redirect* |
| Disallow | /admissions/undergraduate/sandbox* |
| Disallow | /admissions/undergraduate/visit/*update.html |
| Disallow | /admissions/undergraduate/visit/*cancel.html |
| Disallow | /admissions/undergraduate/visit/*reprint.html |
| Disallow | /admissions/undergraduate/visit/hokiefocus/closed.html |
| Disallow | /admissions/undergraduate/visit/hokiepreview/closed.html |
| Disallow | /admissions/undergraduate/visit/gateway/closed.html |
| Disallow | /admissions/undergraduate/visit/fallvisitation/closed.html |
| Disallow | /admissions/undergraduate/visit/openhouse/closed.html |
| Disallow | /admissions/undergraduate/apply/freshmen/secondchoicemajor-us.html |
| Disallow | /innovationcampus/Ref.html |