its.unc.edu
robots.txt
Robots Exclusion Standard data for its.unc.edu
Resource Scan
Scan Details
| Site Domain | its.unc.edu |
| Base Domain | unc.edu |
| Scan Status | Ok |
| Last Scan | 2024-05-28T11:45:43+00:00 |
| Next Scan | 2024-06-27T11:45:43+00:00 |
Last Scan
| Scanned | 2024-05-28T11:45:43+00:00 |
| URL | https://its.unc.edu/robots.txt |
| Domain IPs | 23.185.0.4, 2620:12a:8000::4, 2620:12a:8001::4 |
| Response IP | 23.185.0.4 |
| Found | Yes |
| Hash | 6da6910d432092c7b4389bdebdce5c551efc68214ad2c599ce72d03d9b6de8ce |
| SimHash | 23bc2804d2b5 |
Groups
siteimprovebot-crawler
| Rule | Path |
|---|---|
| Disallow | */wp-login.php |
| Disallow | */wp-json/* |
| Disallow | */wp-admin/* |
| Disallow | */?attachment_id=* |
| Disallow | */?s=* |
| Disallow | */?taxonomy=nav_menu* |
| Disallow | */?eventDisplay=past* |
| Disallow | */?eventDisplay=photo* |
| Disallow | */?post_type=tribe_events&eventDisplay=day* |
| Disallow | */?post_type=tribe_events&eventDisplay=week* |
| Disallow | */?post_type=tribe_events&eventDisplay=month* |
| Disallow | */?tribe-bar-date=* |
| Disallow | *%26eventDisplay%3Dpast* |
| Disallow | *%26eventDisplay%3Dphoto* |
| Disallow | *%26tribe-bar-date%3D* |
| Disallow | */2009/* |
| Disallow | */2010/* |
| Disallow | */2011/* |
| Disallow | */2012/* |
| Disallow | */2013/* |
| Disallow | */2014/* |
| Disallow | */2015/* |
| Disallow | */2016/* |
| Disallow | */2017/* |
| Disallow | */2018/* |
| Disallow | */2019/* |
| Disallow | */2020/* |
| Disallow | */2021/* |
| Disallow | */2022/* |
| Disallow | */2023/* |
| Disallow | */author/* |
| Disallow | */category/* |
| Disallow | */events/* |
| Disallow | */organizer/* |
| Disallow | */scripts/webalert.js?__ver=* |
| Disallow | */tag/* |
| Disallow | */venue/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
*
| Rule | Path |
|---|---|
| Disallow |
Other Records
| Field | Value |
|---|---|
| sitemap | https://its.unc.edu/sitemap_index.xml |
Comments