Log In Sign Up

cs.unc.edu
robots.txt

Robots Exclusion Standard data for cs.unc.edu

Resource Scan

Scan Details

Site Domain cs.unc.edu
Base Domain unc.edu
Scan Status Ok
Last Scan2024-05-30T20:05:31+00:00
Next Scan 2024-06-29T20:05:31+00:00

Last Scan

Scanned2024-05-30T20:05:31+00:00
URL https://cs.unc.edu/robots.txt
Domain IPs 23.185.0.4, 2620:12a:8000::4, 2620:12a:8001::4
Response IP 23.185.0.4
Found Yes
Hash 4e9ff611efafdee69b032a1f374446dc3153fb9c1065b12f0e2ba57ae11a2005
SimHash 63bc2814d2f1

Groups

siteimprovebot-crawler

Rule Path
Disallow */wp-login.php
Disallow */wp-json/*
Disallow */wp-admin/*
Disallow */?attachment_id=*
Disallow */?s=*
Disallow */?taxonomy=nav_menu*
Disallow */?eventDisplay=past*
Disallow */?eventDisplay=photo*
Disallow */?post_type=tribe_events&eventDisplay=day*
Disallow */?post_type=tribe_events&eventDisplay=week*
Disallow */?post_type=tribe_events&eventDisplay=month*
Disallow */?tribe-bar-date=*
Disallow *%26eventDisplay%3Dpast*
Disallow *%26eventDisplay%3Dphoto*
Disallow *%26tribe-bar-date%3D*
Disallow */2009/*
Disallow */2010/*
Disallow */2011/*
Disallow */2012/*
Disallow */2013/*
Disallow */2014/*
Disallow */2015/*
Disallow */2016/*
Disallow */2017/*
Disallow */2018/*
Disallow */2019/*
Disallow */2020/*
Disallow */2021/*
Disallow */2022/*
Disallow */2023/*
Disallow */author/*
Disallow */category/*
Disallow */events/*
Disallow */organizer/*
Disallow */scripts/webalert.js?__ver=*
Disallow */tag/*
Disallow */venue/*

Other Records

Field Value
crawl-delay 3

*

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://cs.unc.edu/sitemap_index.xml
Back to top

Comments

  • Site Improve blocking
  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK
Back to top