Log In Sign Up

its.unc.edu
robots.txt

Robots Exclusion Standard data for its.unc.edu

Resource Scan

Scan Details

Site Domain its.unc.edu
Base Domain unc.edu
Scan Status Ok
Last Scan2024-08-26T11:46:52+00:00
Next Scan 2024-09-25T11:46:52+00:00

Last Scan

Scanned2024-08-26T11:46:52+00:00
URL https://its.unc.edu/robots.txt
Domain IPs 23.185.0.4, 2620:12a:8000::4, 2620:12a:8001::4
Response IP 23.185.0.4
Found Yes
Hash 442f75fc7469876c393daf715fa0d59c847c84e1c447e84e394fc2867dee6f84
SimHash 22bca894d2a5

Groups

timpibot
seekportbot

Rule Path
Disallow /

siteimprovebot-crawler

Rule Path
Disallow */wp-login.php
Disallow */wp-json/*
Disallow */wp-admin/*
Disallow */?attachment_id=*
Disallow */?s=*
Disallow */?taxonomy=nav_menu*
Disallow */?eventDisplay=past*
Disallow */?eventDisplay=photo*
Disallow */?post_type=tribe_events&eventDisplay=day*
Disallow */?post_type=tribe_events&eventDisplay=week*
Disallow */?post_type=tribe_events&eventDisplay=month*
Disallow */?tribe-bar-date=*
Disallow *%26eventDisplay%3Dpast*
Disallow *%26eventDisplay%3Dphoto*
Disallow *%26tribe-bar-date%3D*
Disallow */2009/*
Disallow */2010/*
Disallow */2011/*
Disallow */2012/*
Disallow */2013/*
Disallow */2014/*
Disallow */2015/*
Disallow */2016/*
Disallow */2017/*
Disallow */2018/*
Disallow */2019/*
Disallow */2020/*
Disallow */2021/*
Disallow */2022/*
Disallow */2023/*
Disallow */author/*
Disallow */category/*
Disallow */events/*
Disallow */organizer/*
Disallow */scripts/webalert.js?__ver=*
Disallow */tag/*
Disallow */venue/*

Other Records

Field Value
crawl-delay 3

*

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://its.unc.edu/sitemap_index.xml
Back to top

Comments

  • Blocking these bots
  • Site Improve blocking
  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK
Back to top