Log In Sign Up

utsouthwestern.edu
robots.txt

Robots Exclusion Standard data for utsouthwestern.edu

Resource Scan

Scan Details

Site Domain utsouthwestern.edu
Base Domain utsouthwestern.edu
Scan Status Ok
Last Scan2024-10-25T16:26:40+00:00
Next Scan 2024-11-24T16:26:40+00:00

Last Scan

Scanned2024-10-25T16:26:40+00:00
URL https://utsouthwestern.edu/robots.txt
Redirect https://www.utsouthwestern.edu/robots.txt
Redirect Domain www.utsouthwestern.edu
Redirect Base utsouthwestern.edu
Domain IPs 199.242.239.105
Redirect IPs 199.242.239.105
Response IP 199.242.239.105
Found Yes
Hash 045b455d5671a435472888e2dc9da6741ae125a674ffb6e35ae96200f9056b03
SimHash 3844c9363fc2

Groups

*

Rule Path
Disallow /patientcare/globalsearch/
Disallow /utsw-ext-templating/org/jsp/
Disallow /wp-login.php
Disallow /connectors/system/settings.php
Disallow /EWS/Exchange.asmx
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content
Disallow /UTSW/CMA/CMA_applications/UTSWPageStencils/human_resource/
Disallow /about-us/administrative-offices/information-resources/academic-information-systems/systems/core-lims.html
Disallow /_googlesearch
Disallow /_googlesearch?q=&site=
Disallow /_channelcache
Disallow /_collectioncache
Disallow /_assetcache
Disallow /_websitepagecache
Disallow /shared/
Disallow /nfis/getGraduateSchoolProgramsByProgramNameFilter.jsonp
Disallow /nfis/getClinicalKeywordsByKeywordNameFilter.jsonp
Disallow /nfis/getDepartmentsByDepartmentNameFilter.jsonp
Disallow /nfis/getProfileFacultiesByNameFilter.jsonp
Disallow /newsroom/media-relations/st-paul-demolition.html
Disallow /edumedia/edufiles/newsroom/demolition-map.pdf
Disallow /edumedia/edufiles/about_us/admin_offices/Purchasing/
Disallow /sugar
Disallow /legal/open-records-request.html
Disallow /sites/cuh-email
Disallow /sites/html-email
Disallow /sites/campus-updates
Disallow /open-records-request-detail.html?requestId=
Disallow /openrecordsdoc/
Disallow /about-us/administrative-offices/purchasing/transparency.html
Disallow /newsroom/in-the-news/year
Disallow /sites/campus-updates
Disallow /test
Disallow /edu-guide
Disallow /resources
Disallow /lp/
Disallow /alerts/

googlebot

Rule Path
Disallow /sites/campus-news

bingbot

Rule Path
Disallow /sites/campus-news

slurp

Rule Path
Disallow /sites/campus-news

duckduckbot

Rule Path
Disallow /sites/campus-news

baiduspider

Rule Path
Disallow /sites/campus-news

yandexbot

Rule Path
Disallow /sites/campus-news

vegi bot (we follow your robots.txt settings before crawling, you can slow down the bot by change the crawl-delay parameter in the settings.if you have an enquiry, please email to: abuse-report@terrykyleseoagency.com)

Rule Path
Disallow /

vegi bot

Rule Path
Disallow /

libwww

Rule Path
Disallow /

http::lite

Rule Path
Disallow /

phpcrawl

Rule Path
Disallow /

wep search

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://www.utsouthwestern.edu/sitemap.xml
Back to top

Comments

  • SiteMap.xml can be found at this URL
  • Googlebot seeing 500 errors for Patient Care global site search
  • Spammers are linking to hrmsphoto.jsp
  • Googlebot seeing badly-formed URLs from some HR pages
  • Dont allow crawl of the following web page
  • Dont allow jsonp calls from this application
  • Dont allow crawl of demolish
  • Do not crawl purchasing documents
  • Dont crawl any of the sugar content
  • Do not crawl email pages
  • Do not crawl for the new Campus Updates site and any of its pages
  • Do not crawl open record requests
  • Do not crawl purchasing transparency
  • Do not allow indiviual news items is search
  • Do not crawl email pages
  • Do not crawl test folder
  • Disallow: /fonts
  • Disallow: /css
  • Disallow: /js
  • Disallow: /img
  • Do not allow campaign landing pages
  • Dont allow crawl in-pursuit
  • Disallow: /research/in-pursuit/
  • Do not crawl /sites/campus-news (added on 09/05/2018)
Back to top