Log In Sign Up

www.sei.cmu.edu
robots.txt

Robots Exclusion Standard data for www.sei.cmu.edu

Resource Scan

Scan Details

Site Domain www.sei.cmu.edu
Base Domain cmu.edu
Scan Status Ok
Last Scan2024-09-01T16:27:55+00:00
Next Scan 2024-10-01T16:27:55+00:00

Last Scan

Scanned2024-09-01T16:27:55+00:00
URL https://www.sei.cmu.edu/robots.txt
Domain IPs 147.72.252.240
Response IP 147.72.252.240
Found Yes
Hash 14995101fd7d8b5509e29c984dc3d9ccb6af447dc186b2085b862a541e916f0b
SimHash 25425a53ce26

Groups

*

Rule Path
Allow /

webzip

Rule Path
Disallow /

*

Rule Path
Disallow /loader*

*

Rule Path
Disallow /webadmin/*

*

Rule Path
Allow /smartgrid/start/downloads/*

*

Rule Path
Disallow /test/*

*

Rule Path
Disallow /sample-webinar*
Back to top

Other Records

Field Value
sitemap http://www.sei.cmu.edu/sitemap.xml
Back to top

Comments

  • robots.txt for http://www.sei.cmu.edu/
  • Changed 16 Jul 2024 by cch
  • User-agent: *
  • Crawl-delay: 1
  • Changed 16 Jul 2024 by cch
  • User-agent: *
  • Request-rate: 30/1m
  • Changed 20 Nov 2015 by cch
  • User-agent: *
  • Disallow: /events/*
  • last updated: 20 Nov 2015
  • For information, contact: webmaster@sei.cmu.edu
Back to top