summer.jhu.edu
robots.txt

Robots Exclusion Standard data for summer.jhu.edu

Resource Scan

Scan Details

Site Domain summer.jhu.edu
Base Domain jhu.edu
Scan Status Ok
Last Scan2025-10-16T15:19:36+00:00
Next Scan 2025-11-15T15:19:36+00:00

Last Scan

Scanned2025-10-16T15:19:36+00:00
URL https://summer.jhu.edu/robots.txt
Domain IPs 104.18.35.124, 172.64.152.132, 2606:4700:440d::ac40:9884, 2a06:98c1:3100::6812:237c
Response IP 172.64.152.132
Found Yes
Hash beb6452d16fd157523b93011385214adb2b5aebf16b011e9fbf5371c28a70591
SimHash 61105c408292

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /fastspot-test-pages/course-listing/
Disallow /fastspot-test-pages/flexible-content/
Disallow /fastspot-test-pages/flexible-content-video/
Disallow /fastspot-test-pages/flexible-content-image/
Disallow /fastspot-test-pages/forms/
Disallow /fastspot-test-pages/
Disallow /fastspot-test-pages/full-width-components/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://summer.jhu.edu/sitemap_index.xml

Comments

  • www.robotstxt.org/