pg-p.ctme.caltech.edu
robots.txt
Robots Exclusion Standard data for pg-p.ctme.caltech.edu
Resource Scan
Scan Details
| Site Domain | pg-p.ctme.caltech.edu |
| Base Domain | caltech.edu |
| Scan Status | Ok |
| Last Scan | 2025-03-03T21:26:41+00:00 |
| Next Scan | 2025-04-02T21:26:41+00:00 |
Last Scan
| Scanned | 2025-03-03T21:26:41+00:00 |
| URL | https://pg-p.ctme.caltech.edu/robots.txt |
| Domain IPs | 2600:9000:28c2:3c00:7:24aa:4fc0:93a1, 2600:9000:28c2:3e00:7:24aa:4fc0:93a1, 2600:9000:28c2:4c00:7:24aa:4fc0:93a1, 2600:9000:28c2:6400:7:24aa:4fc0:93a1, 2600:9000:28c2:8200:7:24aa:4fc0:93a1, 2600:9000:28c2:b000:7:24aa:4fc0:93a1, 2600:9000:28c2:c800:7:24aa:4fc0:93a1, 2600:9000:28c2:f200:7:24aa:4fc0:93a1, 3.171.198.109, 3.171.198.42, 3.171.198.57, 3.171.198.82 |
| Response IP | 3.171.198.57 |
| Found | Yes |
| Hash | 24391f4a445764f6b4fede1281783f77f06e9d3a09df523688a34efa07bcce32 |
| SimHash | fa104f06ebb2 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
*
| Rule | Path |
|---|---|
| Allow | /wp-content/uploads/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-login.php |
| Disallow | /wp-trackback.php |
| Disallow | /xmlrpc.php |
| Disallow | /*? |
| Disallow | /feed |
Other Records
| Field | Value |
|---|---|
| sitemap | https://pg-p.ctme.caltech.edu/sitemaps/caltech_sitemap_production.xml |
| sitemap | https://pg-p.ctme.caltech.edu/post-sitemap.xml |
| sitemap | https://pg-p.ctme.caltech.edu/page-sitemap.xml |
| sitemap | https://pg-p.ctme.caltech.edu/category-sitemap.xml |