berkeleyrep.org
robots.txt
Robots Exclusion Standard data for berkeleyrep.org
Resource Scan
Scan Details
| Site Domain | berkeleyrep.org |
| Base Domain | berkeleyrep.org |
| Scan Status | Ok |
| Last Scan | 2026-01-27T09:23:35+00:00 |
| Next Scan | 2026-02-26T09:23:35+00:00 |
Last Scan
| Scanned | 2026-01-27T09:23:35+00:00 |
| URL | https://berkeleyrep.org/robots.txt |
| Redirect | https://www.berkeleyrep.org/robots.txt |
| Redirect Domain | www.berkeleyrep.org |
| Redirect Base | berkeleyrep.org |
| Domain IPs | 104.26.14.65, 104.26.15.65, 172.67.70.234, 2606:4700:20::681a:e41, 2606:4700:20::681a:f41, 2606:4700:20::ac43:46ea |
| Redirect IPs | 104.26.14.65, 104.26.15.65, 172.67.70.234, 2606:4700:20::681a:e41, 2606:4700:20::681a:f41, 2606:4700:20::ac43:46ea |
| Response IP | 104.26.15.65 |
| Found | Yes |
| Hash | 029192110d902f2905932c24b506774aecd3ec278fb710cf7e4c48afaba19956 |
| SimHash | 430cdc554790 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /dashboard/ |
| Disallow | /my/ |
| Disallow | /*?* |
| Allow | /*?page=* |
| Allow | /dist*?* |
| Allow | /platform*?* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.berkeleyrep.org/sitemap |
Comments