datastream.org
robots.txt
Robots Exclusion Standard data for datastream.org
Resource Scan
Scan Details
| Site Domain | datastream.org |
| Base Domain | datastream.org |
| Scan Status | Ok |
| Last Scan | 2024-05-19T04:40:23+00:00 |
| Next Scan | 2024-06-02T04:40:23+00:00 |
Last Scan
| Scanned | 2024-05-19T04:40:23+00:00 |
| URL | https://datastream.org/robots.txt |
| Domain IPs | 2600:9000:260f:1a00:7:c1c9:3b40:93a1, 2600:9000:260f:2000:7:c1c9:3b40:93a1, 2600:9000:260f:6000:7:c1c9:3b40:93a1, 2600:9000:260f:9e00:7:c1c9:3b40:93a1, 2600:9000:260f:c200:7:c1c9:3b40:93a1, 2600:9000:260f:c400:7:c1c9:3b40:93a1, 2600:9000:260f:e000:7:c1c9:3b40:93a1, 2600:9000:260f:ee00:7:c1c9:3b40:93a1, 3.163.24.101, 3.163.24.127, 3.163.24.40, 3.163.24.42 |
| Response IP | 108.157.52.17 |
| Found | Yes |
| Hash | bf414403a0db1f9da7ba11de094c349866172ec67cecd00edeb75e17c983fb81 |
| SimHash | 681da8444592 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /api/ |
| Disallow | /en-ca/team* |
| Disallow | /fr-ca/notre-equipe* |
*
| Rule | Path |
|---|---|
| Disallow | /en/* |
| Disallow | /fr/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://datastream.org/sitemap.xml |