developer.okta.com
robots.txt
Robots Exclusion Standard data for developer.okta.com
Resource Scan
Scan Details
| Site Domain | developer.okta.com |
| Base Domain | okta.com |
| Scan Status | Ok |
| Last Scan | 2024-09-20T09:18:21+00:00 |
| Next Scan | 2024-10-20T09:18:21+00:00 |
Last Scan
| Scanned | 2024-09-20T09:18:21+00:00 |
| URL | https://developer.okta.com/robots.txt |
| Domain IPs | 2600:9000:2792:3400:d:5427:c40:93a1, 2600:9000:2792:400:d:5427:c40:93a1, 2600:9000:2792:5600:d:5427:c40:93a1, 2600:9000:2792:800:d:5427:c40:93a1, 2600:9000:2792:9600:d:5427:c40:93a1, 2600:9000:2792:b000:d:5427:c40:93a1, 2600:9000:2792:ce00:d:5427:c40:93a1, 2600:9000:2792:f000:d:5427:c40:93a1, 3.164.182.14, 3.164.182.27, 3.164.182.71, 3.164.182.81 |
| Response IP | 52.85.49.36 |
| Found | Yes |
| Hash | 165fe515407f8cf3b193416c94d0bcc99b282bc3cef39de226bb1489ed6bffc2 |
| SimHash | 6b51dc50cd33 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /docs/guides/migrate-to-oie/ |
| Disallow | /docs/guides/manage-orgs-okta-aerial/ |
| Disallow | /docs/release-notes/2016 |
| Disallow | /docs/release-notes/2017 |
| Disallow | /docs/release-notes/2018 |
| Disallow | /docs/release-notes/2019 |
| Disallow | /docs/release-notes/2020 |
| Disallow | /docs/release-notes/2021 |
| Disallow | /docs/release-notes/2021-okta-identity-engine |
| Disallow | /docs/release-notes/2022 |
| Disallow | /docs/release-notes/2022-okta-identity-engine |
| Disallow | /docs/reference/api/archive-myaccount/ |
| Disallow | /docs/reference/api/myaccount-migration/ |
| Disallow | /docs/reference/csi-delauth-hook/ |
| Disallow | /docs/guides/configure-amr-claims-mapping/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https:/developer.okta.com/sitemap_index.xml |
Warnings
- `host` is not a known field.