nebius.com
robots.txt

Robots Exclusion Standard data for nebius.com

Resource Scan

Scan Details

Site Domain nebius.com
Base Domain nebius.com
Scan Status Ok
Last Scan2024-08-25T00:11:04+00:00
Next Scan 2024-09-24T00:11:04+00:00

Last Scan

Scanned2024-08-25T00:11:04+00:00
URL https://nebius.com/robots.txt
Redirect https://nebius.ai/robots.txt
Redirect Domain nebius.ai
Redirect Base nebius.ai
Domain IPs 195.242.19.154, 2a13:5947:2f::254
Redirect IPs 195.242.19.154, 2a13:5947:2f::254
Response IP 195.242.19.154
Found Yes
Hash 2cb09248dd1b72bf8b1b677420d58f72b419fe088658ea936d96a7b8260e03d0
SimHash 4818ae4275b2

Groups

*

Rule Path
Allow /
Disallow /docs/application-load-balancer
Disallow /docs/network-load-balancer
Disallow /docs/api-design-guide
Disallow /docs/storage/concepts/policy
Disallow /docs/storage/concepts/acl
Disallow /docs/storage/concepts/cors
Disallow /docs/storage/concepts/hosting
Disallow /docs/storage/concepts/s3-select-language
Disallow /docs/storage/concepts/object-lock
Disallow /docs/storage/concepts/presigned-post-forms
Disallow /docs/storage/concepts/storage-class
Disallow /docs/compute/operations/serial-console/
Disallow /docs/compute/concepts/software-accelerated-network
Disallow /docs/compute/concepts/placement-groups
Disallow /docs/compute/concepts/disk-placement-group
Disallow /docs/compute/concepts/image
Disallow /docs/compute/concepts/instance-groups/
Disallow /docs/vpc/operations/connections-view
Disallow /docs/iam/concepts/authorization/api-key
Disallow /docs/iam/operations/roles/grant
Disallow /docs/resource-manager/
Disallow /docs/organization/
Disallow /docs/managed-kubernetes/concepts/network-policy
Disallow /docs/container-registry/operations/registry/registry-access
Disallow /docs/container-registry/concepts/lifecycle-policy
Disallow /docs/managed-redis/concepts/sharding
Disallow /*?
Disallow /*%3D
Disallow /*index$
Disallow /*_includes
Disallow /*_tutorials
Disallow /*_qa
Disallow /feed.*
Disallow /_*

Other Records

Field Value
sitemap https://nebius.ai/sitemap_index.xml

Warnings

  • `host` is not a known field.