nasa.gov
robots.txt

Robots Exclusion Standard data for nasa.gov

Resource Scan

Scan Details

Site Domain nasa.gov
Base Domain nasa.gov
Scan Status Ok
Last Scan2024-10-20T17:30:52+00:00
Next Scan 2024-11-19T17:30:52+00:00

Last Scan

Scanned2024-10-20T17:30:52+00:00
URL https://nasa.gov/robots.txt
Redirect https://www.nasa.gov/robots.txt
Redirect Domain www.nasa.gov
Redirect Base nasa.gov
Domain IPs 192.0.66.108, 2a04:fa87:fffd::c000:426c
Redirect IPs 192.0.66.108, 2a04:fa87:fffd::c000:426c
Response IP 192.0.66.108
Found Yes
Hash 44d72c5ad1a94bb45f25caaba917c39b39b383af3a121b59a5a7e46fbe3ccb05
SimHash 691a8872c331

Groups

usasearch

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

*

Rule Path
Allow /