ca.gov
robots.txt

Robots Exclusion Standard data for ca.gov

Resource Scan

Scan Details

Site Domain ca.gov
Base Domain ca.gov
Scan Status Ok
Last Scan2024-05-04T13:53:44+00:00
Next Scan 2024-06-03T13:53:44+00:00

Last Scan

Scanned2024-05-04T13:53:44+00:00
URL https://ca.gov/robots.txt
Redirect https://www.ca.gov/robots.txt
Redirect Domain www.ca.gov
Redirect Base ca.gov
Domain IPs 13.87.221.220
Redirect IPs 2600:1413:b000:6::17d5:2bcd, 2600:1413:b000:6::17d5:2be0, 96.17.96.16, 96.17.96.19
Response IP 23.32.29.90
Found Yes
Hash 47d1d24dd620b57a34fc4be12ab282f2a533aa53166dce56b9cc08b0409390c9
SimHash e00cc2528392

Groups

*

Rule Path
Allow /
Disallow /ads.txt$
Disallow /app-ads.txt$

Other Records

Field Value
sitemap https://www.ca.gov/sitemap.xml