ca.gov
robots.txt

Robots Exclusion Standard data for ca.gov

Resource Scan

Scan Details

Site Domain ca.gov
Base Domain ca.gov
Scan Status Ok
Last Scan2024-10-31T18:02:14+00:00
Next Scan 2024-11-30T18:02:14+00:00

Last Scan

Scanned2024-10-31T18:02:14+00:00
URL https://ca.gov/robots.txt
Redirect https://www.ca.gov/robots.txt
Redirect Domain www.ca.gov
Redirect Base ca.gov
Domain IPs 13.87.221.220
Redirect IPs 23.209.46.147, 23.209.46.160, 2600:1413:b000:6::17d5:2bc7, 2600:1413:b000:6::17d5:2bcf
Response IP 23.44.5.49
Found Yes
Hash 98d440e9c0dfa35847203c2b43a271b29759332c704a677ae0e1ce8496b731f3
SimHash e004d050c713

Groups

*

Rule Path
Allow /
Disallow /ads.txt$

Other Records

Field Value
sitemap https://www.ca.gov/sitemap.xml