mbc.ca.gov
robots.txt

Robots Exclusion Standard data for mbc.ca.gov

Resource Scan

Scan Details

Site Domain mbc.ca.gov
Base Domain ca.gov
Scan Status Ok
Last Scan2024-09-03T10:11:07+00:00
Next Scan 2024-10-03T10:11:07+00:00

Last Scan

Scanned2024-09-03T10:11:07+00:00
URL https://mbc.ca.gov/robots.txt
Domain IPs 159.145.8.63
Response IP 159.145.8.63
Found Yes
Hash d0b70ecd70c17f6a5174c07dde154d079f85949ecbc2efe24ab7f6725e574bc3
SimHash 111419754553

Groups

siteimprovebot

Rule Path
Allow /

siteimprovebot-crawler

Rule Path
Allow /

*

Rule Path
Disallow /bin/
Disallow /App_Data/
Disallow /App_Start/
Disallow /js/
Disallow /css/
Disallow /Global.asax
Disallow /Web.config
Allow /

Other Records

Field Value
sitemap https://www.mbc.ca.gov/sitemap.xml