service.ct.gov
robots.txt
Robots Exclusion Standard data for service.ct.gov
Resource Scan
Scan Details
| Site Domain | service.ct.gov |
| Base Domain | ct.gov |
| Scan Status | Ok |
| Last Scan | 2024-09-24T19:35:40+00:00 |
| Next Scan | 2024-10-08T19:35:40+00:00 |
Last Scan
| Scanned | 2024-09-24T19:35:40+00:00 |
| URL | https://service.ct.gov/robots.txt |
| Domain IPs | 18.252.90.129, 18.254.215.169, 182.30.36.72 |
| Response IP | 18.252.90.129 |
| Found | Yes |
| Hash | 49443546405feb55567e2d001c1c1212c76a2f987d68ac38980c6e65cbf7459f |
| SimHash | 2084aac44f92 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | hides everything from ALL bots | |
| Allow | /business/s | add path you want to open to bots |
| Allow | /business/s/start-new-business-checklist | - |
Other Records
| Field | Value |
|---|---|
| sitemap | https://service.ct.gov/business/s/sitemap.xml |