data.cdc.gov
robots.txt
Robots Exclusion Standard data for data.cdc.gov
Resource Scan
Scan Details
| Site Domain | data.cdc.gov |
| Base Domain | cdc.gov |
| Scan Status | Ok |
| Last Scan | 2025-03-03T15:27:50+00:00 |
| Next Scan | 2025-04-02T15:27:50+00:00 |
Last Scan
| Scanned | 2025-03-03T15:27:50+00:00 |
| URL | https://data.cdc.gov/robots.txt |
| Domain IPs | 52.206.140.199, 52.206.140.205, 52.206.68.26 |
| Response IP | 52.206.140.199 |
| Found | Yes |
| Hash | 77874860972a39a5270fb350b61b122f64de8220493a05b661a684c077ad8d15 |
| SimHash | ef08f8657936 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /browse?*&category= |
| Disallow | /browse?*&federation_filter= |
| Disallow | /browse?*&limitTo= |
| Disallow | /browse?*&q= |
| Disallow | /browse?*&sortBy= |
| Disallow | /browse?*&tags= |
| Disallow | /browse?*&view_type= |
| Disallow | /browse/*?*&category= |
| Disallow | /browse/*?*&federation_filter= |
| Disallow | /browse/*?*&limitTo= |
| Disallow | /browse/*?*&q= |
| Disallow | /browse/*?*&sortBy= |
| Disallow | /browse/*?*&tags= |
| Disallow | /browse/*?*&view_type= |
| Disallow | /*/browse?*&category= |
| Disallow | /*/browse?*&federation_filter= |
| Disallow | /*/browse?*&limitTo= |
| Disallow | /*/browse?*&q= |
| Disallow | /*/browse?*&sortBy= |
| Disallow | /*/browse?*&tags= |
| Disallow | /*/browse?*&view_type= |
| Disallow | /page/*?*&category= |
| Disallow | /page/*?*&federation_filter= |
| Disallow | /page/*?*&limitTo= |
| Disallow | /page/*?*&q= |
| Disallow | /page/*?*&sortBy= |
| Disallow | /page/*?*&tags= |
| Disallow | /page/*?*&view_type= |
| Disallow | /catalog/*?*&category= |
| Disallow | /catalog/*?*&federation_filter= |
| Disallow | /catalog/*?*&limitTo= |
| Disallow | /catalog/*?*&q= |
| Disallow | /catalog/*?*&sortBy= |
| Disallow | /catalog/*?*&tags= |
| Disallow | /catalog/*?*&view_type= |
| Disallow | /facet/*?*&category= |
| Disallow | /facet/*?*&federation_filter= |
| Disallow | /facet/*?*&limitTo= |
| Disallow | /facet/*?*&q= |
| Disallow | /facet/*?*&sortBy= |
| Disallow | /facet/*?*&tags= |
| Disallow | /facet/*?*&view_type= |
| Disallow | */alt$ |
| Disallow | */alt? |
| Disallow | */edit$ |
| Disallow | /*/*/*/widget_preview |
| Disallow | /OData.svc/ |
| Disallow | /api/odata/ |
| Disallow | /browse-preview |
| Disallow | /*/browse-preview |
| Disallow | /browse/embed |
| Disallow | /browse/select_dataset |
| Disallow | /*/browse/select_dataset |
| Disallow | /login |
| Disallow | /reset_password/ |
| Disallow | /tiles/ |
| Disallow | /views/INLINE/rows.json?*method=clustered2* |
| Disallow | /api/collocate* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://s3.amazonaws.com/sa-socrata-sitemaps-us-east-1-fedramp-prod/sitemaps/sitemap-data.cdc.gov.xml |