chicagoglobaltimes.com
robots.txt
Robots Exclusion Standard data for chicagoglobaltimes.com
Resource Scan
Scan Details
| Site Domain | chicagoglobaltimes.com |
| Base Domain | chicagoglobaltimes.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server redirected incorrectly. |
| Last Scan | 2024-09-11T15:28:08+00:00 |
| Next Scan | 2024-09-25T15:28:08+00:00 |
Last Successful Scan
| Scanned | 2024-08-25T12:13:44+00:00 |
| URL | https://chicagoglobaltimes.com/robots.txt |
| Domain IPs | 2a02:4780:15:ab37:6433:6733:ee4d:bb6, 84.32.84.180 |
| Response IP | 191.101.228.137 |
| Found | Yes |
| Hash | 3afd4b78c15db168e28ad4c902fe219614a9cc9124a71c63f75b4806d6e455b3 |
| SimHash | 789d5959e6e0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search/ |
| Disallow | /pages/search/ |
| Disallow | /pages/news-connect |
| Disallow | /error404.aspx |
| Disallow | /widget/ |
| Disallow | /*ns/local_news* |
| Disallow | /bentoapi/ |
| Disallow | /*?*canonicalCard= |
Other Records
| Field | Value |
|---|---|
| sitemap | /sitemap/nbcnews/sitemap-index |
| sitemap | /sitemap/nbcnews/sitemap-news |
| sitemap | /sitemap/nbcnews/sitemap-curations |
| sitemap | /sitemap/nbcnews/sitemap-select.xml |
| sitemap | /politics/election-results/sitemap.xml |
Comments