streetwise.co
robots.txt
Robots Exclusion Standard data for streetwise.co
Resource Scan
Scan Details
| Site Domain | streetwise.co |
| Base Domain | streetwise.co |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-10-06T22:14:46+00:00 |
| Next Scan | 2025-01-04T22:14:46+00:00 |
Last Successful Scan
| Scanned | 2021-08-29T19:27:17+00:00 |
| URL | http://streetwise.co/robots.txt |
| Redirect | https://www.bizjournals.com/robots.txt |
| Redirect Domain | www.bizjournals.com |
| Redirect Base | bizjournals.com |
| Found | Yes |
| Hash | 1f90b0ab0774b773a17638bed935908c3a4133ffe60af49c0b1ed1e4192de8e3 |
| SimHash | 447f52e85437 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /myaccount |
| Disallow | /myaccount/* |
| Disallow | /ecommerce/* |
| Disallow | /account/ |
| Disallow | /commerce/ |
| Disallow | /api/ |
| Disallow | /privacy/request-data |
| Disallow | /e-dition/* |
| Disallow | /lib/js/ |
| Disallow | /rss/ |
| Disallow | /search?* |
| Disallow | /*/search?* |
| Disallow | /search/results/ |
| Disallow | /newssearch |
| Disallow | /*/newssearch |
| Disallow | /videosearch |
| Disallow | /*/videosearch |
| Disallow | /*/potmsearch/export |
| Disallow | /library/* |
| Disallow | /login/ |
| Disallow | /login/* |
| Disallow | /register/ |
| Disallow | /email/web-view/ |
| Disallow | /email/web/ |
| Disallow | /*/datacenter/bedrockstub* |
| Disallow | /*/lightreg/index.html |
| Disallow | /*?ana=* |
| Disallow | /*%26ana%3D* |
| Disallow | /*?iana=* |
| Disallow | /*?s=* |
| Disallow | /*%26s%3D* |
| Disallow | /*?surround=* |
| Disallow | /*?comments=* |
| Disallow | /*/my_account/register/ |
| Disallow | /*%26nao%3D* |
| Disallow | /*%26nrc%3D* |
| Disallow | /*%26n%3D* |
| Disallow | /*%26tab%3D* |
| Disallow | /*/_employer_name* |
| Disallow | /*/_city_state* |
| Disallow | /*/_city* |
| Disallow | /*/_category* |
| Disallow | /iframe/ad?* |
| Disallow | /*/portal/* |
| Disallow | /*/ecommerce/cart/* |
| Disallow | /*/ecommerce/checkout/* |
| Disallow | /*/ecommerce/product-credit/* |
| Disallow | /*/onboarding |
| Disallow | /*/photo-center/*/* |
| Disallow | /public/index.php/* |
| Disallow | /docroot/* |
| Disallow | /mybookoflists/* |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /*?ana=* |
| Disallow | /*%26ana%3D* |
| Disallow | /ecommerce/* |
| Disallow | /myaccount* |
| Disallow | /myaccount/* |
| Disallow | /ecommerce/* |
| Disallow | /*/ecommerce/cart/* |
| Disallow | /*/ecommerce/checkout/* |
| Disallow | /*/ecommerce/product-credit/* |
| Disallow | /*/onboarding |
| Disallow | /library/* |
| Disallow | /*/portal/* |
| Disallow | /e-dition/* |
twitterbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /myaccount |
| Disallow | /myaccount/* |
| Disallow | /ecommerce/* |
| Disallow | /account/ |
| Disallow | /commerce/ |
| Disallow | /api/ |
| Disallow | /privacy/request-data |
| Disallow | /e-dition/* |
| Disallow | /lib/js/ |
| Disallow | /rss/ |
| Disallow | /search?* |
| Disallow | /*/search?* |
| Disallow | /search/results/ |
| Disallow | /newssearch |
| Disallow | /*/newssearch |
| Disallow | /videosearch |
| Disallow | /*/videosearch |
| Disallow | /*/potmsearch/export |
| Disallow | /library/* |
| Disallow | /login/ |
| Disallow | /login/* |
| Disallow | /register/ |
| Disallow | /email/web-view/ |
| Disallow | /email/web/ |
| Disallow | /*/datacenter/bedrockstub* |
| Disallow | /*/lightreg/index.html |
| Disallow | /*?s=* |
| Disallow | /*%26s%3D* |
| Disallow | /*?surround=* |
| Disallow | /*?comments=* |
| Disallow | /*/my_account/register/ |
| Disallow | /*%26nao%3D* |
| Disallow | /*%26nrc%3D* |
| Disallow | /*%26n%3D* |
| Disallow | /*%26tab%3D* |
| Disallow | /*/_employer_name* |
| Disallow | /*/_city_state* |
| Disallow | /*/_city* |
| Disallow | /*/_category* |
| Disallow | /iframe/ad?* |
| Disallow | /*/portal/* |
| Disallow | /*/ecommerce/cart/* |
| Disallow | /*/ecommerce/checkout/* |
| Disallow | /*/ecommerce/product-credit/* |
| Disallow | /*/onboarding |
| Disallow | /*/photo-center/*/* |
| Disallow | /public/index.php/* |
| Disallow | /docroot/* |
| Disallow | /mybookoflists/* |
facebot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /myaccount |
| Disallow | /myaccount/* |
| Disallow | /ecommerce/* |
| Disallow | /account/ |
| Disallow | /commerce/ |
| Disallow | /api/ |
| Disallow | /privacy/request-data |
| Disallow | /e-dition/* |
| Disallow | /lib/js/ |
| Disallow | /rss/ |
| Disallow | /search?* |
| Disallow | /*/search?* |
| Disallow | /search/results/ |
| Disallow | /newssearch |
| Disallow | /*/newssearch |
| Disallow | /videosearch |
| Disallow | /*/videosearch |
| Disallow | /*/potmsearch/export |
| Disallow | /library/* |
| Disallow | /login/ |
| Disallow | /login/* |
| Disallow | /register/ |
| Disallow | /email/web-view/ |
| Disallow | /email/web/ |
| Disallow | /*/datacenter/bedrockstub* |
| Disallow | /*/lightreg/index.html |
| Disallow | /*?s=* |
| Disallow | /*%26s%3D* |
| Disallow | /*?surround=* |
| Disallow | /*?comments=* |
| Disallow | /*/my_account/register/ |
| Disallow | /*%26nao%3D* |
| Disallow | /*%26nrc%3D* |
| Disallow | /*%26n%3D* |
| Disallow | /*%26tab%3D* |
| Disallow | /*/_employer_name* |
| Disallow | /*/_city_state* |
| Disallow | /*/_city* |
| Disallow | /*/_category* |
| Disallow | /iframe/ad?* |
| Disallow | /*/portal/* |
| Disallow | /*/ecommerce/cart/* |
| Disallow | /*/ecommerce/checkout/* |
| Disallow | /*/ecommerce/product-credit/* |
| Disallow | /*/onboarding |
| Disallow | /*/photo-center/*/* |
| Disallow | /public/index.php/* |
| Disallow | /docroot/* |
| Disallow | /mybookoflists/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.bizjournals.com/sitemap.xml |
| sitemap | https://www.bizjournals.com/sitemap_news.xml |
Warnings
- 3 invalid lines.