bizjournal.com
robots.txt
Robots Exclusion Standard data for bizjournal.com
Resource Scan
Scan Details
| Site Domain | bizjournal.com |
| Base Domain | bizjournal.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2025-09-26T02:47:55+00:00 |
| Next Scan | 2025-12-25T02:47:55+00:00 |
Last Successful Scan
| Scanned | 2024-09-01T15:02:03+00:00 |
| URL | http://bizjournal.com/robots.txt |
| Redirect | https://www.bizjournals.com/robots.txt |
| Redirect Domain | www.bizjournals.com |
| Redirect Base | bizjournals.com |
| Domain IPs | 52.73.191.152, 54.152.42.182 |
| Redirect IPs | 45.64.67.27 |
| Response IP | 45.64.67.27 |
| Found | Yes |
| Hash | 78030e2aad11e75da73768f068f78d69881d553d2073407dfaaa0be9756361d0 |
| SimHash | 443f5a605417 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /myaccount |
| Disallow | /myaccount/* |
| Disallow | /ecommerce/* |
| Disallow | /account/ |
| Disallow | /commerce/ |
| Allow | /api/legendary-data/relay |
| Disallow | /api/ |
| Disallow | /privacy/request-data |
| Disallow | /e-dition/* |
| Disallow | /lib/js/ |
| Disallow | /rss/ |
| Disallow | /search?* |
| Disallow | /*/search?* |
| Disallow | /search/results/ |
| Disallow | /newssearch |
| Disallow | /*/newssearch |
| Disallow | /videosearch |
| Disallow | /*/videosearch |
| Disallow | /*/potmsearch/export |
| Disallow | /library/* |
| Disallow | /login/ |
| Disallow | /login/* |
| Disallow | /register/ |
| Disallow | /email/web-view/ |
| Disallow | /email/web/ |
| Disallow | /*/datacenter/bedrockstub* |
| Disallow | /*/lightreg/index.html |
| Disallow | /*?ana=* |
| Disallow | /*%26ana%3D* |
| Disallow | /*?iana=* |
| Disallow | /*?s=* |
| Disallow | /*%26s%3D* |
| Disallow | /*?surround=* |
| Disallow | /*?comments=* |
| Disallow | /*/my_account/register/ |
| Disallow | /*%26nao%3D* |
| Disallow | /*%26nrc%3D* |
| Disallow | /*%26n%3D* |
| Disallow | /*%26tab%3D* |
| Disallow | */jobs/*q%3D |
| Disallow | /*/_employer_name* |
| Disallow | /*/_city_state* |
| Disallow | /*/_city* |
| Disallow | /*/_category* |
| Disallow | /iframe/ad?* |
| Disallow | /*/offer/* |
| Disallow | /*/ecommerce/cart/* |
| Disallow | /*/ecommerce/checkout/* |
| Disallow | /*/ecommerce/product-credit/* |
| Disallow | /*/onboarding |
| Disallow | /*/photo-center/*/* |
| Disallow | /public/index.php/* |
| Disallow | /docroot/* |
| Disallow | /mybookoflists/* |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /*?ana=* |
| Disallow | /*%26ana%3D* |
| Disallow | /ecommerce/* |
| Disallow | /myaccount* |
| Disallow | /myaccount/* |
| Disallow | /ecommerce/* |
| Disallow | /*/ecommerce/cart/* |
| Disallow | /*/ecommerce/checkout/* |
| Disallow | /*/ecommerce/product-credit/* |
| Disallow | /*/onboarding |
| Disallow | /library/* |
| Disallow | /*/portal/* |
| Disallow | /e-dition/* |
twitterbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /myaccount |
| Disallow | /myaccount/* |
| Disallow | /ecommerce/* |
| Disallow | /account/ |
| Disallow | /commerce/ |
| Disallow | /api/ |
| Disallow | /privacy/request-data |
| Disallow | /e-dition/* |
| Disallow | /lib/js/ |
| Disallow | /rss/ |
| Disallow | /search?* |
| Disallow | /*/search?* |
| Disallow | /search/results/ |
| Disallow | /newssearch |
| Disallow | /*/newssearch |
| Disallow | /videosearch |
| Disallow | /*/videosearch |
| Disallow | /*/potmsearch/export |
| Disallow | /library/* |
| Disallow | /login/ |
| Disallow | /login/* |
| Disallow | /register/ |
| Disallow | /email/web-view/ |
| Disallow | /email/web/ |
| Disallow | /*/datacenter/bedrockstub* |
| Disallow | /*/lightreg/index.html |
| Disallow | /*?s=* |
| Disallow | /*%26s%3D* |
| Disallow | /*?surround=* |
| Disallow | /*?comments=* |
| Disallow | /*/my_account/register/ |
| Disallow | /*%26nao%3D* |
| Disallow | /*%26nrc%3D* |
| Disallow | /*%26n%3D* |
| Disallow | /*%26tab%3D* |
| Disallow | */jobs/*q%3D |
| Disallow | /*/_employer_name* |
| Disallow | /*/_city_state* |
| Disallow | /*/_city* |
| Disallow | /*/_category* |
| Disallow | /iframe/ad?* |
| Disallow | /*/offer/* |
| Disallow | /*/ecommerce/cart/* |
| Disallow | /*/ecommerce/checkout/* |
| Disallow | /*/ecommerce/product-credit/* |
| Disallow | /*/onboarding |
| Disallow | /*/photo-center/*/* |
| Disallow | /public/index.php/* |
| Disallow | /docroot/* |
| Disallow | /mybookoflists/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
facebot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /myaccount |
| Disallow | /myaccount/* |
| Disallow | /ecommerce/* |
| Disallow | /account/ |
| Disallow | /commerce/ |
| Disallow | /api/ |
| Disallow | /privacy/request-data |
| Disallow | /e-dition/* |
| Disallow | /lib/js/ |
| Disallow | /rss/ |
| Disallow | /search?* |
| Disallow | /*/search?* |
| Disallow | /search/results/ |
| Disallow | /newssearch |
| Disallow | /*/newssearch |
| Disallow | /videosearch |
| Disallow | /*/videosearch |
| Disallow | /*/potmsearch/export |
| Disallow | /library/* |
| Disallow | /login/ |
| Disallow | /login/* |
| Disallow | /register/ |
| Disallow | /email/web-view/ |
| Disallow | /email/web/ |
| Disallow | /*/datacenter/bedrockstub* |
| Disallow | /*/lightreg/index.html |
| Disallow | /*?s=* |
| Disallow | /*%26s%3D* |
| Disallow | /*?surround=* |
| Disallow | /*?comments=* |
| Disallow | /*/my_account/register/ |
| Disallow | /*%26nao%3D* |
| Disallow | /*%26nrc%3D* |
| Disallow | /*%26n%3D* |
| Disallow | /*%26tab%3D* |
| Disallow | */jobs/*q%3D |
| Disallow | /*/_employer_name* |
| Disallow | /*/_city_state* |
| Disallow | /*/_city* |
| Disallow | /*/_category* |
| Disallow | /iframe/ad?* |
| Disallow | /*/offer/* |
| Disallow | /*/ecommerce/cart/* |
| Disallow | /*/ecommerce/checkout/* |
| Disallow | /*/ecommerce/product-credit/* |
| Disallow | /*/onboarding |
| Disallow | /*/photo-center/*/* |
| Disallow | /public/index.php/* |
| Disallow | /docroot/* |
| Disallow | /mybookoflists/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.bizjournals.com/sitemap.xml |
| sitemap | https://www.bizjournals.com/sitemap_news.xml |