nordic.businessinsider.com
robots.txt
Robots Exclusion Standard data for nordic.businessinsider.com
Resource Scan
Scan Details
| Site Domain | nordic.businessinsider.com |
| Base Domain | businessinsider.com |
| Scan Status | Ok |
| Last Scan | 2024-04-26T05:19:06+00:00 |
| Next Scan | 2024-05-26T05:19:06+00:00 |
Last Scan
| Scanned | 2024-04-26T05:19:06+00:00 |
| URL | https://nordic.businessinsider.com/robots.txt |
| Redirect | https://www.businessinsider.com/robots.txt?r=nordic |
| Redirect Domain | www.businessinsider.com |
| Redirect Base | businessinsider.com |
| Domain IPs | 151.101.1.171, 151.101.129.171, 151.101.193.171, 151.101.65.171 |
| Redirect IPs | 151.101.1.171, 151.101.129.171, 151.101.193.171, 151.101.65.171 |
| Response IP | 199.232.45.171 |
| Found | Yes |
| Hash | ab5b3458202bd32b79d30232a6113ce46ccb0fcf7de384ce9976e51e357bf823 |
| SimHash | 8343c04deb5f |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*?utm_campaign=Monitor& |
| Disallow | /adframe |
| Disallow | /afp$ |
| Disallow | /ajax/ |
| Disallow | /archives |
| Disallow | /associated-press$ |
| Disallow | /authentication$ |
| Disallow | /author/*/date |
| Disallow | /author/*/mostread |
| Disallow | /categories |
| Disallow | /cms/ |
| Disallow | /comments$ |
| Disallow | /cross-domain$ |
| Disallow | /document/ |
| Disallow | /fonts-async.css |
| Disallow | /insider$ |
| Disallow | /partner/ |
| Disallow | /reuters$ |
| Disallow | /reviews/out? |
| Disallow | /s? |
| Disallow | /guides/s? |
| Disallow | /personal-finance/s? |
| Disallow | /search$ |
| Disallow | /track.gif |
| Disallow | /uk$ |
| Disallow | /ws/ |
| Disallow | /.well-known/amphtml/apikey.pub |
| Disallow | /business-insider$ |
| Disallow | /*/contributor |
| Disallow | /bi$ |
| Disallow | /news-insider$ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.businessinsider.com/sitemap/latest.xml |
| sitemap | https://www.businessinsider.com/sitemap/google-news.xml |
| sitemap | https://www.businessinsider.com/sitemap/index.xml |
| sitemap | https://www.businessinsider.com/sitemap/landing-pages.xml |
| sitemap | https://www.businessinsider.com/sitemap/insider/migrated.xml |
| sitemap | https://feeds.businessinsider.com/custom/tech-sj-sitemap |
| sitemap | https://feeds.businessinsider.com/custom/pfi-sitemap |