borsen.no
robots.txt
Robots Exclusion Standard data for borsen.no
Resource Scan
Scan Details
| Site Domain | borsen.no |
| Base Domain | borsen.no |
| Scan Status | Ok |
| Last Scan | 2024-05-14T19:21:06+00:00 |
| Next Scan | 2024-05-21T19:21:06+00:00 |
Last Scan
| Scanned | 2024-05-14T19:21:06+00:00 |
| URL | https://borsen.no/robots.txt |
| Redirect | https://borsen.dagbladet.no/robots.txt |
| Redirect Domain | borsen.dagbladet.no |
| Redirect Base | dagbladet.no |
| Domain IPs | 2a02:c0:ac:3:db::180, 2a02:c0:ac:3:db::181, 2a02:c0:ac:3:db::182, 2a02:c0:ac:3:db::183, 87.238.33.180, 87.238.33.181, 87.238.33.182, 87.238.33.183 |
| Redirect IPs | 104.18.34.161, 172.64.153.95, 2606:4700:4400::6812:22a1, 2606:4700:4400::ac40:995f |
| Response IP | 104.18.34.161 |
| Found | Yes |
| Hash | 693291a2445fcf0994884a17edc6d33eca1de184caea06b68b3bc9254891b1d3 |
| SimHash | 5a0e8971e115 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /youtube-player/ |
| Disallow | /video/playlist/ |
| Disallow | /video/popular/ |
| Disallow | /video/* |
| Disallow | /app/* |
| Allow | /app/cyclops-router-api/api/v1/ |
| Allow | /app/jw-api-proxy/sitemap/index/borsen.xml |
| Allow | /video/*/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://borsen.dagbladet.no/sitemapindex.xml |
| sitemap | https://borsen.dagbladet.no/app/jw-api-proxy/sitemap/index/borsen.xml |