business.dk
robots.txt
Robots Exclusion Standard data for business.dk
Resource Scan
Scan Details
| Site Domain | business.dk |
| Base Domain | business.dk |
| Scan Status | Ok |
| Last Scan | 2024-09-26T09:03:43+00:00 |
| Next Scan | 2024-10-03T09:03:43+00:00 |
Last Scan
| Scanned | 2024-09-26T09:03:43+00:00 |
| URL | https://www.business.dk/robots.txt |
| Redirect | https://www.berlingske.dk/robots.txt |
| Redirect Domain | www.berlingske.dk |
| Redirect Base | berlingske.dk |
| Domain IPs | 96.17.96.21, 96.17.96.24 |
| Redirect IPs | 96.17.96.21, 96.17.96.24 |
| Response IP | 23.44.4.160 |
| Found | Yes |
| Hash | f09292c04f1df6e64b57db116898d5bc2a2cd60ef145d921a8ce7ddde156f5c6 |
| SimHash | 7a44dc04a873 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /api |
| Disallow | /redirect |
| Disallow | /logout |
| Disallow | /login |
| Disallow | /register |
| Disallow | /opener-reload-and-window-close |
| Disallow | /showMore |
| Disallow | /autocomplete |
| Disallow | /image_gallery/ |
| Disallow | /tracking/image_gallery/ |
| Disallow | /artikel-arkiv |
| Disallow | /sonata/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.berlingske.dk/sitemap.xml |
| sitemap | https://www.berlingske.dk/sitemap.xml/news |
Comments