westermann.de
robots.txt
Robots Exclusion Standard data for westermann.de
Resource Scan
Scan Details
| Site Domain | westermann.de |
| Base Domain | westermann.de |
| Scan Status | Ok |
| Last Scan | 2024-10-21T11:07:13+00:00 |
| Next Scan | 2024-11-20T11:07:13+00:00 |
Last Scan
| Scanned | 2024-10-21T11:07:13+00:00 |
| URL | https://westermann.de/robots.txt |
| Redirect | https://www.westermann.de/robots.txt |
| Redirect Domain | www.westermann.de |
| Redirect Base | westermann.de |
| Domain IPs | 217.13.73.82 |
| Redirect IPs | 217.13.73.82 |
| Response IP | 217.13.73.82 |
| Found | Yes |
| Hash | 13a49b72cfb0092979662dc5ca0f569f83be311273170b48a740719d406695b2 |
| SimHash | a24dd6414b12 |
Groups
node/simplecrawler 0.7.0 (git+https://github.com/cgiffard/node-simplecrawler.git)
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /backend |
| Disallow | /merkzettel |
| Disallow | /suche |
| Disallow | /warenkorb |
| Disallow | /index.php/backend |
| Disallow | /index.php/merkzettel |
| Disallow | /index.php/suche |
| Disallow | /index.php/warenkorb |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.westermann.de/sitemap-production/index.xml |