westermann.at
robots.txt
Robots Exclusion Standard data for westermann.at
Resource Scan
Scan Details
| Site Domain | westermann.at |
| Base Domain | westermann.at |
| Scan Status | Ok |
| Last Scan | 2024-10-01T11:08:27+00:00 |
| Next Scan | 2024-10-31T11:08:27+00:00 |
Last Scan
| Scanned | 2024-10-01T11:08:27+00:00 |
| URL | https://westermann.at/robots.txt |
| Redirect | https://www.westermann.at/robots.txt |
| Redirect Domain | www.westermann.at |
| Redirect Base | westermann.at |
| Domain IPs | 217.13.73.82 |
| Redirect IPs | 217.13.73.82 |
| Response IP | 217.13.73.82 |
| Found | Yes |
| Hash | 35c416eaca0fa484488aef22b2f4da84ce3b25e6b0326a9ce525142d6be64259 |
| SimHash | 8a4dd6554b12 |
Groups
node/simplecrawler 0.7.0 (git+https://github.com/cgiffard/node-simplecrawler.git)
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /backend |
| Disallow | /merkzettel |
| Disallow | /suche |
| Disallow | /warenkorb |
| Disallow | /index.php/backend |
| Disallow | /index.php/merkzettel |
| Disallow | /index.php/suche |
| Disallow | /index.php/warenkorb |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.westermann.at/sitemap-production/index.xml |