layer-grosshandel.de
robots.txt
Robots Exclusion Standard data for layer-grosshandel.de
Resource Scan
Scan Details
Site Domain | layer-grosshandel.de |
Base Domain | layer-grosshandel.de |
Scan Status | Ok |
Last Scan | 2024-10-06T03:15:40+00:00 |
Next Scan | 2024-11-05T03:15:40+00:00 |
Last Scan
Scanned | 2024-10-06T03:15:40+00:00 |
URL | http://layer-grosshandel.de/robots.txt |
Redirect | https://shop.layer-grosshandel.de//robots.txt |
Redirect Domain | shop.layer-grosshandel.de |
Redirect Base | layer-grosshandel.de |
Domain IPs | 213.131.254.53 |
Redirect IPs | 213.131.254.53 |
Response IP | 213.131.254.53 |
Found | Yes |
Hash | 8e45e1ded385660697d372abad40d3669ca1b80635c8cc9af5660922fe6b78d9 |
SimHash | 1ebac850ccea |
Groups
amazonbot
Product | Comment |
---|---|
amazonbot | Amazon's user agent |
Rule | Path | Comment |
---|---|---|
Disallow | /do-not-crawl/ | disallow this directory |
*
Rule | Path |
---|---|
Disallow | /maschinenring.php |
Disallow | *view%3Dprint* |
Disallow | *action2%3Dlogin* |
Disallow | *action%3Dajax_* |
Disallow | *add2compare* |
Disallow | *basket_add_hidden* |
Disallow | *feature_search_temp_table_id* |
Disallow | /eshop.php?action=customer&part=waldner&sub=main&method=show_login |
Other Records
Field | Value |
---|---|
crawl-delay | 120 |
Other Records
Field | Value |
---|---|
sitemap | https://shop.layer-grosshandel.de/sitemap/google_sitemap_index.xml |
Warnings
- 4 invalid lines.
Comments