kleton.com
robots.txt
Robots Exclusion Standard data for kleton.com
Resource Scan
Scan Details
| Site Domain | kleton.com |
| Base Domain | kleton.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-12-21T13:18:11+00:00 |
| Next Scan | 2026-01-20T13:18:11+00:00 |
Last Successful Scan
| Scanned | 2025-11-19T06:15:45+00:00 |
| URL | https://kleton.com/robots.txt |
| Redirect | http://www.kleton.com/robots.txt |
| Redirect Domain | www.kleton.com |
| Redirect Base | kleton.com |
| Domain IPs | 104.18.12.231, 104.18.13.231, 2606:4700::6812:ce7, 2606:4700::6812:de7 |
| Redirect IPs | 104.18.12.231, 104.18.13.231, 2606:4700::6812:ce7, 2606:4700::6812:de7 |
| Response IP | 104.18.12.231 |
| Found | Yes |
| Hash | b58dcf06838eac7e5168fe5149a380af71d1ffa9f88e3d0541f0ff0f88cee1c7 |
| SimHash | 233c8073c8d2 |
Groups
velenpublicwebcrawler
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 120 |
bingbot
msnbot
msnbot-media
adidxbot
bingpreview
linkedinbot
adroll dynamic creative feed parser bot (delight@adroll.com)
| Rule | Path |
|---|---|
| Allow | /category/*?* |
| Allow | /categorie/*?* |
| Allow | /brand/*?* |
| Allow | /marque/*?* |
| Disallow | /search |
| Disallow | /rechercher |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |
storebot-google
| Rule | Path |
|---|---|
| Allow | /shop/*?* |
| Allow | /magasin/*?* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |
*
| Rule | Path |
|---|---|
| Disallow | /category/*?* |
| Disallow | /categorie/*?* |
| Disallow | /brand/*?* |
| Disallow | /marque/*?* |
| Disallow | /campaign/*?* |
| Disallow | /campagne/*?* |
| Disallow | /product/*?* |
| Disallow | /produit/*?* |
| Disallow | /account/login?* |
| Disallow | /*?*sort= |
| Disallow | /*?*pagesize= |
| Disallow | /*%26sort%3D |
| Disallow | /*%26pagesize%3D |
| Disallow | /compte/connexion?* |
| Disallow | /rechercher |
| Disallow | /search |
| Allow | /*?page=* |
| Disallow | /cdn-cgi/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |
Comments