dewalt.top
robots.txt
Robots Exclusion Standard data for dewalt.top
Resource Scan
Scan Details
| Site Domain | dewalt.top |
| Base Domain | dewalt.top |
| Scan Status | Ok |
| Last Scan | 2026-01-30T08:25:00+00:00 |
| Next Scan | 2026-03-01T08:25:00+00:00 |
Last Scan
| Scanned | 2026-01-30T08:25:00+00:00 |
| URL | https://dewalt.top/robots.txt |
| Domain IPs | 104.21.30.130, 172.67.172.239, 2606:4700:3030::6815:1e82, 2606:4700:3035::ac43:acef |
| Response IP | 172.67.172.239 |
| Found | Yes |
| Hash | 296a9197459b24dc1439d963afce95cd76330b8f9375165a622f8f0f79bba775 |
| SimHash | 06298b436d93 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*route%3Daccount/ |
| Disallow | /*route%3Daffiliate/ |
| Disallow | /*route%3Dcheckout/ |
| Disallow | /*route%3Dcommon/ |
| Disallow | /*route%3Dproduct/compare |
| Disallow | /*route%3Dproduct/search |
| Disallow | /*route%3Dproduct/product*%26manufacturer_id%3D |
| Disallow | /catalog/controller |
| Disallow | /catalog/language |
| Disallow | /catalog/model |
| Disallow | /download |
| Disallow | /system |
| Disallow | /search |
| Disallow | */storage/ |
| Disallow | /*?sort= |
| Disallow | /*%26sort%3D |
| Disallow | /*?order= |
| Disallow | /*%26order%3D |
| Disallow | /*?limit= |
| Disallow | /*%26limit%3D |
| Disallow | /*?filter= |
| Disallow | /*%26filter%3D |
| Disallow | /*?filter_name= |
| Disallow | /*%26filter_name%3D |
| Disallow | /*?filter_sub_category= |
| Disallow | /*%26filter_sub_category%3D |
| Disallow | /*?filter_description= |
| Disallow | /*%26filter_description%3D |
| Disallow | /*?tracking= |
| Disallow | /*%26tracking%3D |
| Disallow | /*%26tag%3D* |
| Disallow | /*?tag=* |
| Allow | *.css |
| Allow | *.js |
| Allow | *.jpg |
| Allow | *.jpeg |
| Allow | *.gif |
| Allow | *.png |
| Allow | *.svg |
| Allow | *.webp |
| Allow | *.ttf |
| Allow | *.woff |
| Allow | *.woff2 |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Comments