thulestore.co.za
robots.txt
Robots Exclusion Standard data for thulestore.co.za
Resource Scan
Scan Details
| Site Domain | thulestore.co.za |
| Base Domain | thulestore.co.za |
| Scan Status | Ok |
| Last Scan | 2026-02-11T21:18:36+00:00 |
| Next Scan | 2026-02-25T21:18:36+00:00 |
Last Scan
| Scanned | 2026-02-11T21:18:36+00:00 |
| URL | https://thulestore.co.za/robots.txt |
| Domain IPs | 104.26.14.23, 104.26.15.23, 172.67.68.12, 2606:4700:20::681a:e17, 2606:4700:20::681a:f17, 2606:4700:20::ac43:440c |
| Response IP | 172.67.68.12 |
| Found | Yes |
| Hash | f8e3005e928df275ad63608ec80a8985c93b4676657fce9e9213a524b2b4cf78 |
| SimHash | cd2f7e8bc3b3 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Allow | *?p=* | - |
| Allow | *?q=* | - |
| Allow | /search?q=* | - |
| Disallow | /*?* | - |
| Disallow | /?*cat=* | - |
| Disallow | /?*price=* | - |
| Disallow | /?*brand_name=* | - |
| Disallow | /?*gender* | - |
| Disallow | /404/ | - |
| Disallow | /admin/ | - |
| Disallow | /api/ | - |
| Disallow | /app/ | - |
| Disallow | /cgi-bin/ | - |
| Disallow | /downloader/ | - |
| Disallow | /errors/ | - |
| Disallow | /includes/ | - |
| Disallow | /js/ | - |
| Disallow | /lib/ | - |
| Disallow | /magento/ | - |
| Disallow | /media/email/ | - |
| Disallow | /media/sales/ | - |
| Disallow | /media/customer/ | - |
| Disallow | /media/downloadable/ | - |
| Disallow | /media/pdf-printouts/ | - |
| Disallow | /pkginfo/ | - |
| Disallow | /report/ | - |
| Disallow | /scripts/ | - |
| Disallow | /shell/ | - |
| Disallow | /skin/ | - |
| Disallow | /stats/ | - |
| Disallow | /var/ | - |
| Disallow | /index.php/ | - |
| Disallow | /catalog/product_compare/ | - |
| Disallow | /catalog/category/view/ | - |
| Disallow | /catalog/product/view/ | - |
| Disallow | /catalog/product/gallery/ | - |
| Disallow | /catalogsearch/ | - |
| Disallow | /checkout/ | - |
| Disallow | /control/ | - |
| Disallow | /contacts/ | - |
| Disallow | /customer/ | - |
| Disallow | /customize/ | - |
| Disallow | /newsletter/ | - |
| Disallow | /poll/ | - |
| Disallow | /review/ | - |
| Disallow | /sendfriend/ | - |
| Disallow | /tag/ | - |
| Disallow | /wishlist/ | - |
| Disallow | /cron.php | - |
| Disallow | /cron.sh | - |
| Disallow | /error_log | - |
| Disallow | /install.php | - |
| Disallow | /LICENSE.html | - |
| Disallow | /LICENSE.txt | - |
| Disallow | /LICENSE_AFL.txt | - |
| Disallow | /STATUS.txt | - |
| Disallow | /get.php | Magento 1.5+ |
| Disallow | /*.php$ | - |
| Disallow | /*?id= | - |
| Disallow | /*?dir= | - |
| Disallow | /*?order= | - |
| Disallow | /*?__store= | - |
| Disallow | /*?SID= | - |
| Disallow | /rss* | - |
| Disallow | /*PHPSESSID | - |
| Disallow | /catalog/category/view/* | - |
| Disallow | /catalog/product/view/* | - |
| Disallow | /customer/account/* | - |
| Disallow | /search/* | - |
| Disallow | /catalogsearch/* | - |
| Disallow | /admin/ | - |
| Disallow | /api/ | - |
| Disallow | *mof_data%3D | - |
| Disallow | /review/product/listAjax/id/ | - |
| Disallow | /*?p=* | - |
| Disallow | /*%26p%3D* | - |
| Disallow | /?*cat=* | - |
| Disallow | /?*price=* | - |
| Disallow | /?*brand_name=* | - |
| Disallow | /?*gender* | - |
Comments