rotorbike.com
robots.txt
Robots Exclusion Standard data for rotorbike.com
Resource Scan
Scan Details
| Site Domain | rotorbike.com |
| Base Domain | rotorbike.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-05-08T06:28:18+00:00 |
| Next Scan | 2025-06-07T06:28:18+00:00 |
Last Successful Scan
| Scanned | 2025-03-17T06:25:34+00:00 |
| URL | https://rotorbike.com/robots.txt |
| Domain IPs | 104.26.14.178, 104.26.15.178, 172.67.74.183, 2606:4700:20::681a:eb2, 2606:4700:20::681a:fb2, 2606:4700:20::ac43:4ab7 |
| Response IP | 104.26.15.178 |
| Found | Yes |
| Hash | ba523bbe88c750473d443316cd53d25394af4a43854398e386fd96817d0b705a |
| SimHash | 5741f2438091 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*.csv |
| Disallow | /*.zip$ |
| Disallow | /*.svn$ |
| Disallow | /*.idea$ |
| Disallow | /*.sql$ |
| Disallow | /*.tgz$ |
| Disallow | /app/ |
| Disallow | /bin/ |
| Disallow | /dev/ |
| Disallow | /lib/ |
| Disallow | /phpserver/ |
| Disallow | /pkginfo/ |
| Disallow | /report/ |
| Disallow | /setup/ |
| Disallow | /update/ |
| Disallow | /var/ |
| Disallow | /vendor/ |
| Disallow | /index.php/ |
| Disallow | /catalog/product_compare/ |
| Disallow | /catalog/category/view/ |
| Disallow | /catalog/product/view/ |
| Disallow | /catalogsearch/ |
| Disallow | /checkout/ |
| Disallow | /control/ |
| Disallow | /contacts/ |
| Disallow | /customer/ |
| Disallow | /customize/ |
| Disallow | /newsletter/ |
| Disallow | /review/ |
| Disallow | /sendfriend/ |
| Disallow | /wishlist/ |
| Disallow | /home$ |
| Disallow | /home?* |
| Disallow | /composer.json |
| Disallow | /composer.lock |
| Disallow | /CONTRIBUTING.md |
| Disallow | /CONTRIBUTOR_LICENSE_AGREEMENT.html |
| Disallow | /COPYING.txt |
| Disallow | /Gruntfile.js |
| Disallow | /LICENSE.txt |
| Disallow | /LICENSE_AFL.txt |
| Disallow | /nginx.conf.sample |
| Disallow | /package.json |
| Disallow | /php.ini.sample |
| Disallow | /RELEASE_NOTES.txt |
| Disallow | /*?*discipline= |
| Disallow | /*?*c_shell_widht= |
| Disallow | /*?*c_shell_diameter= |
| Disallow | /*?*c_material_bearings= |
| Disallow | /*?*c_frame_type= |
| Disallow | /*?*c_compatible_crankset_brand= |
| Disallow | /*?*c_color= |
| Disallow | /*?*bcd= |
| Disallow | /*?*c_axle_lenght= |
| Disallow | /*?*c_axle_width= |
| Disallow | /*?*c_chainring_shape= |
| Disallow | /*?*c_chainring_type= |
| Disallow | /*?*c_compatible_crankset_brand= |
| Disallow | /*?*c_compatible_crankset_model= |
| Disallow | /*?*c_compatible_groupsets= |
| Disallow | /*?*c_crank_length= |
| Disallow | /*?*c_frame_type= |
| Disallow | /*?*c_speeds= |
| Disallow | /*?*c_teeth= |
| Disallow | /*?*measurement_type= |
| Disallow | /*?*number_of_chainrings= |
| Disallow | /*?*sprocket_range= |
| Disallow | /*?*product_list_mode= |
| Disallow | /*?*product_list_order= |
| Disallow | /*?*product_list_limit= |
| Disallow | /*?*product_list_dir= |
| Disallow | /*?*price= |
| Disallow | /*?*p= |
| Disallow | /*?*product_list_mode= |
| Disallow | /*?*page= |
| Disallow | /*?*q= |
| Disallow | /*?SID= |
| Disallow | /*? |
| Disallow | /*?* |
| Disallow | /*.php$ |
| Allow | /media/*.css |
| Allow | /media/*.CSS |
| Allow | /media/*.js |
| Allow | /media/*.png |
| Allow | /media/*.gif |
| Allow | /media/*.jpg |
| Allow | /media/*.jpeg |
| Allow | /media/*.svg |
| Allow | /skin/*.css |
| Allow | /skin/*.js |
| Allow | /skin/*.png |
| Allow | /skin/*.gif |
| Allow | /skin/*.jpg |
| Allow | /skin/*.jpeg |
| Allow | /skin/*.svg |
Other Records
| Field | Value |
|---|---|
| sitemap | https://rotorbike.com/media/sitemap/sitemapww.xml |
Comments