dogleggs.com
robots.txt

Robots Exclusion Standard data for dogleggs.com

Resource Scan

Scan Details

Site Domain dogleggs.com
Base Domain dogleggs.com
Scan Status Ok
Last Scan2025-10-31T13:07:35+00:00
Next Scan 2025-11-30T13:07:35+00:00

Last Scan

Scanned2025-10-31T13:07:35+00:00
URL https://dogleggs.com/robots.txt
Redirect https://www.dogleggs.com/robots.txt
Redirect Domain www.dogleggs.com
Redirect Base dogleggs.com
Domain IPs 192.200.160.19
Redirect IPs 192.200.160.19
Response IP 192.200.160.19
Found Yes
Hash bf482355293087a12d8a76c589159b97b1f9d4e4322094f0092058433fe2ca88
SimHash 7193a914c9c5

Groups

*

Rule Path
Disallow /account.php
Disallow /cart.php
Disallow /checkout.php
Disallow /finishorder.php
Disallow /login.php
Disallow /orderstatus.php
Disallow /postreview.php
Disallow /productimage.php
Disallow /productupdates.php
Disallow /remote.php
Disallow /search.php
Disallow /viewfile.php
Disallow /wishlist.php
Disallow /admin/
Disallow /*sort%3D
Disallow /__socialshop/
Disallow /2000-04-03-08-01-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2700-07-05-carpal-support/
Disallow /2100-09-06-20-02-standard-length-adjustable-dogleggs/
Disallow /4300-06-02-velpeau-sling-m-l/
Disallow /2100-07-05-14-01-standard-length-adjustable-dogleggs/
Disallow /2100-10-06-20-01-standard-length-adjustable-dogleggs/
Disallow /2000-04-03-12-02-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2000-03-02-10-02-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2300-14-10-20-01-standard-length-adjustable-dogleggs/
Disallow /2300-12-10-18-01-standard-length-adjustable-dogleggs/
Disallow /2100-08-07-16-01-standard-length-adjustable-dogleggs/
Disallow /2300-08-06-16-02-standard-length-adjustable-dogleggs/
Disallow /2100-07-06-16-02-standard-length-adjustable-dogleggs/
Disallow /2520-99-chest-strap-replacement/
Disallow /2300-06-04-10-02-standard-length-adjustable-dogleggs/
Disallow /2300-03-02-06-01-standard-length-adjustable-dogleggs/
Disallow /2550-99-extension-tabs/
Disallow /3200-03-vest-s/
Disallow /6100-04-surgi-sox-torso-amputee-s-m/
Disallow /6300-05-surgi-sox-leggings-m/
Disallow /3300-08-ehmer-sling-xl/
Disallow /6300-04-surgi-sox-leggings-s-m/
Disallow /7000-03-swim-snood-s/
Disallow /6400-08-surgi-sox-leggings-w-velcro-closure-xl/
Disallow /6400-04-surgi-sox-leggings-w-velcro-closure-s-m/
Disallow /6300-09-surgi-sox-leggings-xxl/
Disallow /6300-07-surgi-sox-leggings-l/
Disallow /6100-1-surgi-sox-torso-amputee-xxxl/
Disallow /3200-00-vest-xxxs/
Disallow /3300-00-ehmer-sling-xxxs/
Disallow /6100-08-surgi-sox-torso-amputee-xl/
Disallow /6400-1-surgi-sox-leggings-w-velcro-closure-xxxl/
Disallow /2000-05-03-08-01-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2300-10-08-16-02-standard-length-adjustable-dogleggs/
Disallow /2700-08-07-carpal-support-m-l/
Disallow /2700-06-045-carpal-support-xs/
Disallow /2700-07-035-carpal-support-s/
Disallow /2700-06-03-carpal-support-xxs/
Disallow /3350-06-m-l-ehmer-sling/
Disallow /vest-with-ehmer-sling-discount/
Disallow /3350-07-l-ehmer-sling/
Disallow /3350-01-xxs-ehmer-sling/
Disallow /3350-02-xs-ehmer-sling/
Disallow /vest-for-cardiac-monitoring-discount/
Disallow /3350-05-m-ehmer-sling/
Disallow /3350-04-s-m-ehmer-sling/
Disallow /3350-03-s-ehmer-sling/
Disallow /ehmer-sling/
Disallow /2100-08-06-18-02-standard-length-adjustable-dogleggs/
Disallow /6500-08-ss-aural-compression-bandage-xl/
Disallow /3300-99-ehmer-sling-custom/
Disallow /2300-07-05-14-02-standard-length-adjustable-dogleggs/
Disallow /4200-34-02-shoulder-stabilization-system-hobble-straps/
Disallow /2300-11-06-20-01-standard-length-adjustable-dogleggs/
Disallow /9500-99-swim-snood-bundle/
Disallow /2000-06-04-12-02-standard-length-adjustable-dogleggs/
Disallow /2000-05-03-10-01-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2000-99-02-standard-length-adjustable-dogleggs/
Disallow /2300-13-10-24-02-standard-length-adjustable-dogleggs/
Disallow /2000-05-03-10-02-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2100-08-05-16-02-standard-length-adjustable-dogleggs/
Disallow /2100-07-04-16-02-standard-length-adjustable-dogleggs/
Disallow /2000-05-03-08-02-breathaprene-standard-length-adjustable-dogleggs/
Disallow /4200-01-32-02-sss-hobbles/
Disallow /2300-11-06-14-02-standard-length-adjustable-dogleggs/
Disallow /2300-11-07-18-02-standard-length-adjustable-dogleggs/
Disallow /2100-09-07-18-01-standard-length-adjustable-dogleggs/
Disallow /2300-13-08-22-01-standard-length-adjustable-dogleggs/
Disallow /4300-04-02-velpeau-sling-s-m/
Disallow /9300-99-vest-for-cardiac-monitoring-bundle/
Disallow /4300-07-01-velpeau-sling-l/
Disallow /9400-99-surgi-sox-aural-compression-bandage-bundle/
Disallow /2300-07-06-16-01-standard-length-adjustable-dogleggs/
Disallow /2520-14-02-chest-strap-replacement/
Disallow /2300-14-10-20-02-standard-length-adjustable-dogleggs/
Disallow /replacement-hobble-straps-2/
Disallow /2100-06-04-12-01-standard-length-adjustable-dogleggs/
Disallow /2100-07-05-14-02-standard-length-adjustable-dogleggs/
Disallow /2100-07-04-16-01-standard-length-adjustable-dogleggs/
Disallow /2100-08-06-12-01-standard-length-adjustable-dogleggs/
Disallow /2100-09-06-16-01-standard-length-adjustable-dogleggs/
Disallow /2100-08-06-20-02-standard-length-adjustable-dogleggs/
Disallow /4300-01-01-velpeau-sling-xxs/
Disallow /4200-36-02-shoulder-stabilization-system-hobble-straps/
Disallow /3400-99-rear-leg-hobble-system/
Disallow /2520-18-03-chest-strap-replacement/
Disallow /2300-08-05-14-01-standard-length-adjustable-dogleggs/
Disallow /vest-small-medium/
Disallow /2300-09-07-20-01-standard-length-adjustable-dogleggs/
Disallow /2100-09-07-18-02-standard-length-adjustable-dogleggs/
Disallow /2000-03-02-06-01-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2100-07-05-16-01-standard-length-adjustable-dogleggs/
Disallow /2300-06-04-14-01-standard-length-adjustable-dogleggs/
Disallow /4200-20-02-shoulder-stabilization-system-hobble-straps/
Disallow /2300-12-10-20-02-standard-length-adjustable-dogleggs/
Disallow /2000-06-04-14-02-standard-length-adjustable-dogleggs/
Disallow /4200-01-28-02-shoulder-stabilization-system-and-vest/
Disallow /2400-99-01-full-length-adjustable-dogleggs/
Disallow /4300-03-01-velpeau-sling-s/
Disallow /2300-09-06-14-01-standard-length-adjustable-dogleggs/
Disallow /2100-08-07-14-01-standard-length-adjustable-dogleggs/
Disallow /2300-10-07-20-02-standard-length-adjustable-dogleggs/
Disallow /6600-99-surgi-sox-body-suit/
Disallow /4200-02-32-02-shoulder-stabilization-system/
Disallow /2100-09-06-14-02-standard-length-adjustable-dogleggs/
Disallow /4200-02-18-01-shoulder-stabilization-system/
Disallow /2100-08-07-16-02-standard-length-adjustable-dogleggs/
Disallow /2300-14-08-18-02-standard-length-adjustable-dogleggs/
Disallow /4200-02-30-02-shoulder-stabilization-system/
Disallow /2300-13-09-22-02-standard-length-adjustable-dogleggs/
Disallow /4200-01-20-02-sss-hobbles/
Disallow /2300-14-10-24-01-standard-length-adjustable-dogleggs/
Disallow /4200-01-18-01-shoulder-stabilization-system-and-vest/
Disallow /2300-13-08-20-01-standard-length-adjustable-dogleggs/
Disallow /2300-12-10-24-01-standard-length-adjustable-dogleggs/
Disallow /2300-99-02-standard-length-adjustable-dogleggs/
Disallow /2300-12-10-22-01-standard-length-adjustable-dogleggs/
Disallow /2300-12-08-22-02-standard-length-adjustable-dogleggs/
Disallow /2520-08-01-chest-strap-replacement/
Disallow /replacement-chest-strap/
Disallow /vest-xxxs/
Disallow /2000-03-02-08-01-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2750-05-carpal-support-extra-stabilization-strap/
Disallow /2000-03-02-10-01-breathaprene-standard-length-adjustable-dogleggs/
Disallow /vest-extra-small/
Disallow /surgi-sox-products/
Disallow /2000-06-04-12-01-standard-length-adjustable-dogleggs/
Disallow /2700-09-06-carpal-support/
Disallow /2000-06-04-14-01-standard-length-adjustable-dogleggs/
Disallow /2100-06-04-10-02-standard-length-adjustable-dogleggs/
Disallow /2700-07-03-carpal-support/
Disallow /shoulder-products-2/
Disallow /9700-99-surgi-sox-bandaging-products-bundle/
Disallow /2100-06-06-14-02-standard-length-adjustable-dogleggs/
Disallow /2100-07-04-14-02-standard-length-adjustable-dogleggs/
Disallow /2100-07-05-12-01-standard-length-adjustable-dogleggs/
Disallow /2100-07-05-12-02-standard-length-adjustable-dogleggs/
Disallow /2100-08-05-14-01-standard-length-adjustable-dogleggs/
Disallow /2520-16-02-chest-strap-replacement/
Disallow /2100-08-06-12-02-standard-length-adjustable-dogleggs/
Disallow /cardiology-monitoring-products-2/
Disallow /2520-10-01-chest-strap-replacement/
Disallow /2520-09-01-chest-strap-replacement/
Disallow /2300-12-08-22-01-standard-length-adjustable-dogleggs/
Disallow /3400-02-rear-leg-hobble-system-xs/
Disallow /2300-08-05-18-02-standard-length-adjustable-dogleggs/
Disallow /2300-09-07-14-01-standard-length-adjustable-dogleggs/
Disallow /2300-09-06-20-01-standard-length-adjustable-dogleggs/
Disallow /6100-99-surgi-sox-torso-amputee-custom/
Disallow /6500-05-ss-aural-compression-bandage-m/
Disallow /6400-99-surgi-sox-leggings-w-velcro-closure-custom/
Disallow /2300-09-05-16-01-standard-length-adjustable-dogleggs/
Disallow /2100-10-08-20-01-standard-length-adjustable-dogleggs/
Disallow /2300-08-07-14-01-standard-length-adjustable-dogleggs/
Disallow /5200-05-device-case-strap-sm/
Disallow /2300-08-06-18-02-standard-length-adjustable-dogleggs/
Disallow /2100-99-01-standard-length-adjustable-dogleggs/
Disallow /2300-08-06-12-01-standard-length-adjustable-dogleggs/
Disallow /2300-09-06-18-02-standard-length-adjustable-dogleggs/
Disallow /5200-99-device-case-strap/
Disallow /2200-99-01-dogleggs-for-amputees/
Disallow /2300-07-05-18-02-standard-length-adjustable-dogleggs/
Disallow /2300-07-05-16-02-standard-length-adjustable-dogleggs/
Disallow /2300-07-05-16-01-standard-length-adjustable-dogleggs/
Disallow /2300-04-03-08-01-standard-length-adjustable-dogleggs/
Disallow /2300-04-03-10-01-standard-length-adjustable-dogleggs/
Disallow /2300-04-03-12-02-standard-length-adjustable-dogleggs/
Disallow /2300-07-04-14-02-standard-length-adjustable-dogleggs/
Disallow /2300-05-03-12-02-standard-length-adjustable-dogleggs/
Disallow /2300-06-04-12-02-standard-length-adjustable-dogleggs/
Disallow /2300-06-04-12-01-standard-length-adjustable-dogleggs/
Disallow /2300-09-08-18-02-standard-length-adjustable-dogleggs/
Disallow /2300-12-08-20-01-standard-length-adjustable-dogleggs/
Disallow /4200-32-02-shoulder-stabilization-system-hobble-straps/
Disallow /2100-09-07-20-01-standard-length-adjustable-dogleggs/
Disallow /8000-15-orthoplast-straps/
Disallow /2300-11-08-18-02-standard-length-adjustable-dogleggs/
Disallow /8000-13-orthoplast-straps/
Disallow /2300-11-06-20-02-standard-length-adjustable-dogleggs/
Disallow /2100-10-06-16-02-standard-length-adjustable-dogleggs/
Disallow /2100-10-06-18-01-standard-length-adjustable-dogleggs/
Disallow /2100-10-07-20-02-standard-length-adjustable-dogleggs/
Disallow /4300-01-02-velpeau-sling-xxs/
Disallow /2300-09-07-18-02-standard-length-adjustable-dogleggs/
Disallow /2300-10-07-18-01-standard-length-adjustable-dogleggs/
Disallow /2300-10-06-16-01-standard-length-adjustable-dogleggs/
Disallow /2300-10-06-20-01-standard-length-adjustable-dogleggs/
Disallow /2300-10-07-16-01-standard-length-adjustable-dogleggs/
Disallow /4300-02-02-velpeau-sling-xs/
Disallow /4200-18-00-05-shoulder-stabilization-system-hobble-straps/
Disallow /3200-99-vest-custom/
Disallow /4200-01-18-00-05-sss-hobbles/
Disallow /4200-02-36-02-shoulder-stabilization-system/
Disallow /4200-99-00-shoulder-stabilization-system-hobble-straps/
Disallow /9000-99-carpal-support-bundle/
Disallow /4300-05-01-velpeau-sling-m/
Disallow /4200-02-28-02-shoulder-stabilization-system/
Disallow /2300-15-09-24-02-standard-length-adjustable-dogleggs/
Disallow /5200-07-device-case-strap-l/
Disallow /4200-99-01-shoulder-stabilization-system-hobble-tabs/
Disallow /device-case-for-vest/
Disallow /5200-06-device-case-strap-ml/
Disallow /9600-99-shoulder-stabilization-system-bundle/
Disallow /2300-06-04-14-02-standard-length-adjustable-dogleggs/
Disallow /2300-15-11-22-01-standard-length-adjustable-dogleggs/
Disallow /2300-15-09-24-01-standard-length-adjustable-dogleggs/
Disallow /2300-13-10-20-01-standard-length-adjustable-dogleggs/
Disallow /2300-05-03-10-01-standard-length-adjustable-dogleggs/
Disallow /2300-05-03-10-02-standard-length-adjustable-dogleggs/
Disallow /2300-13-10-18-01-standard-length-adjustable-dogleggs/
Disallow /2300-05-03-12-01-standard-length-adjustable-dogleggs/
Disallow /2300-05-04-10-01-standard-length-adjustable-dogleggs/
Disallow /2300-12-10-24-02-standard-length-adjustable-dogleggs/
Disallow /2300-12-08-20-02-standard-length-adjustable-dogleggs/
Disallow /2100-06-04-14-02-standard-length-adjustable-dogleggs/
Disallow /2300-06-04-10-01-standard-length-adjustable-dogleggs/
Disallow /2300-13-10-24-01-standard-length-adjustable-dogleggs/
Disallow /2300-11-09-24-02-standard-length-adjustable-dogleggs/
Disallow /2300-11-09-24-01-standard-length-adjustable-dogleggs/
Disallow /2300-11-08-20-01-standard-length-adjustable-dogleggs/
Disallow /2300-07-04-16-01-standard-length-adjustable-dogleggs/
Disallow /2100-06-04-14-01-standard-length-adjustable-dogleggs/
Disallow /2300-10-08-18-02-standard-length-adjustable-dogleggs/
Disallow /2300-10-08-16-01-standard-length-adjustable-dogleggs/
Disallow /2300-09-08-18-01-standard-length-adjustable-dogleggs/
Disallow /2000-06-04-10-02-standard-length-adjustable-dogleggs/
Disallow /2300-09-05-16-02-standard-length-adjustable-dogleggs/
Disallow /2300-08-05-16-02-standard-length-adjustable-dogleggs/
Disallow /2300-08-06-20-01-standard-length-adjustable-dogleggs/
Disallow /2100-09-08-18-02-standard-length-adjustable-dogleggs/
Disallow /2000-03-02-06-02-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2000-04-03-10-01-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2100-08-07-14-02-standard-length-adjustable-dogleggs/
Disallow /2100-09-06-14-01-standard-length-adjustable-dogleggs/
Disallow /2100-09-06-18-01-standard-length-adjustable-dogleggs/
Disallow /2750-08-carpal-support-extra-stabilization-strap/
Disallow /2700-10-10-carpal-support/
Disallow /2100-09-07-14-02-standard-length-adjustable-dogleggs/
Disallow /2100-09-07-16-02-standard-length-adjustable-dogleggs/
Disallow /2100-09-07-20-02-standard-length-adjustable-dogleggs/
Disallow /2700-10-08-carpal-support/
Disallow /2100-08-05-18-02-standard-length-adjustable-dogleggs/
Disallow /2700-045-03-carpal-support/
Disallow /2100-10-06-20-02-standard-length-adjustable-dogleggs/
Disallow /2700-04-03-carpal-support/
Disallow /2100-10-08-18-02-standard-length-adjustable-dogleggs/
Disallow /2400-99-02-full-length-adjustable-dogleggs/
Disallow /2520-12-01-chest-strap-replacement/
Disallow /2300-05-03-08-01-standard-length-adjustable-dogleggs/
Disallow /2300-08-06-18-01-standard-length-adjustable-dogleggs/
Disallow /2300-07-06-16-02-standard-length-adjustable-dogleggs/
Disallow /2300-05-03-08-02-standard-length-adjustable-dogleggs/
Disallow /2100-06-04-12-02-standard-length-adjustable-dogleggs/
Disallow /6000-99-surgi-sox-torso/
Disallow /2100-08-06-18-01-standard-length-adjustable-dogleggs/
Disallow /2300-07-05-14-01-standard-length-adjustable-dogleggs/
Disallow /2100-07-04-14-01-standard-length-adjustable-dogleggs/
Disallow /2100-10-08-18-01-standard-length-adjustable-dogleggs/
Disallow /9100-99-surgi-sox-bandaging-bundle/
Disallow /2100-10-08-16-02-standard-length-adjustable-dogleggs/
Disallow /8100-casting-kit/
Disallow /2100-10-07-16-02-standard-length-adjustable-dogleggs/
Disallow /2100-10-08-22-02-standard-length-adjustable-dogleggs/
Disallow /2100-08-06-16-01-standard-length-adjustable-dogleggs/
Disallow /rear-leg-products-2/
Disallow /2300-14-10-22-02-standard-length-adjustable-dogleggs/
Disallow /4200-30-02-shoulder-stabilization-system-hobble-straps/
Disallow /4200-28-02-shoulder-stabilization-system-hobble-straps/
Disallow /2300-12-08-18-01-standard-length-adjustable-dogleggs/
Disallow /4200-02-34-02-shoulder-stabilization-system/
Disallow /2300-13-09-22-01-standard-length-adjustable-dogleggs/
Disallow /2300-13-10-18-02-standard-length-adjustable-dogleggs/
Disallow /4200-02-20-02-shoulder-stabilization-system/
Disallow /2300-13-10-22-01-standard-length-adjustable-dogleggs/
Disallow /4300-99-02-velpeau-sling/
Disallow /2510-99-elbow-overlay/
Disallow /2000-05-04-10-01-standard-length-adjustable-dogleggs/
Disallow /2700-09-07-carpal-support/
Disallow /2700-09-08-carpal-support/
Disallow /2700-10-09-carpal-support/
Disallow /2000-05-03-12-02-breathaprene-standard-length-adjustable-dogleggs/
Disallow /vest-small/
Disallow /2300-12-07-18-01-standard-length-adjustable-dogleggs/
Disallow /5200-03-device-case-strap-s/
Disallow /4300-99-01-velpeau-sling/
Disallow /4300-07-02-velpeau-sling-l/
Disallow /2300-08-07-16-02-standard-length-adjustable-dogleggs/
Disallow /4300-06-01-velpeau-sling-m-l/
Disallow /2300-09-06-14-02-standard-length-adjustable-dogleggs/
Disallow /2300-09-06-18-01-standard-length-adjustable-dogleggs/
Disallow /2300-10-06-20-02-standard-length-adjustable-dogleggs/
Disallow /2300-11-06-16-02-standard-length-adjustable-dogleggs/
Disallow /4300-00-01-velpeau-sling-xxxs/
Disallow /2000-04-03-12-01-breathaprene-standard-length-adjustable-dogleggs/
Disallow /2520-16-03-chest-strap-replacement/
Disallow /3300-09-ehmer-sling-xxl/
Disallow /7000-05-swim-snood-m/
Disallow /6200-04-surgi-sox-torso-with-velcro-closure-s-m/
Disallow /6000-05-surgi-sox-torso-m/
Disallow /7000-08-swim-snood-xl/
Disallow /3300-01-ehmer-sling-xxs/
Disallow /7000-07-swim-snood-l/
Disallow /6000-08-surgi-sox-torso-xl/
Disallow /4100-04-shoulder-stabilization-vest-sm/
Disallow /3300-03-ehmer-sling-s/
Disallow /6400-06-surgi-sox-leggings-w-velcro-closure-m-l/
Disallow /6300-08-surgi-sox-leggings-xl/
Disallow /4100-03-shoulder-stabilization-vest-s/
Disallow /6300-03-surgi-sox-leggings-s/
Disallow /7100-99-buddy-bag-5/
Disallow /6200-08-surgi-sox-torso-with-velcro-closure-xl/
Disallow /3200-08-vest-xl/
Disallow /3200-04-vest-s-m/
Disallow /6100-09-surgi-sox-torso-amputee-xxl/
Disallow /3300-07-ehmer-sling-l/
Disallow /6200-05-surgi-sox-torso-with-velcro-closure-m/
Disallow /6200-06-surgi-sox-torso-with-velcro-closure-m-l/
Disallow /7000-02-swim-snood-xs/
Disallow /4100-07-shoulder-stabilization-vest-l/
Disallow /6400-05-surgi-sox-leggings-w-velcro-closure-m/
Disallow /6100-02-surgi-sox-torso-amputee-xs/
Disallow /6000-07-surgi-sox-torso-l/
Disallow /6000-04-surgi-sox-torso-s-m/
Disallow /7100-01-buddy-bag-2/
Disallow /7100-00-buddy-bag-1/
Disallow /6200-02-surgi-sox-torso-with-velcro-closure-xs/
Disallow /6100-06-surgi-sox-torso-amputee-m-l/
Disallow /6000-10-surgi-sox-torso-xxxl/
Disallow /3300-06-ehmer-sling-m-l/
Disallow /3000-99-01a-hock-sock/
Disallow /3200-01-vest-xxs/
Disallow /6400-03-surgi-sox-leggings-w-velcro-closure-s/
Disallow /6200-1-surgi-sox-torso-with-velcro-closure-xxxl/
Disallow /3000-99-02a-hock-sock-right/
Disallow /2600-99-front-leg-wrap/
Disallow /3300-05-ehmer-sling-m/
Disallow /3200-02-vest-xs/

*

Rule Path
Disallow /*?_bc_fsnf=1*
Disallow /*%26_bc_fsnf%3D1*

ai2bot
ai2bot-dolma
amazonbot
applebot
applebot-extended
bytespider
ccbot
chatgpt-user
claude-web
claudebot
diffbot
facebookbot
friendlycrawler
gptbot
google-extended
googleother
googleother-image
googleother-video
icc-crawler
isscyberriskcrawler
imagesiftbot
kangaroo bot
meta-externalagent
meta-externalfetcher
oai-searchbot
perplexitybot
petalbot
scrapy
sidetrade indexer bot
timpibot
velenpublicwebcrawler
webzio-extended
youbot
anthropic-ai
cohere-ai
facebookexternalhit
iaskspider/2.0
img2dataset
omgili
omgilibot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10