horseandrideruk.com
robots.txt

Robots Exclusion Standard data for horseandrideruk.com

Resource Scan

Scan Details

Site Domain horseandrideruk.com
Base Domain horseandrideruk.com
Scan Status Ok
Last Scan2024-08-31T23:02:26+00:00
Next Scan 2024-09-30T23:02:26+00:00

Last Scan

Scanned2024-08-31T23:02:26+00:00
URL https://horseandrideruk.com/robots.txt
Domain IPs 104.26.8.220, 104.26.9.220, 172.67.73.154, 2606:4700:20::681a:8dc, 2606:4700:20::681a:9dc, 2606:4700:20::ac43:499a
Response IP 172.67.73.154
Found Yes
Hash 2502a2597091bd21b61532b6dfa3c2476be388e674cda3f220902b0066c94797
SimHash 407889098e83

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /shop/catalog/product_compare/
Disallow /shop/catalog/category/view/
Disallow /shop/catalog/product/view/
Disallow /shop/catalogsearch/
Disallow /shop/checkout/
Disallow /shop/control/
Disallow /shop/contacts/
Disallow /shop/customer/
Disallow /shop/customize/
Disallow /shop/newsletter/
Disallow /shop/poll/
Disallow /shop/review/
Disallow /shop/sendfriend/
Disallow /shop/tag/
Disallow /shop/wishlist/

mj12bot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrush

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

obot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10