Log In Sign Up

domains5.shoplineapp.com
robots.txt

Robots Exclusion Standard data for domains5.shoplineapp.com

Resource Scan

Scan Details

Site Domain domains5.shoplineapp.com
Base Domain shoplineapp.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-09-17T11:07:58+00:00
Next Scan 2024-12-16T11:07:58+00:00

Last Successful Scan

Scanned2022-07-25T21:06:41+00:00
URL https://domains5.shoplineapp.com/robots.txt
Response IP 3.1.254.167
Found Yes
Hash d82f687c44ec6b0e2f4249b8887cb7b2bfc663af445751222af8099ebd6b154b
SimHash 855cbf01cdd6

Groups

mj12bot

Rule Path
Disallow /

hubspot

Rule Path
Disallow /

*

Rule Path
Disallow /closed
Disallow /preview/
Disallow /users/
Disallow /orders
Disallow /*?*debug=*
Disallow /*?*theme_preview=*
Disallow /*?*price_range_preview=*
Disallow /*?*draft=*
Disallow /api/
Disallow /themes/
Back to top

Comments

  • robots.txt file for Shopline Merchant
  • split user-agent disallows for different bots, as not all bots may follow google's multi-user-agent standard
  • Allow crawling of all content except
Back to top