rasney.com
robots.txt

Robots Exclusion Standard data for rasney.com

Resource Scan

Scan Details

Site Domain rasney.com
Base Domain rasney.com
Scan Status Ok
Last Scan2025-08-19T02:14:47+00:00
Next Scan 2025-09-18T02:14:47+00:00

Last Scan

Scanned2025-08-19T02:14:47+00:00
URL https://rasney.com/robots.txt
Redirect https://www.rasney.com/robots.txt
Redirect Domain www.rasney.com
Redirect Base rasney.com
Domain IPs 104.21.52.233, 172.67.205.61, 2606:4700:3035::ac43:cd3d, 2606:4700:3037::6815:34e9
Redirect IPs 104.21.52.233, 172.67.205.61, 2606:4700:3035::ac43:cd3d, 2606:4700:3037::6815:34e9
Response IP 104.21.52.233
Found Yes
Hash 77b098a605381a11f2c3a903f79c1f38b0c1d325d6c5e1ea6ec2603ff291d9ef
SimHash af149f4afcc0

Groups

*

Rule Path
Disallow /a/downloads/-/*
Disallow /admin
Disallow /cart
Disallow /orders
Disallow /checkout
Disallow /*/checkouts
Disallow /*/orders
Disallow /carts
Disallow /account
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*design_theme_id*
Disallow /*preview_theme_id*
Disallow /*preview_script_id*
Disallow /policies/
Disallow /search
Disallow /apple-app-site-association
Disallow /pages/*

adsbot-google

Rule Path
Disallow /checkout
Disallow /carts
Disallow /orders
Disallow /*/checkouts
Disallow /*/orders
Disallow /*design_theme_id*
Disallow /*preview_theme_id*
Disallow /*preview_script_id*

nutch

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ahrefssiteaudit

Rule Path
Disallow /a/downloads/-/*
Disallow /admin
Disallow /cart
Disallow /orders
Disallow /checkout
Disallow /*/checkouts
Disallow /*/orders
Disallow /carts
Disallow /account
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*design_theme_id*
Disallow /*preview_theme_id*
Disallow /*preview_script_id*
Disallow /policies/
Disallow /search
Disallow /apple-app-site-association

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Comments

  • we use Shopify as our ecommerce platform
  • Google adsbot ignores robots.txt unless specifically named!