myus.com
robots.txt

Robots Exclusion Standard data for myus.com

Resource Scan

Scan Details

Site Domain myus.com
Base Domain myus.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-04-09T20:59:33+00:00
Next Scan 2024-07-08T20:59:33+00:00

Last Successful Scan

Scanned2021-10-13T20:46:27+00:00
URL https://myus.com/robots.txt
Redirect https://www.myus.com/robots.txt
Redirect Domain www.myus.com
Redirect Base myus.com
Found Yes
Hash ebff6510bd37a4fc92e9ba4ebefbcc9c466a6f36f0d032d365c568d44dcbe58a
SimHash 2413fec1d1ff

Groups

*

Rule Path
Disallow /content/images/
Disallow /content/styles/images/
Disallow /?*Query=
Disallow /?Query=
Disallow /?*query=
Disallow /?query=
Disallow /*?tab=*
Disallow /?tab=*
Disallow /?ref=*
Disallow /*?ref=*
Disallow /?review-page*
Disallow /*?review-page*
Disallow /*?gclid=*
Disallow /?gclid=*
Disallow /?ReturnUrl=*
Disallow /benefits/myus-reviews/?*countryCode=*
Disallow /benefits/myus-reviews/?countryCode=*
Disallow /benefits/myus-reviews/?*countrycode=*
Disallow /benefits/myus-reviews/?countrycode=*
Disallow /benefits/shopping/deals-coupons-f2/
Disallow /pk/?hc_location=ufi*
Disallow /blog/?*page=*
Disallow /blog/?page=*
Disallow /about/news/?*page=*
Disallow /about/news/?page=*
Allow /welcome/amex/
Allow /welcome/mastercard/
Allow /welcome/visa/
Disallow /welcome/adslgate/
Disallow /welcome/aussie/
Disallow /welcome/bankaudi/
Disallow /welcome/burgan/
Disallow /welcome/dinersclub/
Disallow /welcome/friendsandfamily/
Disallow /welcome/jcb/
Disallow /welcome/jcb/jp/
Disallow /welcome/middle-east-ar/
Disallow /welcome/middle-east-en/
Disallow /welcome/landing/
Disallow /welcome/linkshare/
Disallow /welcome/paypal/
Disallow /welcome/middle-east/
Disallow /welcome/sem/
Disallow /welcome/ndani/
Disallow /welcome/icici/
Disallow /welcome/sign-up-with-myus-start-shipping/
Disallow /welcome/sign-up-with-myus-start-shipping-today/
Disallow /welcome/usunlocked/
Disallow /welcome/welcomeback/
Disallow /welcome/amexinvita/
Disallow /mastercard/ap/
Disallow /us/get-a-us-address-angola/
Disallow /us/get-a-us-address-angola-2/
Disallow /us/get-a-us-address-australia/
Disallow /us/get-a-us-address-canada/
Disallow /us/get-a-us-address-japan/
Disallow /us/get-a-us-address-kuwait/
Disallow /us/get-a-us-address-nigeria/
Disallow /us/get-a-us-address-saudi_arabia/
Disallow /us/get-a-us-address-south_africa/
Disallow /us/get-a-us-address-uae/
Disallow /us/get-a-us-address-uk/
Disallow /us/ship-from-us-to-japan-2/
Disallow /us/myus-angola/
Disallow /us/myus-australia/
Disallow /us/myus-canada/
Disallow /us/myus-india/
Disallow /us/myus-japan/
Disallow /us/ship-from-us-to-kuwait/
Disallow /us/ship-from-us-to-nigeria/
Disallow /us/shop-us-saudi_arabia/
Disallow /us/shop-us-south_africa/
Disallow /us/shop-us-uae/
Disallow /us/shop-us-uk/
Disallow /us/myus-australia-1/
Disallow /us/ab/
Disallow /us/ab-1/
Disallow /us/ship-from-us-to-china/
Disallow /us/get-a-us-address-india/
Disallow /myus-tutorial-faq/
Disallow /cgi-bin/
Disallow /en/
Disallow /signup/
Disallow /email/
Disallow /banner/
Disallow /_?
Disallow /_/

addsearchbot

Rule Path
Allow /faqs/

*
rogerbot
googlebot
googlebot-image
googlebot-video
googlebot-news
bingbot
msnbot
msnbot-media
bingpreview
slurp
duckduckbot
baiduspider
baiduspider-mobile
yandexbot
facebot
teoma
aolbuild
naverbot
applebot

No rules defined. All paths allowed.

Other Records

Field Value
sitemap https://www.myus.com/sitemap.xml

Comments

  • allow all crawlers
  • images
  • search
  • country page tabs
  • 10-23-2017 Update
  • Meeting On 6/7
  • Meeting On 6/21
  • member reviews by country
  • Meeting on 3/29
  • blog and news paging
  • 6524
  • Landing pages
  • allow - not fully supported, add entries to sitemap.xml
  • block the rest
  • 7230
  • block all country-specific landing pages
  • 7229
  • 3-15-2016 Meeting
  • sitemap - Supported by Google, Ask, Bing, Yahoo; defined on sitemaps.org
  • 8235
  • 2957
  • banners
  • Ajax requests
  • AddSearchBot

Warnings

  • 1 invalid line.