walmart.ca
robots.txt

Robots Exclusion Standard data for walmart.ca

Resource Scan

Scan Details

Site Domain walmart.ca
Base Domain walmart.ca
Scan Status Ok
Last Scan2025-03-12T21:22:30+00:00
Next Scan 2025-03-19T21:22:30+00:00

Last Scan

Scanned2025-03-12T21:22:30+00:00
URL https://walmart.ca/robots.txt
Redirect https://www.walmart.ca/robots.txt
Redirect Domain www.walmart.ca
Redirect Base walmart.ca
Domain IPs 23.15.99.176
Redirect IPs 23.210.106.171
Response IP 23.210.106.171
Found Yes
Hash a8032dee4068828bd56aabbcf6d3df38e4f21822948585e69644f3ae7a05589f
SimHash 66965d0a4725

Groups

*

Rule Path
Disallow /en/*%2B*
Disallow /fr/*%2B*
Disallow /ws/*
Disallow /webjars/*
Disallow /css-pattern-library
Disallow /*/app$
Disallow /*/global$
Disallow /cart
Disallow /panier
Disallow /search/*
Disallow /cherche/*
Disallow /search?*
Disallow /recherche?*
Disallow /en/search
Disallow /search?*z=1
Disallow /en/search?*z=1
Disallow /en/kiosk/*
Disallow /fr/kiosk/*
Disallow /*/associates
Disallow /*?f=*
Disallow /ce/*?c=*
Disallow /cf/*?c=*
Disallow /ouvrir-une-session*
Disallow /sign-in*
Disallow /en/ip/*/*null$
Disallow /fr/ip/*/*null$
Disallow /en/ip/*
Allow /en/ip/*/*
Disallow /fr/ip/*
Allow /fr/ip/*/*
Disallow /fr/en/ip/*
Disallow /en/*%26b%3D1*
Disallow /en/*?facet=*&b=1*
Disallow /fr/*%26b%3D1*
Disallow /fr/*?facet=*&b=1*
Disallow /en/search?*x=1
Disallow /fr/search?*x=1
Disallow /en/email/gifts-voucher
Disallow /en/financial-services/walmart-rewards-mastercard-apply-now
Disallow /fr/services-financiers/recompenses-walmart-mastercard-faire-une-demande

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.walmart.ca/sitemap-categories.xml
sitemap https://www.walmart.ca/sitemap-brand-1p-en.xml
sitemap https://www.walmart.ca/sitemap-brand-3p-en.xml
sitemap https://www.walmart.ca/sitemap-brand-1p-fr.xml
sitemap https://www.walmart.ca/sitemap-brand-3p-fr.xml
sitemap https://www.walmart.ca/sitemap-brands-legacy.xml
sitemap https://www.walmart.ca/sitemap-collections.xml
sitemap https://www.walmart.ca/sitemap-collections-ca.xml
sitemap https://www.walmart.ca/sitemap-product-1p-en.xml
sitemap https://www.walmart.ca/sitemap-product-3p-head-en.xml
sitemap https://www.walmart.ca/sitemap-product-3p-en.xml
sitemap https://www.walmart.ca/sitemap-product-1p-fr.xml
sitemap https://www.walmart.ca/sitemap-product-3p-head-fr.xml
sitemap https://www.walmart.ca/sitemap-product-3p-fr.xml
sitemap https://www.walmart.ca/sitemap-topic.xml
sitemap https://www.walmart.ca/sitemap-brand-category.xml
sitemap https://www.walmart.ca/sitemap-cp.xml

Comments

  • Prevent refined browse pages from being crawled, avoiding millions of near-duplicate entries. MG IG
  • Now allowing assets to be crawled. MG
  • Disallow: /assets/*
  • Prevents Financial Services One and Done page from being hit. 11-16-2016 NV
  • Prevents Financial WM MC page from being hit, contains promocodes. 11-16-2016 NV
  • Rate limiting for Bingbots
  • Adding new rules
  • Ending of robots.txt