wallmart.com
robots.txt

Robots Exclusion Standard data for wallmart.com

Resource Scan

Scan Details

Site Domain wallmart.com
Base Domain wallmart.com
Scan Status Ok
Last Scan2024-06-17T09:27:29+00:00
Next Scan 2024-07-17T09:27:29+00:00

Last Scan

Scanned2024-06-17T09:27:29+00:00
URL http://www.wallmart.com/robots.txt
Redirect https://www.walmart.com/robots.txt
Redirect Domain www.walmart.com
Redirect Base walmart.com
Domain IPs 125.56.219.50, 72.247.127.240
Redirect IPs 23.53.216.168
Response IP 23.75.213.236
Found Yes
Hash b1e06f34c36d58a709787ceef6a4e4b48f53ac6a9489a26f02f9129a39c56675
SimHash 6430d2e0cc9a

Groups

*

Rule Path
Disallow /0/
Disallow /55875582/walmart-us/catalog/
Disallow /account/
Disallow /api/
Disallow /collection/api/logger
Disallow /cp/-201
Disallow /cp/-302
Disallow /cp/-306
Disallow /cp/-309
Disallow /cp/-506
Disallow /cp/-509
Disallow /cp/api/logger
Disallow /cp/api/wpa
Disallow /cservice/
Disallow /cservice/ya_index.gsp
Disallow /electrode/api/logger
Disallow /email_collect/
Disallow /giftregistry/
Disallow /msp
Disallow /nonConfig/api/wpa
Disallow /popup_security.jsp
Disallow /product/electrode/api/logger
Disallow /product/electrode/api/wpa
Allow /reviews/product/
Allow /reviews/seller/
Disallow /search
Disallow /search/api/wpa
Disallow */api/wpa
Disallow */midas/*
Disallow */wrd.walmart.com/*
Disallow wrd.walmart.com/
Disallow /search/search-ng.do
Disallow /solutions/
Disallow /store/ajax/detail-navigation
Disallow /store/ajax/local-store
Disallow /store/ajax/preferred
Disallow /store/ajax/search
Disallow /store/ajax/visited-store
Disallow /store/category/
Disallow /store/electrode/api/fetch-coupons
Disallow /store/electrode/api/logger
Disallow /store/electrode/api/p13n
Disallow /store/electrode/api/search
Disallow /store/electrode/api/stores
Allow /store/finder
Disallow /store/popular_in_grade/
Disallow /storeLocator/
Disallow /storeLocator/ca_storefinder_results.do
Disallow /tealeaf
Disallow /topic/electrode/api/logger
Disallow /topic/electrode/api/wpa
Disallow /typeahead/
Disallow /wmflows/
Disallow */undefined/*
Disallow /store/*/search
Disallow /c/kp/*/page/*
Disallow /feeds/*
Disallow /orders
Disallow /es/
Disallow /swag/*
Disallow /dad/trk

adsbot-google
mediapartners-google
slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.walmart.com/sitemap_category.xml
sitemap https://www.walmart.com/sitemap_store_main.xml
sitemap https://www.walmart.com/help/sitemap_gm.xml
sitemap https://www.walmart.com/sitemap_browse_fst.xml
sitemap https://www.walmart.com/sitemap_store_dept.xml
sitemap https://www.walmart.com/sitemap_tp_legacy.xml
sitemap https://www.walmart.com/sitemap_tp_at.xml
sitemap https://www.walmart.com/sitemap_tp_br.xml
sitemap https://www.walmart.com/sitemap_tp_kp.xml
sitemap https://www.walmart.com/sitemap_brp_02.xml
sitemap https://www.walmart.com/sitemap_brp_03.xml
sitemap https://www.walmart.com/sitemap_brp_04.xml
sitemap https://www.walmart.com/sitemap_brp_05.xml
sitemap https://www.walmart.com/sitemap_brp_06.xml
sitemap https://www.walmart.com/sitemap_brp_07.xml
sitemap https://www.walmart.com/sitemap_brp_08.xml
sitemap https://www.walmart.com/sitemap_brp_09.xml
sitemap https://www.walmart.com/sitemap_itp_01.xml
sitemap https://www.walmart.com/sitemap_itp_02.xml
sitemap https://www.walmart.com/sitemap_itp_03.xml
sitemap https://www.walmart.com/sitemap_auto_dir.xml
sitemap https://www.walmart.com/sitemap_brndshp_brand.xml
sitemap https://www.walmart.com/sitemap_sp.xml
sitemap https://www.walmart.com/sitemap_hi_ip.xml
sitemap https://www.walmart.com/sitemap_hi_tp.xml
sitemap https://www.walmart.com/sitemap_hi_bp.xml
sitemap https://www.walmart.com/sitemap_hi_mp.xml

Comments

  • Sitemaps-https
  • Disallow select URLs
  • Crawler specific settings
  • slow down Yahoo