wal-mart.com
robots.txt

Robots Exclusion Standard data for wal-mart.com

Resource Scan

Scan Details

Site Domain wal-mart.com
Base Domain wal-mart.com
Scan Status Ok
Last Scan2024-11-13T10:30:15+00:00
Next Scan 2024-11-20T10:30:15+00:00

Last Scan

Scanned2024-11-13T10:30:15+00:00
URL https://www.wal-mart.com/robots.txt
Domain IPs 23.73.13.192
Response IP 23.207.181.199
Found Yes
Hash 1f2a6af29490c928ef3a5e013a9a1bdceba1133fba49643ea2cf3675e87a28b5
SimHash 6434d2e0ecb8

Groups

*

Rule Path
Disallow /0/
Disallow /55875582/walmart-us/catalog/
Disallow /account/
Disallow /api/
Disallow /collection/api/logger
Disallow /cp/-201
Disallow /cp/-302
Disallow /cp/-306
Disallow /cp/-309
Disallow /cp/-506
Disallow /cp/-509
Disallow /cp/api/logger
Disallow /cp/api/wpa
Disallow /cservice/
Disallow /cservice/ya_index.gsp
Disallow /electrode/api/logger
Disallow /email_collect/
Disallow /giftregistry/
Disallow /msp
Disallow /nonConfig/api/wpa
Disallow /popup_security.jsp
Disallow /product/electrode/api/logger
Disallow /product/electrode/api/wpa
Allow /reviews/product/
Allow /reviews/seller/
Disallow /search
Disallow /search/api/wpa
Disallow */api/wpa
Disallow */midas/*
Disallow /search/search-ng.do
Disallow /solutions/
Disallow /store/ajax/detail-navigation
Disallow /store/ajax/local-store
Disallow /store/ajax/preferred
Disallow /store/ajax/search
Disallow /store/ajax/visited-store
Disallow /store/category/
Disallow /store/electrode/api/fetch-coupons
Disallow /store/electrode/api/logger
Disallow /store/electrode/api/p13n
Disallow /store/electrode/api/search
Disallow /store/electrode/api/stores
Allow /store/finder
Disallow /store/popular_in_grade/
Disallow /storeLocator/
Disallow /storeLocator/ca_storefinder_results.do
Disallow /tealeaf
Disallow /topic/electrode/api/logger
Disallow /topic/electrode/api/wpa
Disallow /typeahead/
Disallow /wmflows/
Disallow */undefined/*
Disallow /store/*/search
Disallow /c/kp/*/page/*
Disallow /feeds/*
Disallow /orders
Disallow /es/
Disallow /swag/*
Disallow /dad/trk

adsbot-google
mediapartners-google
slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.walmart.com/sitemap_category.xml
sitemap https://www.walmart.com/sitemap_store_main.xml
sitemap https://www.walmart.com/help/sitemap_gm.xml
sitemap https://www.walmart.com/sitemap_browse_fst.xml
sitemap https://www.walmart.com/sitemap_store_dept.xml
sitemap https://www.walmart.com/sitemap_tp_legacy.xml
sitemap https://www.walmart.com/sitemap_tp_at.xml
sitemap https://www.walmart.com/sitemap_tp_br.xml
sitemap https://www.walmart.com/sitemap_tp_kp.xml
sitemap https://www.walmart.com/sitemap_brp_02.xml
sitemap https://www.walmart.com/sitemap_brp_03.xml
sitemap https://www.walmart.com/sitemap_brp_04.xml
sitemap https://www.walmart.com/sitemap_brp_05.xml
sitemap https://www.walmart.com/sitemap_brp_06.xml
sitemap https://www.walmart.com/sitemap_brp_07.xml
sitemap https://www.walmart.com/sitemap_brp_08.xml
sitemap https://www.walmart.com/sitemap_brp_09.xml
sitemap https://www.walmart.com/sitemap_itp_01.xml
sitemap https://www.walmart.com/sitemap_itp_02.xml
sitemap https://www.walmart.com/sitemap_itp_03.xml
sitemap https://www.walmart.com/sitemap_auto_dir.xml
sitemap https://www.walmart.com/sitemap_brndshp_brand.xml
sitemap https://www.walmart.com/sitemap_sp.xml
sitemap https://www.walmart.com/sitemap_hi_ip.xml
sitemap https://www.walmart.com/sitemap_hi_tp.xml
sitemap https://www.walmart.com/sitemap_hi_bp.xml
sitemap https://www.walmart.com/sitemap_hi_mp.xml

Comments

  • Sitemaps-https
  • Disallow select URLs
  • Crawler specific settings
  • slow down Yahoo