aldi.us
robots.txt

Robots Exclusion Standard data for aldi.us

Resource Scan

Scan Details

Site Domain aldi.us
Base Domain aldi.us
Scan Status Ok
Last Scan2024-04-18T12:26:41+00:00
Next Scan 2024-05-18T12:26:41+00:00

Last Scan

Scanned2024-04-18T12:26:41+00:00
URL https://aldi.us/robots.txt
Redirect https://www.aldi.us/robots.txt
Redirect Domain www.aldi.us
Redirect Base aldi.us
Domain IPs 109.235.139.138
Redirect IPs 108.139.243.120, 108.139.243.125, 108.139.243.23, 108.139.243.99
Response IP 18.165.171.96
Found Yes
Hash 9c5c0b2b9a20a238d7d726b87075370baab4e335efde02c2cf4e650ceeb02cbf
SimHash 695dca15eb51

Groups

mj12bot

Rule Path
Disallow /

googlebot
searchmetricsbot
slurp
bingbot
teoma

Rule Path
Disallow /*/template.html$
Disallow /*/contenttemplate.html$

*

Rule Path
Disallow /stores/

Other Records

Field Value
sitemap https://www.aldi.us/en/sitemap.xml
sitemap https://www.aldi.us/en/sitemap-index.xml
sitemap https://corporate.aldi.us/en/sitemap.xml
sitemap https://corporate.aldi.us/en/sitemap-index.xml