stuff4beauty.com
robots.txt

Robots Exclusion Standard data for stuff4beauty.com

Resource Scan

Scan Details

Site Domain stuff4beauty.com
Base Domain stuff4beauty.com
Scan Status Ok
Last Scan2024-09-08T09:06:28+00:00
Next Scan 2024-10-08T09:06:28+00:00

Last Scan

Scanned2024-09-08T09:06:28+00:00
URL https://stuff4beauty.com/robots.txt
Redirect http://www.stuff4beauty.com/robots.txt
Redirect Domain www.stuff4beauty.com
Redirect Base stuff4beauty.com
Domain IPs 162.214.68.107
Redirect IPs 162.214.68.107
Response IP 162.214.68.107
Found Yes
Hash 95f9bb86ddff6b073ecc1c8428d8a53a5a8ff103aa126a5ec3af07a42fc0021d
SimHash 8d5dd52bcfb0

Groups

ia_archiver

Rule Path
Disallow /

xombot

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

*

Rule Path
Disallow /search/
Disallow /sara/
Disallow /barb/
Disallow /logs/
Disallow /scripts/
Disallow /secure/
Disallow /shared/
Disallow /shopadmin/
Disallow /sls/
Disallow /test/
Disallow /user/
Disallow /home/
Disallow /HDWFormCaptcha/
Disallow /cgi-bin/

Other Records

Field Value
sitemap https://www.stuff4beauty.com/sitemap.gz

Comments

  • Disallow: /SpryAssets/