beauty-10.com
robots.txt
Robots Exclusion Standard data for beauty-10.com
Resource Scan
Scan Details
Site Domain | beauty-10.com |
Base Domain | beauty-10.com |
Scan Status | Ok |
Last Scan | 2024-09-11T21:24:45+00:00 |
Next Scan | 2024-10-11T21:24:45+00:00 |
Last Scan
Scanned | 2024-09-11T21:24:45+00:00 |
URL | https://beauty-10.com/robots.txt |
Domain IPs | 104.21.87.114, 172.67.143.5, 2606:4700:3031::ac43:8f05, 2606:4700:3033::6815:5772 |
Response IP | 104.21.87.114 |
Found | Yes |
Hash | f4c8dd14d9009b6b9f2c3874292a6f6f9a87988cf057e575b52a67ae14b23327 |
SimHash | 63476f7109a7 |
Groups
*
Rule | Path |
---|---|
Allow | / |
Disallow | /admin |
Disallow | /download |
Disallow | /export |
Disallow | /system |
Disallow | /index.php?route=information%2Fcontact%2Fcaptcha |
Disallow | /index.php?route=checkout%2Fcart |
Disallow | /index.php?route=checkout%2Fshipping |
Disallow | /index.php?route=account |
Disallow | /index.php?route=account%2Faccount |
Disallow | /index.php?route=account%2Flogin |
Other Records
Field | Value |
---|---|
crawl-delay | 20 |
Other Records
Field | Value |
---|---|
sitemap | /sitemap.xml |
Warnings
- 20 invalid lines.