katescloset.com.au
robots.txt

Robots Exclusion Standard data for katescloset.com.au

Resource Scan

Scan Details

Site Domain katescloset.com.au
Base Domain katescloset.com.au
Scan Status Ok
Last Scan2024-05-27T20:25:46+00:00
Next Scan 2024-06-03T20:25:46+00:00

Last Scan

Scanned2024-05-27T20:25:46+00:00
URL https://katescloset.com.au/robots.txt
Redirect https://www.katescloset.com.au/robots.txt
Redirect Domain www.katescloset.com.au
Redirect Base katescloset.com.au
Domain IPs 199.34.228.45
Redirect IPs 199.34.228.45
Response IP 199.34.228.45
Found Yes
Hash 8a14119a498178b130a777fae2a9c0925e823807c5508dbc9c700f222f896dcf
SimHash c05c5dee6880

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /adidas-x-stella-mccartney-barricade-skort.html
Disallow /https%3A//www.katescloset.com.au/style-file.html
Disallow /style-file-2022.html
Disallow /style-file-2021.html
Disallow /style-file.html
Disallow /style-file-2020.html
Disallow /style-file-2019.html
Disallow /style-file-2018.html
Disallow /boden-kristen-pleated-skirt-in-maroon.html
Disallow /style-file-2017.html
Disallow /colour-pages.html
Disallow /style-file-2016.html
Disallow /style-file-2015.html
Disallow /edeline-lee-plait-skirt-in-burgundy.html
Disallow /erdem-ottoman-jacquard-flared-midi-skirt-in-navy.html
Disallow /privacy-policy.html
Disallow /template---skirts.html
Disallow /really-wild-flippy-suede-skirt-in-nutmeg.html
Disallow /gabriela-hearst-costa-turtleneck-sweater-in-windsor-wine.html
Disallow /sezane-naelle-skirt-in-natural.html
Disallow /zara-houndstooth-tweed-midi-skirt.html
Disallow /cornelia-james-maud-sueded-cotton-glove-in-parchment.html
Disallow /zara-black-velvet-headband.html
Disallow /missoma-gold-small-molten-hoops.html

Other Records

Field Value
sitemap https://www.katescloset.com.au/sitemap.xml