justsunnies.com.au
robots.txt

Robots Exclusion Standard data for justsunnies.com.au

Resource Scan

Scan Details

Site Domain justsunnies.com.au
Base Domain justsunnies.com.au
Scan Status Ok
Last Scan2024-09-16T11:57:53+00:00
Next Scan 2024-10-16T11:57:53+00:00

Last Scan

Scanned2024-09-16T11:57:53+00:00
URL https://justsunnies.com.au/robots.txt
Redirect https://www.justsunnies.com.au/robots.txt
Redirect Domain www.justsunnies.com.au
Redirect Base justsunnies.com.au
Domain IPs 76.76.21.164, 76.76.21.98
Redirect IPs 76.76.21.9, 76.76.21.98
Response IP 76.76.21.98
Found Yes
Hash 65b3921a46f6dbf5fc95cfffab9f2024a566a5e7d9417b0f3218029bbdfbad97
SimHash 6a00ddfb6a3b

Groups

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /api/*
Disallow /admin/*
Disallow /search/
Disallow /*?sort=*
Disallow /*?resultsPerPage=*
Disallow /*?filter*
Disallow /*?gender*
Disallow /*price-min*
Disallow /*price-max*
Disallow /*style%3D*
Disallow /*lens-type*
Disallow /*show%3D*
Disallow /*?results=*
Disallow /*view%3D*
Disallow /*?_ke=ey*
Disallow /*?group=*
Disallow /*?desktopmode*
Disallow /*?yoReviews*
Disallow /*?view-colours*

Other Records

Field Value
sitemap https://www.justsunnies.com.au/sitemap.xml