spatz.com.au
robots.txt

Robots Exclusion Standard data for spatz.com.au

Resource Scan

Scan Details

Site Domain spatz.com.au
Base Domain spatz.com.au
Scan Status Ok
Last Scan2026-01-24T00:23:35+00:00
Next Scan 2026-02-23T00:23:35+00:00

Last Scan

Scanned2026-01-24T00:23:35+00:00
URL https://spatz.com.au/robots.txt
Domain IPs 104.26.14.57, 104.26.15.57, 172.67.69.225, 2606:4700:20::681a:e39, 2606:4700:20::681a:f39, 2606:4700:20::ac43:45e1
Response IP 104.26.14.57
Found Yes
Hash ccc07d8c80205e77923ff93d7b051d97ad3596e0aadf1644a75dd0e85cc6e0d3
SimHash 600b5cc1ebad

Groups

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

googlebot-image

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

serpstatbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

bingpreview

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

applebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

facebookexternalhit

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

adsbot-google-mobile

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

semrushbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

bytespider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

wallabyupbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://www.spatz.com.au/sitemap_index.xml