livefish.com.au
robots.txt

Robots Exclusion Standard data for livefish.com.au

Resource Scan

Scan Details

Site Domain livefish.com.au
Base Domain livefish.com.au
Scan Status Ok
Last Scan2025-10-21T07:28:29+00:00
Next Scan 2025-11-20T07:28:29+00:00

Last Scan

Scanned2025-10-21T07:28:29+00:00
URL https://livefish.com.au/robots.txt
Domain IPs 104.26.14.84, 104.26.15.84, 172.67.68.140, 2606:4700:20::681a:e54, 2606:4700:20::681a:f54, 2606:4700:20::ac43:448c
Response IP 172.67.68.140
Found Yes
Hash d75f103da52bbc25048d355307e73198b509caa9753c8c455d9ec967956b3a44
SimHash f51efb5323b0

Groups

*

Rule Path
Allow /freshwater/
Allow /marine/
Allow /plants/
Allow /accessories/
Allow /aquaponics/
Allow /specials/
Allow /static/
Allow /media/
Disallow /index.php/
Disallow /enable-cookies/
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /STATUS.txt
Disallow /get.php
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt
Disallow /*SID%3D
Disallow /*.cvs
Disallow /*.zip$
Disallow /*.svn$
Disallow /*.hg$
Disallow /*.git$
Disallow /*.idea$
Disallow /*.sql$
Disallow /*.tgz$
Disallow /*PHPSESSID
Disallow /*?SID=*
Disallow /*?___store=*
Disallow /*?*add-to-cart=*
Disallow /*?*dir=*
Disallow /*?*order=*
Disallow /*?*mode=*
Disallow /404/
Disallow /cgi-bin/
Disallow /errors/
Disallow /includes/
Disallow /magento/
Disallow /js/
Disallow /directory/
Disallow /scripts/
Disallow /shell/
Disallow /stats/
Disallow /lib/
Disallow /pkginfo/
Disallow /report/
Disallow /control/
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /phpserver/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/
Disallow /pub/
Disallow /media/captcha/
Disallow /media/css/
Disallow /media/css_secure/
Disallow /media/customer/
Disallow /media/downloadable/
Disallow /media/import/
Disallow /media/js/
Disallow /media/pdf/
Disallow /media/sales/
Disallow /media/tmp/
Disallow /media/wysiwyg/
Disallow /media/xmlconnect/
Disallow /wishlist/
Disallow /admin/
Disallow /claims/
Disallow /claim/
Disallow /sendfriend/
Disallow /newsletter/
Disallow /poll/
Disallow /review/
Disallow /review/product/
Disallow /productalert/
Disallow /question/render/
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /customize/
Disallow /catalogsearch/
Disallow /catalog/
Disallow /tag/
Disallow /w/
Disallow /where/
Disallow /*?*dir=
Disallow /*?*order=
Disallow /*?*mode=
Disallow /*?*limit=
Disallow /*%26dir%3D
Disallow /*%26order%3D
Disallow /*%26mode%3D
Disallow /*%26limit%3D
Disallow /*?*filter=
Disallow /*?*attribute
Disallow /*?*price=
Disallow /*?*color=
Disallow /*?*size=
Disallow /*?*brand=
Disallow /*?*species=
Disallow /*?*availability=
Disallow /*?*cat=
Disallow /*?*product_list_

chatgpt-user

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

claude-user

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

claude-searchbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

perplexitybot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

google-extended

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

semrushbot-sa

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

semrushbot-ba

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

ahrefsbot
ahrefssiteaudit
ahrefsbot-feed

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

guzzle

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

megaindex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bspider

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://livefish.com.au/sitemap.xml
sitemap https://livefish.com.au/sitemap.xml

Comments

  • Sort / view mode / page size
  • Layered navigation / filters (color/size/brand/price/species/etc.)
  • Ahrefs

Warnings

  • `host` is not a known field.