quokk.au
robots.txt

Robots Exclusion Standard data for quokk.au

Resource Scan

Scan Details

Site Domain quokk.au
Base Domain quokk.au
Scan Status Ok
Last Scan2025-11-19T16:47:58+00:00
Next Scan 2025-11-20T16:47:58+00:00

Last Scan

Scanned2025-11-19T16:47:58+00:00
URL https://quokk.au/robots.txt
Domain IPs 82.180.161.92
Response IP 82.180.161.92
Found Yes
Hash b8d221b9d3fdb7fe2b56b5d9f5ddfd63455866e6a6ba3b1a6fce81f3a5a24f36
SimHash 4607d962ea48

Groups

*

Rule Path
Disallow /community/add_local
Disallow /community/add_remote
Disallow /auth/register
Disallow /auth/reset_password_request
Disallow /auth/login
Disallow /admin
Disallow /inbox
Disallow /search/
Disallow /modlog
Disallow /u/
Disallow /d/
Disallow /static/media/users/
Disallow /post/*/options

gptbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

imagesiftbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

velenpublicwebcrawler

Rule Path
Disallow /

baiduspider
yisouspider
petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

serankingbacklinksbot

Rule Path
Disallow /

perplexitybot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

meta-externalagent
meta-externalagent
meta-externalfetcher
meta-externalfetcher
meta-webindexer
tiktokspider
duckassistbot
anthropic-ai

Rule Path
Disallow /

Warnings

  • 1 invalid line.
  • `license` is not a known field.