bleen.com.au
robots.txt

Robots Exclusion Standard data for bleen.com.au

Resource Scan

Scan Details

Site Domain bleen.com.au
Base Domain bleen.com.au
Scan Status Ok
Last Scan2025-12-01T04:22:25+00:00
Next Scan 2025-12-08T04:22:25+00:00

Last Scan

Scanned2025-12-01T04:22:25+00:00
URL https://bleen.com.au/robots.txt
Domain IPs 104.21.21.244, 172.67.201.113, 2606:4700:3030::6815:15f4, 2606:4700:3035::ac43:c971
Response IP 104.21.21.244
Found Yes
Hash eb191969b5e6bbc78bc0a8055fc3d771292db9304031139bca8c5d11b0e74812
SimHash 68022954c093

Groups

*

Rule Path
Disallow /admin/
Disallow /images/
Disallow /img/
Disallow /css/
Disallow /js/
Disallow /plugins/
Allow /images/*.jpg
Allow /img/*.png
Allow /css/*.css
Allow /js/*.css
Allow /js/*.js
Allow /plugins/*.css

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://bleen.com.au/sitemap.xml