Log In Sign Up

cibis.com.au
robots.txt

Robots Exclusion Standard data for cibis.com.au

Resource Scan

Scan Details

Site Domain cibis.com.au
Base Domain cibis.com.au
Scan Status Ok
Last Scan2025-12-17T09:42:55+00:00
Next Scan 2025-12-31T09:42:55+00:00

Last Scan

Scanned2025-12-17T09:42:55+00:00
URL https://cibis.com.au/robots.txt
Redirect https://www.cibis.com.au/robots.txt
Redirect Domain www.cibis.com.au
Redirect Base cibis.com.au
Domain IPs 16.176.88.121, 52.64.191.92
Redirect IPs 16.176.88.121, 52.64.191.92
Response IP 52.64.191.92
Found Yes
Hash 860d53817ef38eb5df215a2e0c6a435b8846344d5cfd4d02f57f2532f78c26d7
SimHash 59988b02c674

Groups

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-user

Rule Path
Disallow /

claude-searchbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

google-cloudvertexbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

tiktokspider

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

perplexity‑user

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

youbot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

ai2bot-dolma

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

*

Rule Path
Allow /
Back to top

Comments

  • Block OpenAI bots
  • Block Anthropic bots
  • Block Google AI bots
  • Block Meta/Facebook bots
  • Block Amazon bots
  • Block TikTok
  • Block Perplexity
  • Block Apple AI
  • Block other known AI crawlers
  • Allow all other bots
Back to top