Log In Sign Up

builk.com
robots.txt

Robots Exclusion Standard data for builk.com

Resource Scan

Scan Details

Site Domain builk.com
Base Domain builk.com
Scan Status Ok
Last Scan2025-10-27T07:53:28+00:00
Next Scan 2025-11-26T07:53:28+00:00

Last Scan

Scanned2025-10-27T07:53:28+00:00
URL https://builk.com/robots.txt
Domain IPs 210.246.201.242
Response IP 210.246.201.242
Found Yes
Hash f9f6a1ea1273faa6df9c92155518c82e8e46bf31d4020ea4c8d97025553ee30e
SimHash 48b07850c515

Groups

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

firecrawlagent

Rule Path
Allow /

andibot

Rule Path
Allow /

exabot

Rule Path
Allow /

phindbot

Rule Path
Allow /

youbot

Rule Path
Allow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

*

Rule Path
Disallow /admin/
Disallow /internal/
Back to top

Comments

  • ===============================
  • Allow AI Search and Agent Bots
  • ===============================
  • =====================================
  • Disallow AI Training Data Collection
  • =====================================
  • =============================
  • Allow Traditional Search Bots
  • =============================
  • ==================================
  • General Rules – Disallow Admin Area
  • ==================================
Back to top