biberk.com
robots.txt

Robots Exclusion Standard data for biberk.com

Resource Scan

Scan Details

Site Domain biberk.com
Base Domain biberk.com
Scan Status Ok
Last Scan2025-11-29T22:29:04+00:00
Next Scan 2025-12-29T22:29:04+00:00

Last Scan

Scanned2025-11-29T22:29:04+00:00
URL https://biberk.com/robots.txt
Redirect https://www.biberk.com/robots.txt
Redirect Domain www.biberk.com
Redirect Base biberk.com
Domain IPs 20.72.136.132
Redirect IPs 20.72.136.132
Response IP 20.72.136.132
Found Yes
Hash d8e8dd7ef9d4f9e4905a769bc0399e3a303723eafba6edd1c1cad6dba958360a
SimHash 69949c40d59c

Groups

*

Rule Path
Disallow /BSP/RouteOverrides
Disallow /landing
Disallow /partners
Disallow /templates/
Disallow /umbraco

gptbot

Rule Path
Disallow

google-extended

Rule Path
Disallow

claudebot

Rule Path
Disallow

perplexitybot

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.biberk.com/sitemap

Comments

  • Allow GPTBot (OpenAI)
  • Allow Google-Extended (Google AI services like Bard/Gemini)
  • Allow Anthropic's ClaudeBot
  • Allow PerplexityBot