Models: the Handler and the LLM Registry

Two facts shape every model call an extension makes:

1. The Handler is the default. The Handler is bRRAIn's own fine-tuned model, served locally on the brain pod. It is not a sidecar, container or proxy you deploy.
2. Commercial models are the organization's opt-in. Claude, GPT, Gemini and others are available only when an administrator registers them in the organization's LLM Registry. Your code must never make a commercial model the default on its own.

All model calls go through c.LLM(), so the organization's registry, allowlist and policies apply. Credentials never leave the platform.

Discover, then choose

LLM().Models(ctx) returns the organization's registry entries (ID, Label, Provider, Model, Default, Health, ...) with every secret redacted. An empty list is a normal state: it means "use the Handler".

// HandlerProviderID routes a call straight to the organization's Handler.
const HandlerProviderID = "brrain-handler"

// PickModel prefers the organization's default registry entry, else the
// Handler. It never picks a commercial model on its own.
func PickModel(ctx context.Context, c *platformsdk.Client) (string, error) {
	models, err := c.LLM().Models(ctx)
	if err != nil {
		return "", err
	}
	for _, m := range models {
		if m.Default {
			return m.ID, nil
		}
	}
	return HandlerProviderID, nil
}

Never pick "the first entry". When a feature needs a capability only some entries have, such as image input, let an administrator choose the entry in your extension's settings and store that ID. Re-discover periodically; administrators rename and re-default entries.

Calling a model

LLMCall supports two shapes. Pick one per call; if both are set, Messages wins.

func Summarize(ctx context.Context, c *platformsdk.Client, providerID, notes string) (string, error) {
	out, err := c.LLM().Call(ctx, platformsdk.LLMCall{
		ProviderID:  providerID,
		System:      "Summarize meeting notes as decisions, actions and open questions. Use only the notes.",
		Prompt:      notes,
		Temperature: 0.2,
		MaxTokens:   600,
	})
	if err != nil {
		return "", err
	}
	return ReplyText(out)
}

func FollowUp(ctx context.Context, c *platformsdk.Client, providerID, notes, firstAnswer, question string) (string, error) {
	out, err := c.LLM().Call(ctx, platformsdk.LLMCall{
		ProviderID: providerID,
		Messages: []platformsdk.LLMMessage{
			{Role: "system", Content: "Answer only from the supplied notes."},
			{Role: "user", Content: notes},
			{Role: "assistant", Content: firstAnswer},
			{Role: "user", Content: question},
		},
	})
	if err != nil {
		return "", err
	}
	return ReplyText(out)
}

Extracting the reply

Call returns map[string]any so the wire shape is not pinned to one provider. Handler calls put the text in reply; relayed calls return the provider's parsed response. Read the first non-empty of reply, response, text and content, and treat "no text" as an error:

// ReplyText pulls the answer text out of a Call result.
func ReplyText(out map[string]any) (string, error) {
	for _, k := range []string{"reply", "response", "text", "content"} {
		if s, ok := out[k].(string); ok && strings.TrimSpace(s) != "" {
			return s, nil
		}
	}
	return "", errors.New("llm: response carried no reply text")
}

Images and documents

LLMCall.Images and LLMCall.Documents (or the same fields on an LLMMessage) attach base64-encoded files for vision- and document-capable models. The platform converts them to each provider's native shape; a provider that cannot accept an attachment returns an error.

// DescribeImage sends a vault image to a vision-capable registry entry
// an administrator chose for this feature.
func DescribeImage(ctx context.Context, c *platformsdk.Client, providerID, path string) (string, error) {
	img, err := c.Vault().ReadBytes(ctx, path)
	if err != nil {
		return "", err
	}
	out, err := c.LLM().Call(ctx, platformsdk.LLMCall{
		ProviderID: providerID,
		Prompt:     "Describe this site photo in two sentences.",
		Images: []platformsdk.LLMImage{{
			MediaType: "image/jpeg",
			Data:      base64.StdEncoding.EncodeToString(img),
		}},
	})
	if err != nil {
		return "", err
	}
	return ReplyText(out)
}

Sending a file to a model moves its content out of the vault into that model's context. Check policy first and record the call in the audit chain (path and model ID, never the content).

Failure modes

| What you see | Cause | Response | | --- | --- | --- | | APIError 502 mentioning the Handler | The Handler is starting or its model is not loaded | Tell the user; retry later with backoff; do not silently switch to a commercial model | | APIError 502, other body | The registry relay or the provider failed | Retry read-only features with backoff; surface persistent failures | | APIError 501 | The pod has no model backend wired | Configuration problem; report it | | context deadline exceeded (Client.Timeout ...) | Your transport timeout is shorter than the answer time | Raise Options.HTTPClient's timeout; use per-call context deadlines | | A 200 whose reply says the registry relay is not wired | You dispatched a registry ID on a pod without a configured relay | Check Models first; on an empty list use the Handler |

Registering models

c.LLMRegistry() (Register, Unregister) writes registry entries. It is for extensions that provision models, such as one that registers a deployment when it becomes ready. Discovery is always LLM().Models. Most extensions never need LLMRegistry.

Next

• Examples: a grounded answer built from vault search, a model call and an audit event.
• Error handling