98 lines
5.2 KiB
Markdown
98 lines
5.2 KiB
Markdown
# Providers and models
|
|
|
|
## Model keys
|
|
|
|
Models are addressed as `provider/model-id`, for example `anthropic/claude-opus-5`, `openai/gpt-5`,
|
|
`claude-code/sonnet`, `openrouter/deepseek/deepseek-chat`. `AgentSession.SetModelAsync("provider/id")`,
|
|
`AgenticCoreOptions.DefaultModel` and `AgentSessionOptions.Model` all take this form.
|
|
|
|
`Model.Provider` is the account/auth scope; `Model.Api` is the wire protocol implementation
|
|
(`anthropic-messages`, `openai-completions`, `claude-code-headless`, ...). Many providers share one API with a
|
|
different base URL.
|
|
|
|
## Authentication
|
|
|
|
For each provider the registry resolves an API key in this order:
|
|
|
|
1. The registration's `ApiKey` spec: a literal, `$ENV_VAR` / `${ENV_VAR}`, or `!command` (stdout is the key,
|
|
cached per process). `$$` and `$!` escape a literal `$`.
|
|
2. `~/.volcanic/agent/auth.json`: `{ "anthropic": { "apiKey": "sk-..." }, "openai": { "env": "MY_OPENAI_VAR" } }`
|
|
3. The `<PROVIDER>_API_KEY` environment variable (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`).
|
|
|
|
Providers with `RequiresApiKey = false` (Claude Code, Ollama) skip this. `IModelRegistry.IsAvailable(model)`
|
|
tells a UI whether a model can be selected.
|
|
|
|
## Built-in provider extensions
|
|
|
|
| Extension | Provider id | API | Notes |
|
|
|---|---|---|---|
|
|
| `AnthropicProviderExtension` | `anthropic` | `anthropic-messages` | Thinking budgets from the thinking level, prompt caching on system prompt and last user block, images, tools |
|
|
| `OpenAIProviderExtension` | `openai` (configurable) | `openai-completions` | `OpenAIProviderOptions` sets base URL, provider id, models, header names; helpers `OpenAIProviderOptions.OpenRouter(models)` and `Ollama(models)` |
|
|
| `ClaudeCodeProviderExtension` | `claude-code` | `claude-code-headless` | Uses the `claude` CLI on a subscription; see [claude-code-provider.md](claude-code-provider.md) |
|
|
|
|
Add the same extension twice with different options to serve several OpenAI-compatible endpoints:
|
|
|
|
```csharp
|
|
o.AddExtension(new OpenAIProviderExtension()); // openai/*
|
|
o.AddExtension(new OpenAIProviderExtension(OpenAIProviderOptions.OpenRouter([
|
|
new Model { Id = "deepseek/deepseek-chat", Name = "DeepSeek", Provider = "openrouter", Api = "openai-completions", ContextWindow = 128_000, MaxTokens = 8_192 },
|
|
])));
|
|
o.AddExtension(new OpenAIProviderExtension(OpenAIProviderOptions.Ollama([
|
|
new Model { Id = "qwen2.5-coder", Name = "Qwen Coder", Provider = "ollama", Api = "openai-completions", ContextWindow = 32_000, MaxTokens = 4_096 },
|
|
])));
|
|
```
|
|
|
|
## Writing a provider
|
|
|
|
A provider is a `ProviderRegistration` with a `Stream` delegate:
|
|
|
|
```csharp
|
|
api.RegisterProvider(new ProviderRegistration
|
|
{
|
|
Id = "my-provider",
|
|
Name = "My Provider",
|
|
Api = "my-api",
|
|
BaseUrl = "https://llm.example.com",
|
|
ApiKey = "$MY_PROVIDER_API_KEY",
|
|
Models = [new Model { Id = "fast", Name = "Fast", Provider = "my-provider", Api = "my-api", ContextWindow = 128_000, MaxTokens = 8_192 }],
|
|
Stream = StreamAsync,
|
|
});
|
|
|
|
static async IAsyncEnumerable<AssistantMessageEvent> StreamAsync(Model model, LlmContext context, StreamOptions options, [EnumeratorCancellation] CancellationToken ct)
|
|
{
|
|
var builder = new AssistantMessageBuilder(model);
|
|
yield return builder.Start();
|
|
// ... call your API, then for each delta:
|
|
yield return builder.AppendText("Hello");
|
|
// tool calls:
|
|
yield return builder.StartToolCall("call_1", "bash");
|
|
yield return builder.AppendToolCallDelta("{\"command\":\"ls\"}");
|
|
yield return builder.EndToolCall();
|
|
builder.SetUsage(new Usage(Input: 10, Output: 5, CacheRead: 0, CacheWrite: 0));
|
|
yield return builder.Done(StopReason.Stop); // becomes ToolUse automatically when tool calls exist
|
|
}
|
|
```
|
|
|
|
Contract:
|
|
|
|
- Emit `Start`, balanced block events, then exactly one `Done` or `Error`. `AssistantMessageBuilder` does the
|
|
bookkeeping and produces the cumulative `Partial` on every event.
|
|
- Never throw. Encode failures with `builder.Fail(message)`; cancellation with `builder.Fail("aborted", aborted: true)`.
|
|
The registry converts stray exceptions and missing terminal events into `Error` events anyway.
|
|
- Honour `StreamOptions`: `ApiKey` (already resolved), `MaxTokens`, `Temperature`, `Reasoning` (a
|
|
`ThinkingLevel`; map it through `model.ThinkingLevelMap`), `CacheRetention`, `SessionId` (cache affinity),
|
|
`Headers`, and the hooks `OnPayload` (replace the payload), `OnResponse` (status and headers),
|
|
`OnProviderStreamEvent` (raw events). The hooks are what power the `before_provider_request` extension event.
|
|
- The registry normalises the transcript before calling you: failed assistant messages are dropped, missing
|
|
tool results are synthesised, custom messages become user messages, thinking from other models becomes text,
|
|
and images are replaced for text-only models. You still receive `UserMessage`, `AssistantMessage` and
|
|
`ToolResultMessage` and must map all three.
|
|
- Set `Usage` before `Done`; cost is computed from `Model.Cost` when you leave it at zero.
|
|
|
|
Error message conventions the core reacts to:
|
|
|
|
- Retryable errors should mention the HTTP status (`429`, `503`, `529`), `overloaded`, `rate limit`, or
|
|
`stream ended before`.
|
|
- Context overflow should contain `context_length_exceeded` or `prompt is too long`; the core compacts and
|
|
retries once.
|