docs: user documentation and headless sample host
This commit is contained in:
97
docs/providers.md
Normal file
97
docs/providers.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# Providers and models
|
||||
|
||||
## Model keys
|
||||
|
||||
Models are addressed as `provider/model-id`, for example `anthropic/claude-sonnet-4-5`, `openai/gpt-5`,
|
||||
`claude-code/sonnet`, `openrouter/deepseek/deepseek-chat`. `AgentSession.SetModelAsync("provider/id")`,
|
||||
`AgenticCoreOptions.DefaultModel` and `AgentSessionOptions.Model` all take this form.
|
||||
|
||||
`Model.Provider` is the account/auth scope; `Model.Api` is the wire protocol implementation
|
||||
(`anthropic-messages`, `openai-completions`, `claude-code-headless`, ...). Many providers share one API with a
|
||||
different base URL.
|
||||
|
||||
## Authentication
|
||||
|
||||
For each provider the registry resolves an API key in this order:
|
||||
|
||||
1. The registration's `ApiKey` spec: a literal, `$ENV_VAR` / `${ENV_VAR}`, or `!command` (stdout is the key,
|
||||
cached per process). `$$` and `$!` escape a literal `$`.
|
||||
2. `~/.volcanic/agent/auth.json`: `{ "anthropic": { "apiKey": "sk-..." }, "openai": { "env": "MY_OPENAI_VAR" } }`
|
||||
3. The `<PROVIDER>_API_KEY` environment variable (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`).
|
||||
|
||||
Providers with `RequiresApiKey = false` (Claude Code, Ollama) skip this. `IModelRegistry.IsAvailable(model)`
|
||||
tells a UI whether a model can be selected.
|
||||
|
||||
## Built-in provider extensions
|
||||
|
||||
| Extension | Provider id | API | Notes |
|
||||
|---|---|---|---|
|
||||
| `AnthropicProviderExtension` | `anthropic` | `anthropic-messages` | Thinking budgets from the thinking level, prompt caching on system prompt and last user block, images, tools |
|
||||
| `OpenAIProviderExtension` | `openai` (configurable) | `openai-completions` | `OpenAIProviderOptions` sets base URL, provider id, models, header names; helpers `OpenAIProviderOptions.OpenRouter(models)` and `Ollama(models)` |
|
||||
| `ClaudeCodeProviderExtension` | `claude-code` | `claude-code-headless` | Uses the `claude` CLI on a subscription; see [claude-code-provider.md](claude-code-provider.md) |
|
||||
|
||||
Add the same extension twice with different options to serve several OpenAI-compatible endpoints:
|
||||
|
||||
```csharp
|
||||
o.AddExtension(new OpenAIProviderExtension()); // openai/*
|
||||
o.AddExtension(new OpenAIProviderExtension(OpenAIProviderOptions.OpenRouter([
|
||||
new Model { Id = "deepseek/deepseek-chat", Name = "DeepSeek", Provider = "openrouter", Api = "openai-completions", ContextWindow = 128_000, MaxTokens = 8_192 },
|
||||
])));
|
||||
o.AddExtension(new OpenAIProviderExtension(OpenAIProviderOptions.Ollama([
|
||||
new Model { Id = "qwen2.5-coder", Name = "Qwen Coder", Provider = "ollama", Api = "openai-completions", ContextWindow = 32_000, MaxTokens = 4_096 },
|
||||
])));
|
||||
```
|
||||
|
||||
## Writing a provider
|
||||
|
||||
A provider is a `ProviderRegistration` with a `Stream` delegate:
|
||||
|
||||
```csharp
|
||||
api.RegisterProvider(new ProviderRegistration
|
||||
{
|
||||
Id = "my-provider",
|
||||
Name = "My Provider",
|
||||
Api = "my-api",
|
||||
BaseUrl = "https://llm.example.com",
|
||||
ApiKey = "$MY_PROVIDER_API_KEY",
|
||||
Models = [new Model { Id = "fast", Name = "Fast", Provider = "my-provider", Api = "my-api", ContextWindow = 128_000, MaxTokens = 8_192 }],
|
||||
Stream = StreamAsync,
|
||||
});
|
||||
|
||||
static async IAsyncEnumerable<AssistantMessageEvent> StreamAsync(Model model, LlmContext context, StreamOptions options, [EnumeratorCancellation] CancellationToken ct)
|
||||
{
|
||||
var builder = new AssistantMessageBuilder(model);
|
||||
yield return builder.Start();
|
||||
// ... call your API, then for each delta:
|
||||
yield return builder.AppendText("Hello");
|
||||
// tool calls:
|
||||
yield return builder.StartToolCall("call_1", "bash");
|
||||
yield return builder.AppendToolCallDelta("{\"command\":\"ls\"}");
|
||||
yield return builder.EndToolCall();
|
||||
builder.SetUsage(new Usage(Input: 10, Output: 5, CacheRead: 0, CacheWrite: 0));
|
||||
yield return builder.Done(StopReason.Stop); // becomes ToolUse automatically when tool calls exist
|
||||
}
|
||||
```
|
||||
|
||||
Contract:
|
||||
|
||||
- Emit `Start`, balanced block events, then exactly one `Done` or `Error`. `AssistantMessageBuilder` does the
|
||||
bookkeeping and produces the cumulative `Partial` on every event.
|
||||
- Never throw. Encode failures with `builder.Fail(message)`; cancellation with `builder.Fail("aborted", aborted: true)`.
|
||||
The registry converts stray exceptions and missing terminal events into `Error` events anyway.
|
||||
- Honour `StreamOptions`: `ApiKey` (already resolved), `MaxTokens`, `Temperature`, `Reasoning` (a
|
||||
`ThinkingLevel`; map it through `model.ThinkingLevelMap`), `CacheRetention`, `SessionId` (cache affinity),
|
||||
`Headers`, and the hooks `OnPayload` (replace the payload), `OnResponse` (status and headers),
|
||||
`OnProviderStreamEvent` (raw events). The hooks are what power the `before_provider_request` extension event.
|
||||
- The registry normalises the transcript before calling you: failed assistant messages are dropped, missing
|
||||
tool results are synthesised, custom messages become user messages, thinking from other models becomes text,
|
||||
and images are replaced for text-only models. You still receive `UserMessage`, `AssistantMessage` and
|
||||
`ToolResultMessage` and must map all three.
|
||||
- Set `Usage` before `Done`; cost is computed from `Model.Cost` when you leave it at zero.
|
||||
|
||||
Error message conventions the core reacts to:
|
||||
|
||||
- Retryable errors should mention the HTTP status (`429`, `503`, `529`), `overloaded`, `rate limit`, or
|
||||
`stream ended before`.
|
||||
- Context overflow should contain `context_length_exceeded` or `prompt is too long`; the core compacts and
|
||||
retries once.
|
||||
Reference in New Issue
Block a user