Content step 2 (InboundMedia.AddFilesAsync, InboxFiles): files land in <cwd>/inbox/<date>-<safe name>[-N], image documents within Telegram:InlineImageMaxBytes are attached inline, oversize files and failed downloads get a notice and no run. Documents, videos, audio and voice are accepted; the unsupported reply is now 'I can read text, photos, files and voice messages.' Spec: Updates helpers are named DocumentMessage/VideoMessage/VoiceMessage/AudioMessage; the getFile 'file is too big' notice reads 'over 20 MB'; the too-large inline note is a parenthesised suffix after the size; size formats spelled out.
28 KiB
Telegram attachments (files in, voice in, files out) β design
Date: 2026-09-26
Status: draft
Depends on: 2026-09-26-telegram-message-context-design.md (ChatInbox, InboundComposer content steps, lease
contract); 2026-09-26-process-environment-hook-design.md (keeps OPENAI_API_KEY out of bash). Used by:
2026-09-26-scheduler-design.md (IChatChannel.SendTextAsync).
Problem
The Telegram host accepts only text and photos (UpdateRouter answers everything else with "I can read text and
photos."). The agent can't receive a document, can't hear a voice note, and can only answer with text: a generated
PDF, a chart or a screenshot never reaches the user.
The reference is VolcanicSecretary (/tmp/VolcanicSecretary): src/telegram/inbound.ts (documents to inbox/,
voice through src/stt/openai.ts), src/telegram/bridge.ts (the π€ echo, notes to the user) and
src/tools/send.ts plus docs/superpowers/specs/2026-09-08-send-attachments-design.md (the send tool). This spec
ports that behaviour. The one structural difference is that VolcanicSecretary's send tool holds the grammY Bot and
the owner's chat id directly. Here tools are host-agnostic and the Telegram host has many chats, so the tool needs a
contract to reach "this session's chat".
Decisions
| Topic | Decision |
|---|---|
| Outbound contract | New IChatChannel in Abstractions (Ui/), next to IUserInterface. The host supplies one per session through AgentSessionOptions.Channel; tools and extensions reach it as IExtensionContext.Channel, and host code (the scheduler) as AgentSession.Channel |
send tool |
A core built-in (Features/Tools/BuiltIn/SendTool.cs) that is registered and active only when the session has a channel. Path handling, kind choice and limit checks live in the tool, once for every host |
| Inbound files | Saved to <cwd>/inbox/<yyyy-MM-dd>-<safe name> (-1, -2β¦ on collision). The prompt carries the caption plus a line with the workspace-relative path |
| Size cap | Telegram bots download at most 20 MB. Bigger files are refused before download, with a reply, and not sent to the agent |
| Image documents | JPEG/PNG/WebP/GIF sent "as file" are saved like any document and attached to the prompt as ImageContent when β€ InlineImageMaxBytes (3.75 MB, so base64 stays under the 5 MB image cap providers enforce) |
| Speech to text | A host service (Speech/ISpeechToText, OpenAiSpeechToText), not an abstraction: it normalises input before a prompt exists, and the core only ever sees text |
| STT model | gpt-4o-transcribe by default, configurable. Off when no API key is configured; voice then gets a clear "not configured" reply |
| Transcript echo | π€ <transcript> goes to the chat before the run starts, cut to Telegram's 4096-character limit |
| Ordering | Inbound preparation (download, save, transcribe) runs as content steps inside InboundComposer.ComposeAsync, under the message's ChatInbox lease (message-context spec). Polling never blocks, and parts merge in Telegram order whatever order they finish in. Notices, the π€ echo and outbound files go through the chat renderer's queue (RunInOrderAsync) so they stay in order with rendered text |
| Commands / answers | A message with an attachment is never a command and never answers a pending InputAsync question. Its caption is prompt text. This extends today's photo rule |
Why a contract in Abstractions, not a host-registered tool
The alternative was a Telegram-only TelegramSendExtension added per chat through
AgentSessionOptions.ExtraExtensions, holding chatId and TelegramSender. It's less code today, but:
- "Send this file to the user" is host-agnostic. Path confinement, kind choice, limits and the error messages the model reads would be re-implemented by every host (a Slack or web host would copy all of it).
- The scheduler (
2026-09-26-scheduler-design.md) has to deliver text to a chat with no tool call and possibly no run. A host-private tool gives it nothing to reuse.IChatChannel.SendTextAsyncdoes. - Extensions loaded from
.volcanic/extensionsreference only Abstractions. With the contract, an extension can deliver a report without knowing the host.
IUserInterface wasn't extended either. It's the dialog surface: request/response with the user, Notify for
transient notices. Delivering content into the conversation is a separate capability that a host may lack
(NullUserInterface hosts, the headless sample). Adding methods would force every UI to stub them.
Components
1. Abstractions β IChatChannel
src/Volcanic.AgenticCore.Abstractions/Ui/IChatChannel.cs:
namespace Volcanic.AgenticCore.Abstractions.Ui;
public enum ChatFileKind { Auto, Photo, Document }
/// <param name="Path">Absolute path of an existing file; the caller has already confined and validated it.</param>
/// <param name="Caption">Plain text shown with the file, at most <see cref="ChatChannelLimits.MaxCaptionLength"/>.</param>
public sealed record ChatFile(string Path, ChatFileKind Kind = ChatFileKind.Auto, string? Caption = null);
/// <param name="SentAs">How a file was actually delivered (Photo or Document); null for text.</param>
/// <param name="MessageId">Host message id(s), for logs and tool details.</param>
public sealed record ChatDelivery(ChatFileKind? SentAs, string? MessageId);
public sealed record ChatChannelLimits(long MaxPhotoBytes, long MaxDocumentBytes, int MaxCaptionLength);
/// <summary>Delivers content into the conversation's chat, outside the normal assistant-message rendering.</summary>
public interface IChatChannel
{
ChatChannelLimits Limits { get; }
/// <summary>Markdown, rendered like assistant text (split when long). Not added to the session transcript.</summary>
Task<ChatDelivery> SendTextAsync(string markdown, CancellationToken cancellationToken);
/// <summary>
/// Deliver a file. <see cref="ChatFileKind.Auto"/> lets the channel choose, and it may fall back from photo to
/// document. Throws <see cref="ChatDeliveryException"/> with a message fit for the model or the user.
/// </summary>
Task<ChatDelivery> SendFileAsync(ChatFile file, CancellationToken cancellationToken);
}
public sealed class ChatDeliveryException(string message, Exception? inner = null) : Exception(message, inner);
IExtensionContextgetsIChatChannel? Channel => null;, a default interface member, soStubExtensionContextand third-party implementations keep compiling.ExtensionContextreturns the session's channel.AgentSessionOptions.Channel(IChatChannel?) andAgentSession.Channel(read-only) are new.- Contract rules: channels never throw anything except
ChatDeliveryExceptionandOperationCanceledException, and are safe to call from any thread, including while no run is active (the scheduler case). - Nothing about a delivery is persisted by the core. The
sendtool's result message records it. The scheduler does not append aCustomMessageeither: direct reminders and watcher output stay out of the transcript (see Scheduler reuse).
2. Core β the send tool
src/Volcanic.AgenticCore/Features/Tools/BuiltIn/SendTool.cs, a Tool<SendArgs>:
name: send
args: path string, required workspace-relative (or absolute inside the workspace)
kind "auto" | "photo" | "document", optional, default auto
caption string, optional
snippet: "send: send a workspace file to the user (images inline, other files as documents)"
guidelines:
- When the user asks for a file, image, screenshot or report, write it into the workspace and call send;
your reply arrives after the attachment, so a one-line confirmation is enough.
- Files the user sends you are saved under inbox/.
ExecutionMode: Sequential (two sends in one turn arrive in call order)
Execution:
ctx.Channelis null β throwsend is not available: this session has no chat(defensive; see registration).- Resolve with the existing
Paths.Resolve(ctx.Cwd, path). Reject unless the full path is insidectx.Cwd(compare with a trailing separator, case-insensitively on Windows). If the file or any parent is a symlink, resolve it withResolveLinkTarget(returnFinalTarget: true)and apply the same check to the target. Errors arepath escapes the workspace: <path>,file not found: <path>andpath is a directory, not a file: <path>. - Caption longer than
Limits.MaxCaptionLengthβcaption is <n> characters; the limit is <m>. Shorten it or put the text in your reply.The check runs before anything is sent, so a failure never delivers half an attachment. - Kind:
photo: overMaxPhotoBytesβphoto too large: <x> MB > <y> MB. Use kind "document" or downscale it.document: overMaxDocumentBytesβfile too large: <x> MB > <y> MB (the chat's upload limit).auto:.jpg/.jpeg/.png/.webpwithin the photo limit βAutois passed through (the channel tries a photo and falls back). Anything else, including.gif(a photo upload loses the animation), is sent asDocumentunder the document limit.
await channel.SendFileAsync(...). AChatDeliveryExceptionis rethrown as-is, which gives an error result the model can read.- Result:
Sent <path> as <photo|document>.with details{ path, kind, bytes, messageId }.
Registration in AgentSessionFactory: when options.Channel is not null and IncludeBuiltInTools, add SendTool
as a built-in, active by default (it's appended to DefaultActive for that session). An explicit
options.ActiveTools still wins. Sessions without a channel never see the tool. The Claude Code provider exposes it
as mcp__volcanic__send automatically, like every active tool.
send doesn't refuse to send files from inbox/. The confinement is about intent, not secrecy: bash can
already copy any readable file into the workspace. See Security.
3. Telegram β outbound: TelegramChatChannel
hosts/Volcanic.AgenticCore.Telegram/Chats/TelegramChatChannel.cs, one per chat, created in
ChatRegistry.CreateAsync next to TelegramUi and passed as AgentSessionOptions.Channel. ChatContext exposes
it (chat.Channel).
Limits: photo 10 MB, document 50 MB, caption 1024 (Bot API limits), inTelegramLimitsconstants.- Ordering through the renderer. Session events are rendered asynchronously from
ChatRenderer's queue. A file sent directly could overtake an assistant message that ended just before the tool ran.ChatRenderergetsTask<T> RunInOrderAsync<T>(Func<CancellationToken, Task<T>> work). It queues a host-internal work item behind the pending events and completes when the item has run. The renderer marks the status message as buried afterwards, as it does for replies, so theβοΈ Working...message moves back to the bottom on the next tool update.SendTextAsyncandSendFileAsyncboth go through it. The event channel becomes a channel of render items (an event or a work item). A work item still queued when the renderer is disposed is cancelled.ChatRenderer.SayAsync(text)(aRunInOrderAsyncaroundTelegramSender.SendTextAsync) sends plain-text notices and theπ€echo. The reply path (ChatRenderer.SendReplyAsync) returns the first message id and whether every chunk was delivered. - Files.
TelegramSender.SendFileAsync(chatId, path, asPhoto, caption, ct) β Task<FileSendResult>, withFileSendResult(int? MessageId, string? Error, bool PhotoRejected). It follows the sender's never-throw convention. Each attempt opens a freshFileStreamand usesInputFile.FromStream(stream, Path.GetFileName(path))withSendPhotoorSendDocument. Captions go as plain text (no parse mode), so the model's caption can't break the HTML.- A network failure (
RequestExceptionthat is not anApiRequestException) is retried after 1 s and 3 s, like replies. The delays areTelegramSender.FileRetryDelays, an internal test seam. - An API error is not retried. A 400 on
sendPhotocomes back asPhotoRejected = true. Autowhose photo upload is rejected with a 400 (PHOTO_INVALID_DIMENSIONS,IMAGE_PROCESS_FAILED,PHOTO_SAVE_FILE_INVALID, or any 400 onsendPhoto) is resent once as a document.SentAs = Document.- Anything else becomes
ChatDeliveryException("Telegram refused the file: <description>").
- A network failure (
- Text.
SendTextAsync(markdown)reuses the renderer's reply path (TelegramFormatter.SplitβToHtmlβSendHtmlAsyncwith the plain fallback). It returns the id of the first message, or throwsChatDeliveryException("Telegram did not accept the message.")when a chunk still failed after its retries (the other chunks are still sent).
4. Telegram β inbound: InboundMedia
hosts/Volcanic.AgenticCore.Telegram/Inbound/InboundMedia.cs provides the content steps this spec adds to
InboundComposer (message-context spec). It depends on TelegramSender (downloads), ISpeechToText?,
TelegramOptions and TimeProvider. Each step adds body lines and images to the message's InboundPart, or sends a
notice. A step can also refuse the message (oversize file, failed download, voice with no usable transcript and
no caption): ComposeAsync then returns null and nothing is prompted, and no "I can readβ¦" reply is sent because
the notice already explains it. The steps run in this order after step 1 (text/caption, largest photo):
- Step 2, files:
Document,VideoandAudiosaved to the inbox, image documents also attached asImageContent. - Step 3, speech:
VoiceandAudiotranscription, with theπ€echo.
So a part's body is: caption, then [File received: β¦], then [Voice message] β¦.
Router changes (UpdateRouter.OnMessageAsync), on top of the message-context spec:
- Accepted:
Text,Photo,Document,Video,Audio,Voice. Anything else (stickers, video notes, polls, locationsβ¦) getsI can read text, photos, files and voice messages.and takes no lease. - The command and pending-answer checks run only for messages with no attachment (as today for photos).
- Every accepted message takes its
ChatInboxlease synchronously and runsComposeAsyncin the background, as the message-context spec describes. All slow work (download, save, transcribe) happens insideComposeAsyncunder the lease, so polling never blocks and commands (/stop) act immediately. Ordering comes from the lease slots: a text sent right after a voice note lands in the same batch, after the transcript. There is no separate inbound queue. - Notices and the echo are posted through the renderer (
ChatRenderer.SayAsync, plain text), so they appear in order with rendered output and, on an idle chat, before the run'sβοΈ Working....
Files (Document, and Video with FileName ?? "video.mp4"):
FileSize > 20 MB(TelegramLimits.MaxDownloadBytes) β noticeThat file is <x> MB; Telegram lets bots download at most 20 MB. Share a link or a smaller file.The message isn't prompted, caption included (the caption is about the file). AgetFileerror saying "file is too big" (a missingFileSize) is reported the same way, asThat file is over 20 MB; β¦.- Target:
<cwd>/inbox/<date>-<stem><-N><ext>.dateisTimeProvider.GetLocalNow()asyyyy-MM-dd. The name isFileName ?? "file":Path.GetFileNamedrops any directory part, and every character other than a Unicode letter or digit,.,-or_becomes_. Leading dots and underscores are trimmed (no hidden files). Stem is capped at 80 characters and extension at 16, withfileas the fallback. Opened withFileMode.CreateNew; anIOExceptionfor an existing file tries the next suffix. - Download straight into that stream (
GetInfoAndDownloadFile). On failure delete the partial file and send the noticeI could not download that file.The message isn't prompted. - Prompt text: the caption (if any), a blank line, then
[File received: inbox/2026-09-26-report.pdf (application/pdf, 1.2 MB)]. The path is relative to the workspace and uses/. Sizes readN bytes,N.N KBorN.N MB. - Image documents: when
MimeTypeisimage/jpeg|png|webp|gifand the size is β€Telegram:InlineImageMaxBytes(default 3 750 000), the saved bytes are also attached asImageContent(base64, mime). Larger images are path-only, and the line gets a parenthesised suffix after the size:[File received: inbox/x.png (image/png, 5.0 MB) (too large to show inline; downscale it with bash if you need to see it)].
Voice and audio (Voice, Audio):
- No
ISpeechToTextconfigured β noticeVoice messages need speech-to-text: set OPENAI_API_KEY (or Telegram:Speech:ApiKey) on the bot host.Audiois still saved to the inbox as a file (below);Voiceisn't prompted unless it has a caption. - Over 20 MB β the file-size notice, as for files.
- Download into memory. Voice arrives as
.oga, which OpenAI rejects, so it's renamed tovoice.ogg(audio/ogg). Audio keepsFileName ?? "audio"and itsMimeType. Transcript = await stt.TranscribeAsync(bytes, fileName, mime, ct), trimmed. Empty β noticeI couldn't make out any words in that voice message.An exception β noticeCould not transcribe the voice message: <reason>.- Echo
π€ <transcript>as plain text, cut to 4096 characters withβ¦. - Prompt text: the caption (if any), then
[Voice message] <transcript>. ForAudio, the file is also saved to the inbox like a document and gets its own[File received: β¦]line: it's something the user might want processed, not just heard. - Transcripts never answer a pending
InputAsyncquestion (see Decisions).
UpdateRouter.DownloadPhotoAsync moves into InboundComposer's step 1 (message-context spec) unchanged in behaviour;
InboundMedia shares its download helper.
5. Telegram β speech to text
hosts/Volcanic.AgenticCore.Telegram/Speech/:
public interface ISpeechToText
{
Task<string> TranscribeAsync(ReadOnlyMemory<byte> audio, string fileName, string mimeType, CancellationToken cancellationToken);
}
OpenAiSpeechToText(HttpClient http, SpeechOptions options) is the port of stt/openai.ts:
POST {BaseUrl}/audio/transcriptions,Authorization: Bearer <key>, multipart withfile(name and content type as given),model,response_format=json, andlanguagewhen configured.- Non-2xx β
SpeechToTextException("OpenAI transcription failed: HTTP <status> <first 200 chars of body>"). A non-JSON body or a missingtextgives its own message. The key never appears in messages or logs. HttpClientcomes fromAddHttpClient<ISpeechToText, OpenAiSpeechToText>()(packageMicrosoft.Extensions.Http), with a 2-minute timeout.
SpeechOptions, the Telegram:Speech section:
| Setting | Default | Notes |
|---|---|---|
Telegram:Speech:ApiKey |
OPENAI_API_KEY |
Empty β STT off (the only way to turn it off) |
Telegram:Speech:Model |
gpt-4o-transcribe |
also STT_MODEL, as in VolcanicSecretary. gpt-4o-mini-transcribe and whisper-1 also work |
Telegram:Speech:BaseUrl |
https://api.openai.com/v1 |
any OpenAI-compatible transcription endpoint |
Telegram:Speech:Language |
β | ISO-639-1 hint, optional |
The environment fallbacks (OPENAI_API_KEY, STT_MODEL) are read in TelegramOptions.ApplyEnvironment, next to
the existing TELEGRAM_* shortcuts, and only when the Telegram:Speech setting is empty. An empty variable counts
as unset (the Docker compose file passes both through, empty by default). The OpenAI provider
extension, if loaded, reads OPENAI_API_KEY on its own; the two don't share state.
Program.cs registers ISpeechToText only when a key is present, and logs Speech to text: <model> or
Speech to text: off (no OPENAI_API_KEY). OPENAI_API_KEY isn't on the Claude Code provider's denied-variable
list, and its child environment is allow-listed, so the key never reaches claude. The process environment hook's
default deny-list (*_API_KEY, *__APIKEY) removes it, and Telegram:Speech:ApiKey under any other name (the host
registers the value in SecretValues), from bash, scheduled scripts and MCP servers.
6. Other options
TelegramOptions gains InlineImageMaxBytes (default 3 750 000) and Speech (SpeechOptions). Validate()
rejects a non-positive InlineImageMaxBytes and a Speech:BaseUrl that isn't an absolute http(s) URI.
Scheduler reuse
The scheduler spec delivers text through the same contract. It gets it from AgentSession.Channel (or
ChatContext.Channel for host code) and calls SendTextAsync(markdown, ct). That works whether or not a run is
active, and ordering with a running turn's output holds because the Telegram channel goes through the renderer
queue. If a scheduled job runs the agent instead, its output renders through the normal session events, and send
works in that run as in any other. Direct deliveries (reminders, watcher output, failure alerts) are not added to
the transcript: the channel never persists, and the scheduler doesn't append a CustomMessage. The scheduler spec
accepts the consequence that the agent doesn't see them.
Error handling summary
| Situation | Result |
|---|---|
| file > 20 MB sent to the bot | size notice, nothing reaches the agent |
| download fails | I could not download that file., partial file deleted, no prompt |
| voice with no STT configured | not-configured notice; caption (if any) is still prompted |
| STT HTTP/parse error | Could not transcribe the voice message: β¦, no prompt unless there's a caption |
send path outside the workspace / missing / directory |
error tool result, nothing sent |
send over the photo / document limit |
error tool result naming the limit |
| caption over 1024 | error tool result, nothing sent |
| Telegram rejects a photo (dimensions) with kind auto | resent as a document, result says as document |
Telegram unreachable during send |
2 retries, then an error tool result |
| session without a channel (headless sample) | send isn't registered |
Security
- Downloaded names are sanitised and confined to
inbox/.CreateNewnever overwrites. sendconfines to the workspace, symlinks included. It doesn't stop exfiltration by a determined agent (bash cpworks), but a prompt-injected "send ~/.ssh/id_rsa" fails loudly and visibly instead of silently.sendcan only reach the session's own chat. There's no chat-id parameter.- The OpenAI key lives in the bot process's environment but not in the processes the agent spawns:
envinbashdoesn't show it, norTELEGRAM_BOT_TOKEN(2026-09-26-process-environment-hook-design.md). On Linux the bot is non-dumpable, so/proc/<bot pid>/environis closed too, and the key's value is replaced by[REDACTED]in any tool output (that spec's Β§8βΒ§9). Its caveat applies: it isn't a sandbox, and a determined agent running as the bot's user can still read files that user can read.
Docs
docs/telegram-bot.md: accepted inputs,inbox/, the voice setup and settings table,send, the size limits.docs/ui-integration.md: a new section "5. ImplementIChatChannel" (optional, enablessend), and document / voice rows in the input-mapping table.docs/architecture.mdandCLAUDE.md: listIChatChannelamong the Abstractions.
Testing
xunit, no network, no paid API.
Core (tests/Volcanic.AgenticCore.Tests)
SendToolTestswith a recordingFakeChatChanneland a temp workspace:auto.pngpassesAuto;auto.pdfand.gifpassDocument; explicitphoto/documentare honoured.- An
autoimage over the photo limit goes asDocument; an explicitphotoover the limit fails. - A document over the limit, a caption over the limit,
../x, an absolute path outside, and a symlink inside the workspace pointing outside all fail, and the channel is never called. - Missing file and directory errors. Absolute paths inside the workspace are accepted.
- A
ChatDeliveryExceptionbecomes an error tool result with its message. - Result text and details. Sparse files (
FileStream.SetLength) stand in for big files.
AgentSessionFactoryTests:sendis registered and active with a channel, absent without one, and an explicitActiveToolswins.ExtensionContext.Channelis the session's channel.- End-to-end with
FauxProvider: a scriptedsendcall reaches the fake channel and the run continues to the final text.
Telegram (tests/Volcanic.AgenticCore.Telegram.Tests)
FakeBotClient grows:
Files(file id β bytes and path).GetFileRequestanswers with aTGFile(path and size), andDownloadFilewrites the scripted bytes.SendPhotoRequest/SendDocumentRequestreturn aMessage. Their stream is copied insideSendRequest, so tests can assert the uploaded bytes, name and caption (Uploads).OnSendcan throw anApiRequestException(400, "PHOTO_INVALID_DIMENSIONS")to exercise the fallback.
Updates gets DocumentMessage(...), VideoMessage(...), VoiceMessage(...) and AudioMessage(...), and reuses
StickerMessage() (a member named like a Telegram type would hide that type inside the class). ChatHost takes an optional
ISpeechToText (a FakeSpeechToText returning a scripted transcript or throwing) and uses the optional
TimeProvider the message-context spec adds (FakeTimeProvider from Microsoft.Extensions.TimeProvider.Testing).
- Document with caption: the file lands in
<workspace>/inbox/2026-09-26-report.pdfwith the right bytes. The prompt the faux provider receives contains the caption and the[File received: β¦]line. A second upload of the same name gets-1. - A hostile name (
../../evil.sh,.bashrc, a 300-character name) is sanitised and stays ininbox/. - Over 20 MB: the notice is sent, no file is written, the faux provider isn't called.
- Image document β€ limit: saved and passed as
ImageContent. Above the limit: path only, with the "too large" note. - Voice with STT: the
π€echo comes before the answer, the prompt contains[Voice message] β¦, and the file name handed to STT ends in.ogg. A 5000-character transcript echo is cut to 4096. - Voice without STT: the not-configured notice, and no run. STT throwing: the failure notice, and no run.
- Ordering: a voice note followed immediately by text produces one prompt with the transcript before the text (the
fake STT is delayed past the debounce window).
/stopsent during a slow transcription is handled at once. - A document with a
/newcaption doesn't run/new. A voice note doesn't answer a pendingInputAsync. - A sticker gets the "I can readβ¦" reply.
sendend to end: the faux provider callssendforout/chart.png, asendPhotoupload with the caption appears before the final answer message, and the status message is moved back to the bottom by the next tool line. A.pdfβsendDocument. A photo rejected with a 400 β a second upload as a document, and the tool result saysas document.TelegramChatChannel.SendTextAsyncwhile idle (the scheduler case): the text is delivered as HTML, split past 4096.
Speech (OpenAiSpeechToTextTests) with a stub HttpMessageHandler: the multipart form has file (name and
content type), model and response_format=json, plus language when set; the bearer header is present; the
transcript is trimmed; non-2xx, non-JSON and missing text each give their own message, which never contains the
key.
Out of scope
- A local Bot API server (it lifts the 20 MB / 50 MB limits). The limits are constants for now.
- Albums and media groups as one prompt, and the debounce of forward + comment: both come from
ChatInboxin the message-context spec. - Sending video, audio, voice or animations natively (
sendVideo,sendVoice, β¦): everything that isn't a photo goes as a document. Several files persendcall. URLs or base64 assendinput. - Stickers, video notes, locations, contacts and polls as input.
- Local STT (whisper.cpp) or other providers behind
ISpeechToText: the interface allows them, but only OpenAI is built. - Cleaning up
inbox/: files stay until the user or agent removes them. - Reply and forward context headers (the message-context spec).
Open questions
- Should
Audiofiles (music, podcasts) be transcribed by default, or only saved? This spec follows VolcanicSecretary and transcribes, which costs money on long files. - Is 3.75 MB the right inline threshold for image documents, or should the host downscale larger images (with a dependency such as ImageSharp) rather than send only the path?