admin/Playa
1
0
Fork 0
forked from admin/VolcanicAgenticCore
Code Pull Requests Activity

docs(telegram): attachments, voice, send and IChatChannel

Browse Source 
telegram-bot.md: what you can send, files from the agent, speech settings, security notes. ui-integration.md: document/voice input rows and a new section 5 on IChatChannel. architecture, extensions and CLAUDE.md list IChatChannel. Drop the interim paragraph from the message-context spec; mark the attachments spec implemented with resolved questions.
This commit is contained in:
EugeneTes
2026-09-26 22:16:29 +02:00
parent f90731baf0
commit c5ecd017a0
7 changed files with 95 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
docs/telegram-bot.md
View File

@@ -34,6 +34,11 @@ settings use the `GoogleCalendar` section):
| `Telegram:EditInterval` | `00:00:01.5` | Minimum time between edits of the "Working..." status message |
| `Telegram:InputDebounce` | `00:00:00.6` | How long the bot waits after a message for more before sending them to the agent as one request. `0` disables the wait; messages still being read together (e.g. a photo still downloading and a text after it) are still sent as one request. Must not be negative |
| `Telegram:ReconnectInitialDelay` / `ReconnectMaxDelay` | `1 s` / `60 s` | Backoff after lost connection |
| `OPENAI_API_KEY` / `Telegram:Speech:ApiKey` | — | Turns voice transcription on. Hidden from `bash` and redacted from tool output |
| `STT_MODEL` / `Telegram:Speech:Model` | `gpt-4o-transcribe` | `gpt-4o-mini-transcribe` and `whisper-1` also work |
| `Telegram:Speech:BaseUrl` | `https://api.openai.com/v1` | Any OpenAI-compatible transcription endpoint |
| `Telegram:Speech:Language` | — | Optional ISO-639-1 hint, e.g. `de` |
| `Telegram:InlineImageMaxBytes` | `3750000` | Image files up to this size are also shown to the model; must be positive |
| `GOOGLE_CLIENT_ID` / `GoogleCalendar:ClientId` | — | Optional. With the secret, every chat gets the `calendar` tool and the `/calendar_*` commands; see [google-calendar.md](google-calendar.md) |
| `GOOGLE_CLIENT_SECRET` / `GoogleCalendar:ClientSecret` | — | Optional; redacted from tool output and hidden from `bash` |
| `PROCESS_ENV_ALLOW` | — | Comma-separated names or globs let through to `bash`, scheduled scripts and MCP servers despite the deny-list (e.g. `GH_TOKEN` for `gh`). Added to `ProcessEnvironment:Allow` |
@@ -61,7 +66,7 @@ It lists the last five tool calls (`▶` running, `✓` done, `✗` failed) and
Answers are not streamed: each one is sent complete when the model finishes it (split into several messages past
Telegram's 4096 character limit).
A message sent while the agent works steers the current run, and the bot reacts to it with 👀 so you can see it was
added. Photos (with or without a caption) are passed to the model.
added. Besides text, you can send photos, files and voice messages (see [What you can send](#what-you-can-send)).
Along with your text, the agent sees:
@@ -103,11 +108,33 @@ gets the re-login link and code prompt in their own private chat, regardless of
it. That user must already have started a chat with the bot — Telegram bots cannot message a user first — or
the login goes unanswered and the triggering request fails ("no code was entered"). The link, the steps and the
code prompt arrive as one message. While that prompt is pending, the owner's next plain text message in that chat
is taken as the code and is not sent to the agent; photos and `/commands` still behave normally. A forwarded message
never counts as the code. Don't ask the bot something else in the owner chat until the code is in: a second free-text
is taken as the code and is not sent to the agent; photos, files, voice messages and `/commands` still behave
normally. A forwarded message never counts as the code. Don't ask the bot something else in the owner chat until the code is in: a second free-text
question in that chat (for example from an extension) replaces the pending prompt and cancels the login. See
[claude-code-provider.md](claude-code-provider.md#session-expiry-and-re-login).
### What you can send
- **Text and photos.** A photo, with or without a caption, is passed to the model as an image.
- **Documents, videos and audio files** are saved to `inbox/<yyyy-MM-dd>-<name>` in the chat's workspace (a number is
added when the name is taken: `-1`, `-2`, ...). The agent gets the caption plus a line such as
`[File received: inbox/2026-09-26-report.pdf (application/pdf, 1.2 MB)]`. An image sent "as file" (JPEG, PNG, WebP,
GIF) up to 3.75 MB (`Telegram:InlineImageMaxBytes`) is also shown to the model; a larger one reaches it as a path
only.
- **Voice messages and audio files** are transcribed with OpenAI (`gpt-4o-transcribe` by default). The bot echoes
`🎤 <transcript>` and the agent gets `[Voice message] <transcript>` (an audio file also gets its `[File received: …]`
line). Without an API key the bot replies with how to set one up (`OPENAI_API_KEY`).
- Telegram lets bots download at most 20 MB. A bigger file is refused with a note, and nothing reaches the agent.
- A message with an attachment is never a command and never answers a pending question: its caption is prompt text.
- Anything else (stickers, video notes, locations, polls) gets `I can read text, photos, files and voice messages.`
### Files from the agent
The agent's `send` tool delivers a file from the chat's workspace into the chat, before the agent's reply. Images go
as photos (up to 10 MB), other files as documents (up to 50 MB), with an optional caption of up to 1024 characters. A
photo Telegram rejects (for example for its dimensions) is resent as a document. `send` accepts only files inside the
chat's workspace and delivers only to that chat.
### Google Calendar
Each chat connects its own Google account with `/calendar_connect`, and the agent's `calendar` tool only uses that
@@ -121,6 +148,10 @@ The agent has the built-in tools, including `bash`, and runs them as the bot's O
are served (everyone else is ignored and logged), and only in private chats. Run the bot as a dedicated user or in a
container, and treat the workspace root as writable by anyone on the allow-list.
`send` refuses paths outside the chat's workspace, symlinks included, so a prompt-injected "send ~/.ssh/id_rsa" fails
visibly (it is no barrier against `bash cp`). The speech API key is kept out of the agent's processes like the bot
token.
Secrets are removed from the environment of `bash`, scheduled scripts and MCP servers: every variable on the deny-list
(see `ProcessEnvironment:Deny` above) is dropped, and so is any variable whose value is the bot token, whatever its
name (a Coolify alias, for example). `PROCESS_ENV_ALLOW` lets a named variable through, for example `GH_TOKEN` so the