feat(telegram): ElevenLabs text to speech for the voice tool
ElevenLabs speaks for the voice tool when ELEVENLABS_API_KEY and ELEVENLABS_VOICE_ID are set (TTS_PROVIDER=elevenlabs|openai picks, the other is the fallback). Opus output (opus_48000_64) goes to Telegram as is; voice names resolve through /v2/voices; instructions become audio tags on eleven_v3 and are ignored on other models. The key is redacted from tool output and never reaches agent processes.
This commit is contained in:
@@ -48,7 +48,12 @@ settings use the `GoogleCalendar` section, the scheduler the `Scheduler` section
|
||||
| `STT_MODEL` / `Telegram:Speech:Model` | `gpt-4o-transcribe` | `gpt-4o-mini-transcribe` and `whisper-1` also work |
|
||||
| `TTS_MODEL` / `Telegram:Speech:TtsModel` | `gpt-4o-mini-tts` | Speech model of the `voice` tool; `tts-1` and `tts-1-hd` also work (they ignore `instructions`) |
|
||||
| `TTS_VOICE` / `Telegram:Speech:TtsVoice` | `nova` | Default voice of the `voice` tool |
|
||||
| `Telegram:Speech:TtsInstructions` | — | Default delivery instructions (tone, pace) for the `voice` tool |
|
||||
| `Telegram:Speech:TtsInstructions` | — | Default delivery instructions (tone, pace) for the `voice` tool (OpenAI only) |
|
||||
| `TTS_PROVIDER` / `Telegram:Speech:TtsProvider` | — | `elevenlabs` or `openai`. Empty prefers ElevenLabs. The preferred service is used when configured, else the other |
|
||||
| `ELEVENLABS_API_KEY` / `Telegram:Speech:ElevenLabs:ApiKey` | — | With a voice id, turns on ElevenLabs for the `voice` tool. Hidden from `bash` and redacted from tool output |
|
||||
| `ELEVENLABS_VOICE_ID` / `Telegram:Speech:ElevenLabs:VoiceId` | — | Default ElevenLabs voice: a voice id, or the name of a voice in the account |
|
||||
| `ELEVENLABS_MODEL` / `Telegram:Speech:ElevenLabs:Model` | `eleven_multilingual_v2` | ElevenLabs model; `eleven_v3` reads `instructions` as audio tags, other models ignore them |
|
||||
| `Telegram:Speech:ElevenLabs:OutputFormat` | `opus_48000_64` | Must be an `opus_48000_*` format (OGG/Opus) |
|
||||
| `Telegram:Speech:BaseUrl` | `https://api.openai.com/v1` | Any OpenAI-compatible audio endpoint (transcription and speech) |
|
||||
| `Telegram:Speech:Language` | — | Optional ISO-639-1 hint, e.g. `de` |
|
||||
| `OPENROUTER_API_KEY` / `Telegram:Triggers:ApiKey` | — | Turns on [groups and triggers](#groups-and-triggers). Hidden from `bash` and redacted from tool output |
|
||||
@@ -170,6 +175,12 @@ question in that chat (for example from an extension) replaces the pending promp
|
||||
text over 1500 characters is split at paragraph, sentence or word boundaries into several voice messages. Longer
|
||||
text is refused. Every part is synthesized before the first is sent, so an API error (returned to the agent as text,
|
||||
key hidden) never leaves half a message; synthesis is retried like transcription.
|
||||
**ElevenLabs** takes over when `ELEVENLABS_API_KEY` and `ELEVENLABS_VOICE_ID` are set (and `TTS_PROVIDER` isn't
|
||||
`openai`); it needs no OpenAI key. It calls `POST /v1/text-to-speech/{voice_id}?output_format=opus_48000_64` with
|
||||
`eleven_multilingual_v2` by default, so the audio is OGG/Opus already. The tool's `voice` is a voice id or the name
|
||||
of a voice in the account (looked up once through `/v2/voices`); `instructions` become audio tags (`[warm]`) on
|
||||
`eleven_v3` and are ignored on other models. Errors such as `HTTP 401 quota_exceeded: …` or `HTTP 429 …` come back
|
||||
as text; 429 and 5xx are retried. The tool description follows the active service. The startup log line names it.
|
||||
- Telegram lets bots download at most 20 MB. A bigger file is refused with a note, and nothing reaches the agent.
|
||||
- A message with an attachment is never a command and never answers a pending question: its caption is prompt text.
|
||||
- Anything else (stickers, video notes, locations, polls) gets `I can read text, photos, files and voice messages.`
|
||||
|
||||
Reference in New Issue
Block a user