Draft design for send-attachments tool
Adds a `send` custom tool so the agent can push files and images from the workspace to the owner as Telegram attachments, closing the gap where the bot could receive files but only ever reply with text.
This commit is contained in:
EugeneTes
131
docs/superpowers/specs/2026-09-08-send-attachments-design.md
Normal file
131
docs/superpowers/specs/2026-09-08-send-attachments-design.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# Sending files and images back to the owner
|
||||
|
||||
## Goal
|
||||
|
||||
Give the agent a `send` custom tool that pushes a file from the workspace back to the owner as a Telegram attachment — either an inline photo or a downloadable document. This closes the gap between the bot already accepting files/photos and only ever replying with text, and makes real the promise in README manual E2E item #15 (Playwright screenshot arrives as an image).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Streaming from URLs. The model can save to the workspace first (e.g. via `bash curl` or a Playwright screenshot-to-path), then hand the path to `send`.
|
||||
- Media groups / albums / multi-file per call. One file per invocation.
|
||||
- Re-sending files from `inbox/` back to the owner — they already have those.
|
||||
- Video / audio / voice-note specific send helpers. Everything non-image goes through `sendDocument`. If the owner wants a native video player later, a `kind:"video"` branch is a one-line addition.
|
||||
- Uploading base64 blobs through the tool-call channel. The tool is path-based.
|
||||
|
||||
## Architecture
|
||||
|
||||
New module `src/tools/send.ts`:
|
||||
|
||||
```ts
|
||||
export interface SendToolOptions {
|
||||
bot: Pick<Bot["api"], "sendPhoto" | "sendDocument">;
|
||||
chatId: number;
|
||||
workspaceDir: string;
|
||||
}
|
||||
|
||||
export function createSendTool(opts: SendToolOptions): CustomTool;
|
||||
```
|
||||
|
||||
The factory returns a Pi custom tool declared with `defineTool` from `@earendil-works/pi-coding-agent`, matching the shape of `createScheduleTool` and `createCalendarTool`.
|
||||
|
||||
### Tool schema
|
||||
|
||||
```
|
||||
name: "send"
|
||||
label: "Send"
|
||||
description: "Send a file from the workspace to the owner as a Telegram
|
||||
attachment. Use for photos, screenshots, generated files,
|
||||
PDFs, or anything the owner should see or keep."
|
||||
parameters:
|
||||
path: string, required — workspace-relative path to the file.
|
||||
kind: "auto" | "photo" | "document", optional — default "auto":
|
||||
photo for image/jpeg|png|webp|gif, document otherwise.
|
||||
caption: string, optional — text shown under the attachment.
|
||||
```
|
||||
|
||||
### Execution
|
||||
|
||||
1. Resolve `params.path` against `workspaceDir`. Reject if the resolved absolute path does not start with `workspaceDir + path.sep` (path-traversal guard).
|
||||
2. `fs.stat` the file. Missing → throw `"file not found: <path>"`. Directory → throw `"path is a directory, not a file: <path>"`.
|
||||
3. Decide `effectiveKind`:
|
||||
- `params.kind === "photo"` → photo
|
||||
- `params.kind === "document"` → document
|
||||
- `params.kind === "auto"` or omitted → photo iff the extension is `.jpg`, `.jpeg`, `.png`, `.webp`, `.gif`; otherwise document
|
||||
4. Enforce size limits (bot-API limits): photo ≤ 10 MB, document ≤ 50 MB. Over → throw a clear message the model can read and either downscale or switch kind.
|
||||
5. Build an `InputFile` from the path (`new InputFile(absPath)` from `grammy`). Call `opts.bot.sendPhoto(opts.chatId, inputFile, {caption})` or `opts.bot.sendDocument(opts.chatId, inputFile, {caption})`.
|
||||
6. Return `{ content: [{ type: "text", text: "Sent <path> as <kind>." }], details: {} }` so the tool observation the model sees is short and unambiguous.
|
||||
|
||||
### Wiring in `main.ts`
|
||||
|
||||
Add one line alongside the existing tools:
|
||||
|
||||
```ts
|
||||
customTools.push(createSendTool({
|
||||
bot: bot.api,
|
||||
chatId: config.telegramOwnerId,
|
||||
workspaceDir: config.workspaceDir,
|
||||
}));
|
||||
```
|
||||
|
||||
Order relative to `schedule`, `calendar`, and Playwright tools does not matter.
|
||||
|
||||
### System prompt
|
||||
|
||||
Append one short paragraph to `secretarySystemPrompt` in `src/agent/system-prompt.ts` under a new `## Sending files` section:
|
||||
|
||||
> To send a file or image to the owner, use the `send` tool with a workspace-relative path. `kind` defaults to `auto` (photos for image extensions, otherwise a downloadable document). Prefer `send` over pasting long text when the owner asked for a file, a screenshot, or a rendered report. The final message you write is separate from the attachment — a one-line confirmation ("Sent, see the photo above.") is enough.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
owner: "screenshot example.com and send it to me"
|
||||
→ agent turn
|
||||
→ browser_navigate("https://example.com")
|
||||
→ browser_take_screenshot({filename: "example.png"}) ← lands in cwd (workspaceDir)
|
||||
→ send({path: "example.png"}) ← Telegram sendPhoto
|
||||
→ assistant final text: "Sent, see above."
|
||||
→ outbound.sendMarkdown final text
|
||||
```
|
||||
|
||||
The attachment arrives before the final text because `send` executes during the streaming turn, while `outbound.sendMarkdown` fires from `run_settled`. That ordering is what the owner wants (image first, caption/confirmation after) and it's also robust: if the model aborts before the final text, the file was already delivered.
|
||||
|
||||
## Error handling
|
||||
|
||||
All failures are thrown from the tool `execute` and surface to the model as a tool error, which the model can react to on the next step. Concrete messages:
|
||||
|
||||
- `path escapes workspace: <path>` — traversal / absolute-path outside workspace.
|
||||
- `file not found: <path>`
|
||||
- `path is a directory, not a file: <path>`
|
||||
- `photo too large: <n> MB > 10 MB (Telegram limit). Use kind:"document" or downscale.`
|
||||
- `document too large: <n> MB > 50 MB (Telegram limit).`
|
||||
- Telegram API errors bubble as-is (`sendPhoto` throwing rethrows). The autoRetry middleware installed on `bot.api` already handles transient 429s.
|
||||
|
||||
## Testing
|
||||
|
||||
New file `src/tools/send.test.ts` with a fake `bot.api` (two vitest `vi.fn()`s for `sendPhoto` and `sendDocument`) covering:
|
||||
|
||||
1. `kind:"auto"` on `foo.png` → `sendPhoto` called with the resolved absolute path, `sendDocument` not called.
|
||||
2. `kind:"auto"` on `report.pdf` → `sendDocument` called, `sendPhoto` not called.
|
||||
3. `kind:"document"` on `foo.png` → `sendDocument` called (explicit override).
|
||||
4. `kind:"photo"` on `report.pdf` → `sendPhoto` called (explicit override, even though extension isn't a photo).
|
||||
5. Missing file → throws `file not found`.
|
||||
6. Directory path → throws `path is a directory`.
|
||||
7. Path traversal (`../etc/passwd`, absolute path outside workspaceDir) → throws `path escapes workspace`.
|
||||
8. Photo > 10 MB → throws photo-too-large; `sendPhoto` not called.
|
||||
9. Document > 50 MB → throws document-too-large; `sendDocument` not called.
|
||||
10. `caption` passed through to the Telegram call.
|
||||
11. Return value is `{ content: [{ type: "text", text: "Sent … as photo." }], details: {} }`.
|
||||
|
||||
Fixtures use `vitest`'s `tmpdir()` per test (already the pattern in `workspace.test.ts` / `inbound.test.ts`). Large-file cases use a sparse file (`fs.truncate`) rather than actually writing 50 MB.
|
||||
|
||||
## README updates
|
||||
|
||||
- Add `send` to the "Talking to it" or a new "Sending files back" note: one line that the assistant can push files and photos, not just text.
|
||||
- Manual E2E item #15 is already correct once this is implemented — no wording change needed. Add a new item #18: "Ask 'save a short poem to poem.txt and send it to me' → `send` tool fires, a `.txt` document attachment arrives with the poem inside."
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
- **Two tools (`send_photo` + `send_document`).** Rejected: doubles the tool surface and forces the model to reason about MIME before choosing a tool. Auto-detection with a manual override covers the same range in one tool.
|
||||
- **Base64 payload in the tool call.** Rejected: bloats the model's context (screenshots are hundreds of KB base64), makes the tool call log noisy in the progress UI, and requires the model to shuttle bytes through its own text stream when a path already exists on disk.
|
||||
- **Do it via a Telegram MCP server.** Rejected: overkill for two API calls, adds a subprocess and a version to pin, and the bot already has an authenticated `Bot` instance in scope.
|
||||
- **Broadcast to arbitrary chat IDs.** Rejected: the bot is single-owner by construction (see `bridge.ts` gatekeeper). The tool takes no chat parameter and always targets `config.telegramOwnerId`.
|
||||
Reference in New Issue
Block a user