141 lines
12 KiB
Markdown
141 lines
12 KiB
Markdown
# Volcanic Secretary
|
|
|
|
A personal assistant you talk to over Telegram. The agent engine is [Pi](https://pi.dev) embedded through its SDK. It lives in one workspace directory, runs tools without asking, remembers things in `MEMORY.md`, and can schedule reminders, prompts, and watcher scripts.
|
|
|
|
## Setup (Windows or any host with Node 24)
|
|
|
|
On Windows the quickest path is `run.bat`, which creates `.env` on the first run, then installs, builds, and starts the bot on every later run. The step by step guide, including how to get the bot token and your user id, is in [SETUP-WINDOWS.md](SETUP-WINDOWS.md).
|
|
|
|
1. Create a bot with @BotFather and copy the token.
|
|
2. Find your numeric Telegram user id (send `/start` to @userinfobot).
|
|
3. Log in to a model provider once with `pi` so `~/.pi/agent/auth.json` exists, or plan to set `PI_MODEL` plus a provider API key.
|
|
4. Copy `.env.example` to `.env` and fill in `TELEGRAM_BOT_TOKEN` and `TELEGRAM_OWNER_ID`. Add `OPENAI_API_KEY` if you want voice notes transcribed.
|
|
5. `npm install`, then `npm run dev` (development) or `npm run build && npm start`.
|
|
|
|
The bot messages you `π’ Online.` when it is ready. Only your user id is accepted; everyone else is ignored.
|
|
|
|
## Talking to it
|
|
|
|
Send text, photos, voice notes, or files. Photos go to the model as images. Voice notes are transcribed with OpenAI and echoed back as `π€ ...`. Files are saved to `workspace/inbox/` and the agent is told the path.
|
|
|
|
While the agent works you see one `βοΈ Workingβ¦` message that updates with tool activity, then the answer. Send another message while it works and it is queued for the current run (you get a π reaction).
|
|
|
|
It can also send files back. When you ask for a screenshot, a generated file, or a rendered report, the agent uses its `send` tool to attach the file directly β photos appear inline, other files arrive as documents.
|
|
|
|
Reply to any of my messages (or "reply with quote" a specific part) and I'll see what you're answering to β useful for coming back to something from earlier in the day.
|
|
|
|
Commands: `/status`, `/new`, `/stop`, `/jobs`, `/cancel <id>`, `/compact`, `/connect_calendar`, `/calendar_done`.
|
|
|
|
## Scheduling
|
|
|
|
Ask in plain language:
|
|
|
|
- "Remind me at 9 tomorrow to call the dentist" (one-off, sent directly)
|
|
- "Every weekday at 8, summarize what's in my inbox folder" (recurring prompt through the agent)
|
|
- "Write a script that checks the price on this page every 10 minutes and tell me only if it drops below 50" (watcher script)
|
|
|
|
Watcher scripts follow one rule: print nothing and exit 0 to stay silent, print a message to trigger delivery, exit non-zero on a real error. Failures are reported once and stay quiet while the same failure repeats.
|
|
|
|
Jobs live in `workspace/.secretary/jobs.json`.
|
|
|
|
## Google Calendar
|
|
|
|
The bot can read and write the owner's Google Calendar. Setup is one-time in Google Cloud, then two commands in the bot.
|
|
|
|
1. In Google Cloud Console, enable the **Google Calendar API** and create an **OAuth 2.0 client of type "TVs and Limited Input devices"**.
|
|
2. Put the client id and secret in `.env` as `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET`. Restart the bot.
|
|
3. In Telegram, send `/connect_calendar`. The bot replies with a `https://google.com/device` link and a short code.
|
|
4. Open the link, sign in, enter the code, approve. Then send `/calendar_done` back to the bot. It replies `β
Calendar connected.`
|
|
|
|
The refresh token is stored at `workspace/.secretary/google-token.json`. Delete that file to disconnect; run `/connect_calendar` again to relink.
|
|
|
|
## Stealth browsing (Fortress)
|
|
|
|
[Fortress](https://github.com/tiliondev/fortress) is a Chromium build that corrects the browser fingerprint inside the engine, so pages behind Cloudflare, DataDome, PerimeterX, or Akamai load instead of returning 403 or a challenge. With `FORTRESS_ENABLED=true` the bot starts its MCP server (`tilion-mcp`) and the agent gets about 30 `fortress_*` tools: `read_page` and `extract_page` (clean markdown, far fewer tokens than raw HTML), `fetch_protected_page`, `search_web`, `crawl_site`, `screenshot_page`, `save_profile` / `load_profile` for keeping logins, and more. It sits next to the Playwright tools; the agent is told to reach for Fortress when reading the web or when a site blocks it.
|
|
|
|
The Docker image ships it ready to use. On a bare host, install it once with `pip install "tilion[mcp]" "mcp<2"` (the pin works around tilion not supporting the 2.x MCP library yet); the engine, about 200 MB, downloads on first use. Linux x64 and Windows x64 only.
|
|
|
|
Screenshots, saved pages, downloads, and saved profiles land in `workspace/fortress/`, so the agent can send them back with `send`. If Fortress fails to start, the bot logs the error and runs without it. Set `LOG_LEVEL=debug` to see the server's own output.
|
|
|
|
## Workspace
|
|
|
|
```
|
|
workspace/
|
|
AGENTS.md persona and standing instructions (edit freely)
|
|
MEMORY.md facts the assistant keeps; it edits this itself
|
|
scripts/ scripts for scheduled jobs
|
|
inbox/ files you sent
|
|
fortress/ files written by the Fortress tools
|
|
.secretary/ runtime state
|
|
```
|
|
|
|
The agent's edits to `MEMORY.md` are local runtime data. Leave them uncommitted, or run `git update-index --assume-unchanged workspace/MEMORY.md`.
|
|
|
|
## Environment variables
|
|
|
|
| Variable | Required | Default | Purpose |
|
|
| --- | --- | --- | --- |
|
|
| `TELEGRAM_BOT_TOKEN` | yes | | Bot token |
|
|
| `TELEGRAM_OWNER_ID` | yes | | Your numeric user id |
|
|
| `WORKSPACE_DIR` | no | `./workspace` | Workspace directory |
|
|
| `OPENAI_API_KEY` | no | | Enables voice transcription |
|
|
| `STT_MODEL` | no | `gpt-4o-transcribe` | OpenAI transcription model |
|
|
| `PI_MODEL` | no | first model with credentials | `provider/model` or `provider/model:thinking`, for example `pi-claude-code-provider/opus` with the Claude Code subscription package; `pi --list-models` shows every id |
|
|
| `PI_CODING_AGENT_DIR` | no | `~/.pi/agent` | Pi config directory (auth, settings, packages) |
|
|
| `SECRETARY_TIMEZONE` | no | system | IANA timezone for cron and time display |
|
|
| `SECRETARY_SHELL` | no | Git Bash / bash | Shell used for scheduled scripts |
|
|
| `LOG_LEVEL` | no | `info` | `debug`, `info`, `warn`, `error` |
|
|
| `PLAYWRIGHT_ENABLED` | no | `false` | Enable the Playwright MCP bridge (browser control tools) |
|
|
| `PLAYWRIGHT_HEADLESS` | no | `true` | Run browser headless. `false` opens a visible window |
|
|
| `PLAYWRIGHT_BROWSER` | no | `chrome` | `chrome`, `firefox`, `webkit`, or `msedge` |
|
|
| `PLAYWRIGHT_EXTRA_ARGS` | no | | Extra flags passed to `@playwright/mcp` (shell-quoted) |
|
|
| `FORTRESS_ENABLED` | no | `false` | Enable the Fortress stealth-browser MCP bridge (`fortress_*` tools) |
|
|
| `FORTRESS_COMMAND` | no | `tilion-mcp` | Command that starts the Fortress MCP server (shell-quoted), e.g. `python -m tilion.mcp` |
|
|
| `FORTRESS_HEADLESS` | no | `true` | `false` opens a visible Fortress window (native hosts only) |
|
|
| `FORTRESS_CHANNEL` | no | `latest` | Fortress engine channel. `stable` currently fails to launch (upstream bundle naming bug) |
|
|
| `TILION_PROXY` | no | | Egress proxy for Fortress, `http://user:pass@host:port`. A residential proxy is the main lever on hard targets |
|
|
| `TILION_REGION` | no | | Region hint for Fortress, e.g. `us`, so timezone and locale match the proxy IP |
|
|
| `CAPTCHA_API_KEY` | no | | Captcha solver key; enables `fortress_solve_captcha` and auto-solve during fetch |
|
|
| `GOOGLE_CLIENT_ID` | no | | OAuth client id (TVs and Limited Input devices) β enables the calendar tool |
|
|
| `GOOGLE_CLIENT_SECRET` | no | | OAuth client secret β required alongside `GOOGLE_CLIENT_ID` |
|
|
|
|
## Keeping it running on Windows
|
|
|
|
Use Task Scheduler (run `node dist/main.js` at logon, restart on failure) or `npm i -g pm2` then `pm2 start dist/main.js --name secretary`. If Git is installed per-user or somewhere else entirely, set `SECRETARY_SHELL` to the full path of `bash.exe`.
|
|
|
|
## Docker / Coolify
|
|
|
|
`docker compose up -d --build` runs it with two named volumes: `/workspace` and `/pi-agent`. Provide the environment through Coolify. For model auth either copy a Pi agent directory containing `auth.json` into the `/pi-agent` volume, or set `ANTHROPIC_API_KEY` and `PI_MODEL=anthropic/claude-opus-4-7` (an id from Pi's built-in Anthropic provider, which is what an API key unlocks). The image includes bash, git, ripgrep, and Python 3 so watcher scripts work the same as on Windows.
|
|
|
|
## Tests
|
|
|
|
`npm test` runs the unit tests. `SECRETARY_SMOKE=1 npx vitest run src/agent/pi-backend.smoke.test.ts` runs one real model call through Pi.
|
|
|
|
### Manual end-to-end checklist
|
|
|
|
Run `npm run dev` with a real bot and walk through every line:
|
|
|
|
1. `hi` β progress message appears then disappears, reply arrives as formatted text.
|
|
2. Long answer (`explain git rebase in detail with code examples`) β multiple messages, code blocks render, no HTML shows raw.
|
|
3. Photo with caption `what is this` β reply describes the image.
|
|
4. Voice note β `π€ <transcript>` then a reply. Without `OPENAI_API_KEY` β the not-configured note.
|
|
5. Document (a small `.csv`) with caption `summarize this` β file appears in `workspace/inbox/` with date prefix, reply uses it.
|
|
6. Send a message while a long run is in progress β π reaction, and the agent picks it up in the same run.
|
|
7. `/stop` during a run β `Stopped.` and no final answer message.
|
|
8. `remind me in 2 minutes to stretch` β confirmation with the fire time; two minutes later `<job name>: <message>` arrives directly.
|
|
9. `every minute run scripts/ping.py and tell me if it prints something` where `ping.py` prints on even minutes β messages arrive only on even minutes; `/jobs` lists it; `/cancel <id>` removes it.
|
|
10. Ask it to remember a fact (`remember that my dog is called Rex`), then `/new`, then `what is my dog called` β answer comes from `MEMORY.md`.
|
|
11. Stop the daemon, start it again β conversation continues (`what did we just talk about`).
|
|
12. `/compact` β reply with token counts.
|
|
13. "Every Monday and Wednesday at 9 create a standup event for 15 minutes" β an event is created with `RRULE:FREQ=WEEKLY;BYDAY=MO,WE`; "list the next 3 standups" returns three occurrences; "move next Wednesday's standup to 10" changes just that instance; "delete the standup series" removes them all.
|
|
14. Start with `PLAYWRIGHT_ENABLED=true`. Startup log shows `playwright tools enabled (N tools)` with N > 20. Ask "open https://example.com and tell me the H1 text" β reply contains `Example Domain`.
|
|
15. With Playwright still enabled, ask "screenshot https://example.com and send it to me" β an image attachment arrives.
|
|
16. Ask "run this on example.com: `async (page) => { await page.goto('https://example.com'); return await page.title(); }`" β reply says `Example Domain`. Exercises `browser_run_code_unsafe`.
|
|
17. Restart with `PLAYWRIGHT_ENABLED=false`. Startup log shows `playwright tools disabled`; no `browser_*` tools appear in the model's tool list.
|
|
18. Ask "write a short poem to poem.txt and send it to me" β a `.txt` document attachment arrives with the poem inside, and the reply confirms with one line.
|
|
19. Reply to one of the bot's earlier messages with a follow-up like "what did you mean by that?" β the answer references that specific message, not the most recent one. Then "reply with quote" a specific phrase from a bot message β the answer references the quoted phrase.
|
|
20. Forward a voice message from another chat and add a text comment. The bot should reply once, referencing both your comment and what the voice said, not twice with each in isolation.
|
|
21. Delete `workspace/.secretary/google-token.json` if it exists. Ask "what's on my calendar tomorrow?" β tool replies with a "not connected" message referencing `/connect_calendar`. Run `/connect_calendar` β bot replies with a `https://google.com/device` URL and a short code. Approve on that page, then `/calendar_done` β bot replies `β
Calendar connected.` and the token file now exists. Ask the same calendar question again β the tool answers.
|
|
22. Start with `FORTRESS_ENABLED=true`. Startup log shows `fortress tools enabled (N tools)` with N around 30. Ask "read https://quotes.toscrape.com/js/ and give me the first three quotes" β the quotes come back (that page is empty without JavaScript).
|
|
23. With Fortress enabled, ask "take a Fortress screenshot of https://example.com and send it" β an image attachment arrives and `workspace/fortress/` has the PNG.
|