Draft design for Playwright MCP bridge
Bridge @playwright/mcp as a stdio subprocess and re-export its tools as Pi custom tools, gated on PLAYWRIGHT_ENABLED. Uses Type.Unsafe to pass MCP inputSchema through to the model without hand-mapping.
This commit is contained in:
EugeneTes
@@ -0,0 +1,156 @@
|
||||
# Playwright browser control via MCP bridge
|
||||
|
||||
## Goal
|
||||
|
||||
Give the agent the ability to drive a real browser (navigate, click, type, evaluate JS, take screenshots, watch network, run arbitrary Playwright code) by embedding the [`@playwright/mcp`](https://github.com/microsoft/playwright-mcp) server as a subprocess and re-exposing every tool it publishes as a Pi custom tool.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- HTTP / SSE transport. Stdio only. `PLAYWRIGHT_MCP_URL` for "connect to a running server" can be added later without touching the tool-wrapping code.
|
||||
- Fine-grained tool allow/block lists. Every tool the MCP server reports is exposed. If the tool surface bloats the prompt too much, add filtering later.
|
||||
- Multiple browser contexts / per-user profiles / storage-state persistence. The MCP server's default profile is fine for v1.
|
||||
- Managing browser binary installation at runtime. Chromium is installed at image-build time via the Dockerfile; a missing browser at runtime is a hard startup failure of the bridge (see "Failure mode").
|
||||
|
||||
## Architecture
|
||||
|
||||
New module `src/agent/playwright-mcp.ts`:
|
||||
|
||||
```ts
|
||||
export interface PlaywrightMcpOptions {
|
||||
headless: boolean; // default true
|
||||
browser?: string; // "chrome" | "firefox" | "webkit" | "msedge"
|
||||
extraArgs?: string[]; // freeform passthrough
|
||||
logger: Logger;
|
||||
}
|
||||
|
||||
export interface PlaywrightMcpBridge {
|
||||
tools: CustomTools; // ready to concat into main.ts customTools
|
||||
dispose: () => Promise<void>; // closes MCP client + kills subprocess
|
||||
}
|
||||
|
||||
export async function createPlaywrightMcpBridge(opts: PlaywrightMcpOptions): Promise<PlaywrightMcpBridge>;
|
||||
```
|
||||
|
||||
Internally:
|
||||
|
||||
1. Compute argv: `["@playwright/mcp"]` plus `--headless` (if `headless`), `--browser <opts.browser>` (if set), plus `opts.extraArgs`. The binary is resolved from the local `node_modules/.bin` after we add `@playwright/mcp` as a dep, so no `npx` at runtime.
|
||||
2. Instantiate `StdioClientTransport` from `@modelcontextprotocol/sdk/client/stdio.js` with `{ command, args }`. Its stderr is piped to `opts.logger.warn` so browser-launch failures surface in our own logs.
|
||||
3. Instantiate `Client` from `@modelcontextprotocol/sdk/client/index.js`, `await client.connect(transport)`.
|
||||
4. `const { tools } = await client.listTools()`.
|
||||
5. For each MCP tool, synthesize a Pi tool:
|
||||
```ts
|
||||
defineTool({
|
||||
name: t.name, // e.g. "browser_navigate"
|
||||
label: t.annotations?.title ?? t.name,
|
||||
description: t.description ?? "",
|
||||
parameters: Type.Unsafe<Record<string, unknown>>(t.inputSchema ?? { type: "object" }),
|
||||
execute: async (_id, params) => {
|
||||
const result = await client.callTool({ name: t.name, arguments: params });
|
||||
return { content: result.content, details: {} };
|
||||
},
|
||||
})
|
||||
```
|
||||
`Type.Unsafe` lets us hand Pi the MCP-provided JSON Schema verbatim without re-encoding — Pi ships it straight to the model.
|
||||
6. Return `{ tools, dispose }` where `dispose` awaits `client.close()` (which also terminates the child process spawned by the transport).
|
||||
|
||||
### Wiring in `main.ts`
|
||||
|
||||
Next to the calendar block:
|
||||
|
||||
```ts
|
||||
let playwrightDispose: (() => Promise<void>) | undefined;
|
||||
if (config.playwright?.enabled) {
|
||||
try {
|
||||
const bridge = await createPlaywrightMcpBridge({
|
||||
headless: config.playwright.headless,
|
||||
browser: config.playwright.browser,
|
||||
extraArgs: config.playwright.extraArgs,
|
||||
logger,
|
||||
});
|
||||
customTools.push(...bridge.tools);
|
||||
playwrightDispose = bridge.dispose;
|
||||
logger.info(`playwright tools enabled (${bridge.tools.length} tools)`);
|
||||
} catch (err) {
|
||||
logger.error("playwright bridge failed to start, continuing without browser tools", err);
|
||||
}
|
||||
} else {
|
||||
logger.info("playwright tools disabled (set PLAYWRIGHT_ENABLED=true to enable)");
|
||||
}
|
||||
```
|
||||
|
||||
Shutdown path awaits `playwrightDispose?.()` alongside the existing `host?.dispose()` call.
|
||||
|
||||
## Configuration
|
||||
|
||||
New optional block on the parsed config, populated from env in `src/config.ts`:
|
||||
|
||||
| Env var | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `PLAYWRIGHT_ENABLED` | `false` | Master switch. Bridge only starts when `true`/`1`/`yes`. |
|
||||
| `PLAYWRIGHT_HEADLESS` | `true` | When `false`, we omit `--headless` so the browser is headed. |
|
||||
| `PLAYWRIGHT_BROWSER` | *(unset → MCP default: chrome)* | Passed as `--browser <value>`. Accepts `chrome`, `firefox`, `webkit`, `msedge`. |
|
||||
| `PLAYWRIGHT_EXTRA_ARGS` | *(empty)* | Shell-style split on whitespace and appended to argv. Escape hatch for flags we haven't first-classed (`--device "iPhone 15"`, `--viewport-size 1280,720`, `--isolated`, etc.). |
|
||||
|
||||
Documented in `README.md` and `.env.example`.
|
||||
|
||||
## Docker
|
||||
|
||||
The current `node:24-bookworm-slim` base has no Chromium. Add a step to `Dockerfile` after `npm run build && npm prune --omit=dev` and before the Claude Code CLI install:
|
||||
|
||||
```dockerfile
|
||||
# Playwright + Chromium and its OS dependencies. Included unconditionally so
|
||||
# PLAYWRIGHT_ENABLED=true works out of the box; adds ~300 MB to the image.
|
||||
RUN npx -y playwright install --with-deps chromium
|
||||
```
|
||||
|
||||
Only Chromium — Firefox/WebKit pull additional deps and aren't the default browser. Users who want them can rebuild with an extended Dockerfile.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Add to `package.json` `dependencies`:
|
||||
|
||||
- `@modelcontextprotocol/sdk` — MCP client + stdio transport.
|
||||
- `@playwright/mcp` — the server itself, pinned. Removes `npx` fetch on every startup and gives us deterministic upgrades.
|
||||
|
||||
Neither library needs to be listed as `optionalDependencies`; they're small and always installed, and the runtime cost is only paid when `PLAYWRIGHT_ENABLED=true`.
|
||||
|
||||
## Failure mode
|
||||
|
||||
- MCP server exits non-zero within a short window of `client.connect`: `createPlaywrightMcpBridge` throws. `main.ts` catches, logs `logger.error`, and continues with the rest of the tools. Bot still boots.
|
||||
- A `browser_*` tool call fails after startup (e.g. navigation error): the MCP `callTool` response comes back with `isError: true`. We forward its content to the model as a tool result; the model sees the failure and can retry or report it. No process crash.
|
||||
- Subprocess dies mid-session (browser crash, OOM, MCP server exit): the stdio transport is closed and every subsequent tool call throws. v1 does NOT auto-restart — the model surfaces the error, the user has to restart the bot process to get browser tools back. Auto-restart / supervision can be added later if this happens often enough to matter.
|
||||
- Shutdown races: `dispose()` is idempotent and safe to call after failure.
|
||||
|
||||
## Schema translation
|
||||
|
||||
Pi's `defineTool` requires a Typebox schema. MCP tools ship JSON Schema in `tool.inputSchema`. The `typebox` package already listed in `dependencies` exposes `Type.Unsafe<T>(schema)`, which wraps an arbitrary JSON Schema object as a Typebox schema without conversion. Pi serializes it back to JSON Schema when handing it to the model, so the model sees exactly what the MCP server documented. This is the whole reason we don't need to hand-map ~60 tool schemas.
|
||||
|
||||
## Tests
|
||||
|
||||
New `src/agent/playwright-mcp.test.ts`:
|
||||
|
||||
1. **Tool wrapping** — pass a fake MCP `Client` (in-memory, no subprocess) that returns two synthetic tools from `listTools()`. Assert `createPlaywrightMcpBridge` returns two Pi tools with matching names, descriptions, and `Type.Unsafe`-wrapped schemas.
|
||||
2. **Execute forwards** — call one wrapped tool's `execute({...})`. Assert the fake client saw `callTool({ name, arguments })` with those params. Assert the returned content matches the fake's response.
|
||||
3. **Dispose closes** — call `dispose()`. Assert `client.close()` was called exactly once.
|
||||
|
||||
The subprocess-spawning code path is factored so the client can be injected; the actual `StdioClientTransport` + `spawn` is only exercised in the manual E2E below.
|
||||
|
||||
Not in CI: real browser launch. Playwright + Chromium in the CI image is out of scope for unit tests.
|
||||
|
||||
### Manual E2E checklist
|
||||
|
||||
Append to the README manual list:
|
||||
|
||||
- Set `PLAYWRIGHT_ENABLED=true`, restart. Startup log shows `playwright tools enabled (N tools)` with N > 20.
|
||||
- Ask the bot: *"Open https://example.com and tell me the H1 text."* Expected: reply with `Example Domain`.
|
||||
- Ask: *"Take a screenshot of https://example.com and send it to me."* Expected: image attachment.
|
||||
- Ask: *"Run this on example.com and tell me the result: `async (page) => { await page.goto('https://example.com'); return await page.title(); }`"*. Expected: `Example Domain`. Exercises `browser_run_code_unsafe`.
|
||||
- Set `PLAYWRIGHT_ENABLED=false` (or unset), restart. Startup log shows the disabled message, no browser tools in the tool list.
|
||||
|
||||
## What we're not doing
|
||||
|
||||
- Filtering / renaming the MCP tool surface.
|
||||
- Custom retry / supervision of the MCP subprocess.
|
||||
- Non-Chromium browsers pre-installed in the default image.
|
||||
- HTTP transport.
|
||||
- Anything user-facing in Telegram beyond the tools existing — no `/browser` command, no separate approval UX.
|
||||
Reference in New Issue
Block a user