Bridge @playwright/mcp as a stdio subprocess and re-export its tools as Pi custom tools, gated on PLAYWRIGHT_ENABLED. Uses Type.Unsafe to pass MCP inputSchema through to the model without hand-mapping.
8.7 KiB
Playwright browser control via MCP bridge
Goal
Give the agent the ability to drive a real browser (navigate, click, type, evaluate JS, take screenshots, watch network, run arbitrary Playwright code) by embedding the @playwright/mcp server as a subprocess and re-exposing every tool it publishes as a Pi custom tool.
Non-goals
- HTTP / SSE transport. Stdio only.
PLAYWRIGHT_MCP_URLfor "connect to a running server" can be added later without touching the tool-wrapping code. - Fine-grained tool allow/block lists. Every tool the MCP server reports is exposed. If the tool surface bloats the prompt too much, add filtering later.
- Multiple browser contexts / per-user profiles / storage-state persistence. The MCP server's default profile is fine for v1.
- Managing browser binary installation at runtime. Chromium is installed at image-build time via the Dockerfile; a missing browser at runtime is a hard startup failure of the bridge (see "Failure mode").
Architecture
New module src/agent/playwright-mcp.ts:
export interface PlaywrightMcpOptions {
headless: boolean; // default true
browser?: string; // "chrome" | "firefox" | "webkit" | "msedge"
extraArgs?: string[]; // freeform passthrough
logger: Logger;
}
export interface PlaywrightMcpBridge {
tools: CustomTools; // ready to concat into main.ts customTools
dispose: () => Promise<void>; // closes MCP client + kills subprocess
}
export async function createPlaywrightMcpBridge(opts: PlaywrightMcpOptions): Promise<PlaywrightMcpBridge>;
Internally:
- Compute argv:
["@playwright/mcp"]plus--headless(ifheadless),--browser <opts.browser>(if set), plusopts.extraArgs. The binary is resolved from the localnode_modules/.binafter we add@playwright/mcpas a dep, so nonpxat runtime. - Instantiate
StdioClientTransportfrom@modelcontextprotocol/sdk/client/stdio.jswith{ command, args }. Its stderr is piped toopts.logger.warnso browser-launch failures surface in our own logs. - Instantiate
Clientfrom@modelcontextprotocol/sdk/client/index.js,await client.connect(transport). const { tools } = await client.listTools().- For each MCP tool, synthesize a Pi tool:
defineTool({ name: t.name, // e.g. "browser_navigate" label: t.annotations?.title ?? t.name, description: t.description ?? "", parameters: Type.Unsafe<Record<string, unknown>>(t.inputSchema ?? { type: "object" }), execute: async (_id, params) => { const result = await client.callTool({ name: t.name, arguments: params }); return { content: result.content, details: {} }; }, })Type.Unsafelets us hand Pi the MCP-provided JSON Schema verbatim without re-encoding — Pi ships it straight to the model. - Return
{ tools, dispose }wheredisposeawaitsclient.close()(which also terminates the child process spawned by the transport).
Wiring in main.ts
Next to the calendar block:
let playwrightDispose: (() => Promise<void>) | undefined;
if (config.playwright?.enabled) {
try {
const bridge = await createPlaywrightMcpBridge({
headless: config.playwright.headless,
browser: config.playwright.browser,
extraArgs: config.playwright.extraArgs,
logger,
});
customTools.push(...bridge.tools);
playwrightDispose = bridge.dispose;
logger.info(`playwright tools enabled (${bridge.tools.length} tools)`);
} catch (err) {
logger.error("playwright bridge failed to start, continuing without browser tools", err);
}
} else {
logger.info("playwright tools disabled (set PLAYWRIGHT_ENABLED=true to enable)");
}
Shutdown path awaits playwrightDispose?.() alongside the existing host?.dispose() call.
Configuration
New optional block on the parsed config, populated from env in src/config.ts:
| Env var | Default | Purpose |
|---|---|---|
PLAYWRIGHT_ENABLED |
false |
Master switch. Bridge only starts when true/1/yes. |
PLAYWRIGHT_HEADLESS |
true |
When false, we omit --headless so the browser is headed. |
PLAYWRIGHT_BROWSER |
(unset → MCP default: chrome) | Passed as --browser <value>. Accepts chrome, firefox, webkit, msedge. |
PLAYWRIGHT_EXTRA_ARGS |
(empty) | Shell-style split on whitespace and appended to argv. Escape hatch for flags we haven't first-classed (--device "iPhone 15", --viewport-size 1280,720, --isolated, etc.). |
Documented in README.md and .env.example.
Docker
The current node:24-bookworm-slim base has no Chromium. Add a step to Dockerfile after npm run build && npm prune --omit=dev and before the Claude Code CLI install:
# Playwright + Chromium and its OS dependencies. Included unconditionally so
# PLAYWRIGHT_ENABLED=true works out of the box; adds ~300 MB to the image.
RUN npx -y playwright install --with-deps chromium
Only Chromium — Firefox/WebKit pull additional deps and aren't the default browser. Users who want them can rebuild with an extended Dockerfile.
Dependencies
Add to package.json dependencies:
@modelcontextprotocol/sdk— MCP client + stdio transport.@playwright/mcp— the server itself, pinned. Removesnpxfetch on every startup and gives us deterministic upgrades.
Neither library needs to be listed as optionalDependencies; they're small and always installed, and the runtime cost is only paid when PLAYWRIGHT_ENABLED=true.
Failure mode
- MCP server exits non-zero within a short window of
client.connect:createPlaywrightMcpBridgethrows.main.tscatches, logslogger.error, and continues with the rest of the tools. Bot still boots. - A
browser_*tool call fails after startup (e.g. navigation error): the MCPcallToolresponse comes back withisError: true. We forward its content to the model as a tool result; the model sees the failure and can retry or report it. No process crash. - Subprocess dies mid-session (browser crash, OOM, MCP server exit): the stdio transport is closed and every subsequent tool call throws. v1 does NOT auto-restart — the model surfaces the error, the user has to restart the bot process to get browser tools back. Auto-restart / supervision can be added later if this happens often enough to matter.
- Shutdown races:
dispose()is idempotent and safe to call after failure.
Schema translation
Pi's defineTool requires a Typebox schema. MCP tools ship JSON Schema in tool.inputSchema. The typebox package already listed in dependencies exposes Type.Unsafe<T>(schema), which wraps an arbitrary JSON Schema object as a Typebox schema without conversion. Pi serializes it back to JSON Schema when handing it to the model, so the model sees exactly what the MCP server documented. This is the whole reason we don't need to hand-map ~60 tool schemas.
Tests
New src/agent/playwright-mcp.test.ts:
- Tool wrapping — pass a fake MCP
Client(in-memory, no subprocess) that returns two synthetic tools fromlistTools(). AssertcreatePlaywrightMcpBridgereturns two Pi tools with matching names, descriptions, andType.Unsafe-wrapped schemas. - Execute forwards — call one wrapped tool's
execute({...}). Assert the fake client sawcallTool({ name, arguments })with those params. Assert the returned content matches the fake's response. - Dispose closes — call
dispose(). Assertclient.close()was called exactly once.
The subprocess-spawning code path is factored so the client can be injected; the actual StdioClientTransport + spawn is only exercised in the manual E2E below.
Not in CI: real browser launch. Playwright + Chromium in the CI image is out of scope for unit tests.
Manual E2E checklist
Append to the README manual list:
- Set
PLAYWRIGHT_ENABLED=true, restart. Startup log showsplaywright tools enabled (N tools)with N > 20. - Ask the bot: "Open https://example.com and tell me the H1 text." Expected: reply with
Example Domain. - Ask: "Take a screenshot of https://example.com and send it to me." Expected: image attachment.
- Ask: "Run this on example.com and tell me the result:
async (page) => { await page.goto('https://example.com'); return await page.title(); }". Expected:Example Domain. Exercisesbrowser_run_code_unsafe. - Set
PLAYWRIGHT_ENABLED=false(or unset), restart. Startup log shows the disabled message, no browser tools in the tool list.
What we're not doing
- Filtering / renaming the MCP tool surface.
- Custom retry / supervision of the MCP subprocess.
- Non-Chromium browsers pre-installed in the default image.
- HTTP transport.
- Anything user-facing in Telegram beyond the tools existing — no
/browsercommand, no separate approval UX.