fix(a2a-client): watches no longer hang on a silently dead task stream

Hour-long remote tasks were never delivered: a subscribe watch held one SSE stream for up to a day, the remote
sent nothing while its agent worked without text, and once the quiet connection died on the way (no FIN/RST) the
watch waited on it with no error and no log line.

- A subscription lasts at most 5 minutes (SubscriptionSlice), then GetTask and a new subscription: a pause, not a
  failure. A slice ended mid-read counts as ours even when it surfaces as an I/O error.
- The shared handler connects with TCP keepalive (60 s, every 15 s, 4 probes) and HTTP/2 pings.
- Log lines when a watch starts or is restored, when a slice ends on a running task, when a result is delivered or
  kept; the Telegram host's console log carries a UTC timestamp.
- action=status on a settled task ends its watch, so the result isn't delivered a second time.
- A "context busy" refusal of send says nothing was sent, names the watched task in that context and that its
  result will arrive by itself; refusals never touch a watch.
This commit is contained in:
EugeneTes
2026-10-16 13:48:32 +00:00
parent 644f72068c
commit f42b00cb1c
9 changed files with 238 additions and 22 deletions

View File

@@ -260,7 +260,7 @@ One tool, `a2a`, with `action` and, depending on it, `agent`, `message`, `contex
|---|---|
| `agents` | For every entry, fetches the card (`CardTimeoutSeconds`) and lists name, description, card name, `streaming`/`push` capabilities and skills (id, name, description, up to three examples), or `unreachable (<reason>)` |
| `send` | Sends `message` and/or `files` to `agent`; `contextId` continues an earlier conversation with it, `taskId` answers its question on that task |
| `status` | `GetTask` for `taskId`: same result shape as `send` |
| `status` | `GetTask` for `taskId`: same result shape as `send`; a settled task's watch ends (its outcome is in the conversation now, so it isn't delivered again) |
| `cancel` | Stops this chat's watch of `taskId`, then `CancelTask` |
**send** calls `SendMessage` with `returnImmediately: true` and `metadata["volcanic.hops"]` (see [Hops](#hops)), then
@@ -314,7 +314,10 @@ because the task already finished), the result says so and carries the task's ou
or doesn't answer in time, the watch is re-armed and the error ends with `The task is still watched.`
Errors are error tool results with a readable message: unknown agent (with the configured names), `rejected the key
(401)`, `Could not reach agent ...`, `Agent ... refused: ... (<code>)`, `did not answer in time`. Every single remote
(401)`, `Could not reach agent ...`, `Agent ... refused: ... (<code>)`, `did not answer in time`. A `send` with a
`contextId` that the remote refuses as busy (`UnsupportedOperation` mentioning "busy", as ours and VolcanicAgents do)
says that nothing was sent, names the task this chat watches in that context (`its result will arrive here by
itself`) and suggests sending again after it or without `contextId`. A refusal never touches a watch. Every single remote
call is bounded by 60 seconds; an agent that ignores `returnImmediately` and needs longer than that to answer
`SendMessage` gets `did not answer in time`, and since its task id never arrived, nothing watches that task.
@@ -333,14 +336,20 @@ How it learns about updates, chosen when the watch starts:
body is never trusted. A safety `GetTask` runs every 10 minutes. If the remote refuses the push config, the watch
subscribes or polls instead.
2. **subscribe**: the card advertises streaming. A stream that drops or ends before the task settles is reopened after
1 s, 5 s, then 15 s; after 3 failures in a row the watch polls instead. A stream is also reopened after a day.
1 s, 5 s, then 15 s; after 3 failures in a row the watch polls instead. No stream is held for more than 5 minutes:
the watch then reads the task with `GetTask` and subscribes again (a pause, not a failure). Remotes send nothing
while their agent works without writing text, and such a quiet stream can die on the way (a proxy or NAT dropping
it) without either side noticing; before this slice a watch could wait on a dead stream until `WatchTimeout`.
3. **poll**: `GetTask` after 15 s, 30 s, 1 min, 2 min, then every 5 min.
Each check reads `agents.json` again: an agent removed from it ends the watch with a `(stopped)` notice; a malformed
file is treated as a passing failure (the user may be editing it) and the watch keeps going. A watch keeps its
connection (card and client) between checks and reconnects after a failure or when the agent's entry changes. After
`WatchTimeout` (1 day by default) from its start the watch gives up with a `(no result)` notice suggesting
`action=status`. On a settled task it saves files like `send` does and, for push watches, deletes the push config
`action=status`. The shared handler turns on TCP keepalive (probes after 60 s of silence, every 15 s, 4 tries) and
HTTP/2 pings, so a dead connection fails instead of hanging. The log has one line when a watch starts or is restored
(task, agent, chat, source, watch id), when a stream slice ends on a running task (`subscribing again`), when a
result is delivered or kept, and for every failed check. On a settled task it saves files like `send` does and, for push watches, deletes the push config
(best effort).
The delivered text is a follow-up prompt:

View File

@@ -102,6 +102,7 @@ The compose file maps them into the container.
`TELEGRAM_MODEL` and `LOG_LEVEL` are read by the compose file only, which passes them to the bot as `Telegram__Model`
and `Logging__LogLevel__Default`; `INSTALL_PLAYWRIGHT` and `CLAUDE_CODE_VERSION` are build arguments (see above).
Every log line starts with its UTC time (`2026-10-16 13:48:00 UTC info: ...`, `Logging:Console` in `appsettings.json`).
> **Never set `ANTHROPIC_API_KEY`, `ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_BASE_URL`, `CLAUDE_CODE_OAUTH_TOKEN`,
> `CLAUDE_CODE_USE_BEDROCK` or `CLAUDE_CODE_USE_VERTEX`.** The Claude Code provider runs on a Claude subscription only

View File

@@ -101,10 +101,13 @@ Arguments: `action` (`agents` \| `send` \| `status` \| `cancel`), `agent`, `mess
- still running → starts a watch, returns `{state: working, taskId, contextId, note: "the result will be delivered to
this conversation when it is ready"}`.
A message-only reply (the agent answered with a `Message`, no task) is returned as text with its `contextId`.
- **`status`** `{agent, taskId}` — `GetTask`; same result shape as a finished `send`.
- **`status`** `{agent, taskId}` — `GetTask`; same result shape as a finished `send`. A settled task's watch ends, so its
outcome isn't delivered a second time.
- **`cancel`** `{agent, taskId}` — `CancelTask`, removes any watch for it.
Errors (network, timeout, 401, JSON-RPC error, unknown agent) become error tool results with a readable message. The
Errors (network, timeout, 401, JSON-RPC error, unknown agent) become error tool results with a readable message. A
"context busy" refusal of a `send` with `contextId` names the task this chat watches there and says its result will
arrive by itself; a refusal never touches a watch. The
tool never throws into the loop.
## Watches
@@ -125,7 +128,9 @@ Watch record, persisted in `<workspace>/.volcanic/a2a/pending.json`:
1. **push**: card has `capabilities.pushNotifications` and `PublicUrl` is set → `CreateTaskPushNotificationConfig`
with URL `<PublicUrl>/a2a/push/<watchId>` and a random 32-byte token. A safety `GetTask` runs every 10 min.
2. **subscribe**: card has `streaming` → `SubscribeToTask`; on disconnect, reconnect with backoff, falling back to polling
after 3 failures.
after 3 failures. A stream is held at most 5 minutes, then `GetTask` and a new subscription (a pause, not a
failure): a quiet stream may have died silently on the way (2026-10-16: hour-long tasks never delivered). The
shared handler uses TCP keepalive and HTTP/2 pings.
3. **poll**: `GetTask` with backoff 15 s → 30 s → 1 min → 2 min → 5 min (cap).
- **Push receiving**: the host wires the server's `A2AServerOptions.OnPush` to `A2AWatchRegistry.Shared.OnPushAsync(watchId,
token, body)`, which checks the token (constant-time) and returns false for unknown watches or bad tokens (→ 404).