docs(spec): pinned node data (n8n-style pin for manual test runs)
This commit is contained in:
Claude
162
docs/superpowers/specs/2026-08-30-pinned-node-data-design.md
Normal file
162
docs/superpowers/specs/2026-08-30-pinned-node-data-design.md
Normal file
@@ -0,0 +1,162 @@
|
||||
# VolcanicCore — Pinned Node Data (n8n-style) Design
|
||||
|
||||
**Status:** Approved for planning
|
||||
**Date:** 2026-08-30
|
||||
|
||||
## Purpose
|
||||
|
||||
Give operators n8n-style "pinned data" on individual nodes so a manual test run substitutes real received/emitted data with an operator-provided JSON blob. Speeds up iteration on downstream nodes without re-running slow upstream steps, and lets operators reproduce a specific past execution's data.
|
||||
|
||||
The pin editor lives inside the existing `NodeEditorDialog` (Plan 5). Pinned data is persisted on the workflow's graph JSON — no new DB table, no new endpoint.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Production runs.** Only manual/test runs honor pins. Webhook events and NATS messages ignore pins. Prevents stale pins from silently corrupting live traffic.
|
||||
- **Skipping upstream node execution.** If node B has a pinned_input and A → B, A still runs. Its output is simply discarded for B's byPort. Skipping upstream is a much larger executor rewrite (dependency analysis, dead branches, error propagation) and out of scope.
|
||||
- **Simultaneous input + output pinning on the same node.** Action nodes have both ports; this plan only exposes the input pin editor for action/terminal nodes and the output pin editor for trigger nodes. A future extension could add the other side.
|
||||
- **Multi-port pinning.** Only port 0 is pinnable. All shipped nodes have a single input port; multi-port pinning is a follow-up.
|
||||
- **Pin sharing / templates.** No copy-pin-between-nodes UI. Manual JSON copy-paste is fine.
|
||||
|
||||
## 1. Architecture
|
||||
|
||||
Frontend + one small backend change. Pinned values live inside the existing `workflows.graph_json` blob (piggybacks the `PUT /api/workflows/{id}` save). Executor consults them on manual runs only.
|
||||
|
||||
```
|
||||
GraphNode { # existing shape, two new fields
|
||||
id, type_id, position, config, connection_id,
|
||||
pinned_input: null | JsonValue, # NEW — action/terminal only
|
||||
pinned_output: null | JsonValue, # NEW — trigger only
|
||||
}
|
||||
|
||||
Executor.RunAsync(triggerKind, graph, triggerItems):
|
||||
if trigger.pinned_output && triggerKind == "manual":
|
||||
triggerItems = wrap(trigger.pinned_output)
|
||||
# ...existing trigger row + BFS setup...
|
||||
|
||||
Executor.RunNodeAsync(node, byPort, ...):
|
||||
if node.pinned_input && triggerKind == "manual":
|
||||
byPort = [ wrap(node.pinned_input) ]
|
||||
# ...existing InterpolateStrings → CoerceToSchema → ExecuteAsync...
|
||||
```
|
||||
|
||||
## 2. Wire format
|
||||
|
||||
Both `pinned_input` and `pinned_output` are stored as raw JSON on `GraphNode`. Absent / `null` → no pin. Present value's shape determines how the executor wraps it into port items:
|
||||
|
||||
| Pin value | Wrapped as (port 0) |
|
||||
|---------------------------------|----------------------------------------------|
|
||||
| `[{a:1}, {b:2}]` (array) | 2 items: `[{json:{a:1}}, {json:{b:2}}]` |
|
||||
| `{a:1}` (object) | 1 item: `[{json:{a:1}}]` |
|
||||
| `"hello"` / `42` / `true` (scalar) | 1 item: `[{json:"hello"}]` etc. |
|
||||
| `[]` (empty array) | 0 items: `[]` — valid; means "empty port" |
|
||||
|
||||
`binary` is always `null` on wrapped items (pinned data is JSON-only).
|
||||
|
||||
Trigger pins bypass the endpoint's `triggerData` parameter — the endpoint keeps working exactly as today for backward compat, but the pin wins on manual runs when set.
|
||||
|
||||
## 3. UI
|
||||
|
||||
**NodeEditorDialog** gets a new **Pinned input** OR **Pinned output** section in the Parameters column, above the WebhookUrlHint / footer buttons area. Which section appears depends on the node's category:
|
||||
|
||||
- **Trigger nodes** → `Pinned output` editor.
|
||||
- **Action / Terminal nodes** → `Pinned input` editor.
|
||||
|
||||
Section contents:
|
||||
|
||||
```
|
||||
Pinned <input|output>
|
||||
[ Load from execution: [latest ▼] ] [ Clear ]
|
||||
┌───────────────────────────────────────────────────────┐
|
||||
│ { "email": "alice@example.com" } │
|
||||
│ │
|
||||
│ │
|
||||
└───────────────────────────────────────────────────────┘
|
||||
(none — this node runs on real upstream data) ← placeholder when empty
|
||||
⚠ Invalid JSON: unexpected token … ← inline parse error
|
||||
```
|
||||
|
||||
- Monospace textarea, ~10 rows.
|
||||
- **State model:** parent `VcNodeData` holds the PARSED value (`pinnedInput?: unknown | null`). The `PinnedDataSection` child holds LOCAL raw-text state initialized from `JSON.stringify(parsed, null, 2)`. On every keystroke, the section tries `JSON.parse(text)`:
|
||||
- **Parse succeeds:** emit `onPatch({ pinnedInput: parsed })` (or `pinnedOutput`); clear any error.
|
||||
- **Parse fails:** DON'T emit — keep parent state unchanged. Show inline error `⚠ Invalid JSON: <message>`.
|
||||
- **Empty text (whitespace only):** emit `onPatch({ pinnedInput: null })`; clear error. This is the "no pin" state.
|
||||
- Toolbar **Save** does NOT need to know about pin parse errors — the parent state only ever has valid parsed values (or null). If the user leaves the dialog with invalid text, the last valid parse is what's persisted; a red banner in the dialog reminds them to fix or clear.
|
||||
- **Load from execution: [dropdown]** — lists the last 10 executions for the workflow (timestamp + status + trigger kind). Selecting one writes `execution.nodes.find(nodeId===n).<inputs|outputs>[0]` into the textarea as pretty-printed JSON (same parse-and-emit flow — this always parses cleanly since it came from the backend). The dropdown resets to "latest" after selection. Uses `inputs[0]` for action/terminal, `outputs[0]` for trigger.
|
||||
- **Clear** empties the textarea (equivalent to save-with-empty → null).
|
||||
|
||||
**Canvas indicator (customNode.tsx):** a small pin/anchor icon in the node's header when either `pinnedInput != null` OR `pinnedOutput != null`. Same icon regardless of which side.
|
||||
|
||||
**Dialog Run button:** unchanged behavior. Kicks a manual run from the first trigger. That run will honor pins.
|
||||
|
||||
## 4. Executor behavior
|
||||
|
||||
`Executor.RunAsync` (see `src/VolcanicCore.Api/Engine/Executor.cs`) takes `triggerKind` today. Two changes:
|
||||
|
||||
1. **Trigger pin override.** After `graph.Nodes.FirstOrDefault(n => n.Id == triggerNodeId)`, if the trigger's `PinnedOutputJson` is non-null AND `triggerKind == "manual"`, replace `triggerItems` with `WrapPin(pinnedOutput)` before recording the trigger row.
|
||||
2. **Per-node input pin.** Inside the main scheduler loop, right before the `running[RunNodeAsync(...)] = nid` line, if the node's `PinnedInputJson` is non-null AND `triggerKind == "manual"`, override `byPort` with `[ WrapPin(pinnedInput) ]` (single-port). Then call `RunNodeAsync` as normal.
|
||||
|
||||
`WrapPin(jsonElement)`:
|
||||
|
||||
```csharp
|
||||
private static IReadOnlyList<Item> WrapPin(JsonElement pin) => pin.ValueKind switch {
|
||||
JsonValueKind.Array => pin.EnumerateArray()
|
||||
.Select(el => new Item(el.Clone()))
|
||||
.ToArray(),
|
||||
_ => new[] { new Item(pin.Clone()) },
|
||||
};
|
||||
```
|
||||
|
||||
**Interpolation and coercion still apply after pinning.** `InterpolateStrings` uses `byPort[0][0].Json` as the template input, so pinned-input templates evaluate against the pinned value (correct — that IS what the node "sees"). `CoerceToSchema` runs unchanged.
|
||||
|
||||
**Failure propagation unchanged.** If an upstream node fails, downstream `skipped_due_to_failure` behavior is untouched. Pins only affect Data arrivals, never Failure arrivals.
|
||||
|
||||
## 5. Persistence
|
||||
|
||||
`WorkflowGraph.Parse` currently reads `nodes[].config`, `type_id`, `connection_id`, `position`. Extend to also read `pinned_input` and `pinned_output` as raw JSON text (like `config`). Store on `WorkflowNode` as `PinnedInputJson: string?` and `PinnedOutputJson: string?` (nullable when absent or explicit `null`).
|
||||
|
||||
`GET /api/workflows/{id}` returns the graph as stored; if the SPA never populates the fields, they stay absent — round-trip safe.
|
||||
|
||||
`PUT /api/workflows/{id}` accepts the graph as sent; new fields flow through the existing JSON serializer without ceremony.
|
||||
|
||||
No migration.
|
||||
|
||||
## 6. Frontend flow
|
||||
|
||||
- `frontend/src/api/types.ts` — `GraphNode` gains `pinned_input?: unknown | null; pinned_output?: unknown | null`.
|
||||
- `frontend/src/editor/graphAdapter.ts` — `fromGraph` copies fields into `VcNodeData`; `toGraph` writes them back. `coerceConfigForSave` runs only against `config`, not the pin fields.
|
||||
- `frontend/src/editor/NodeEditorDialog.tsx` — new `PinnedDataSection` child renders per node category. Owns local textarea state (raw string) and parse-error state. Emits `onPatch` on every successful keystroke parse.
|
||||
- `frontend/src/pages/EditorPage.tsx` — no change to the save mutation. Parent state always holds valid parsed values (or null) so `toGraph` serializes them without ceremony.
|
||||
- `frontend/src/editor/customNode.tsx` — add `Pin` icon (lucide-react `Pin` or similar) when `data.pinnedInput != null || data.pinnedOutput != null`.
|
||||
|
||||
## 7. Tests
|
||||
|
||||
**Backend (xUnit, `tests/VolcanicCore.Tests/ExecutorTests.cs`):**
|
||||
1. `Trigger_pinned_output_overrides_trigger_items_on_manual_run` — trigger has pinned_output `{a:1}`; manual run's downstream receives `{a:1}` on port 0.
|
||||
2. `Trigger_pinned_output_is_ignored_on_non_manual_run` — same graph, triggerKind `webhook`; downstream receives the `triggerItems` passed to the endpoint.
|
||||
3. `Node_pinned_input_overrides_upstream_output_on_manual_run` — chain `t → a → sink`; a has pinned_input `{x:9}`; sink receives `{x:9}` (not a's real output).
|
||||
4. `Node_pinned_input_is_ignored_on_non_manual_run` — same chain, triggerKind `webhook`; sink receives a's real output.
|
||||
5. `Pinned_input_array_wraps_to_multiple_items` — array-shaped pin becomes multiple items on port 0.
|
||||
6. `Pinned_input_object_wraps_to_single_item` — object-shaped pin becomes single item.
|
||||
|
||||
**Frontend (Vitest):**
|
||||
1. `graphAdapter.test.ts` — new tests: fromGraph/toGraph round-trips `pinned_input` and `pinned_output` unchanged.
|
||||
2. `NodeEditorDialog.test.tsx` — new tests:
|
||||
- Trigger nodes render "Pinned output" editor (not "Pinned input")
|
||||
- Action nodes render "Pinned input" editor (not "Pinned output")
|
||||
- Load-from-execution dropdown fills the textarea with the right side (inputs[0] for action, outputs[0] for trigger)
|
||||
- Clear button empties the textarea
|
||||
- Invalid JSON in textarea → inline error visible on the field, `onPatch` not called (parent state unchanged), last valid parsed value preserved
|
||||
- Valid JSON in textarea → inline error clears, `onPatch({ pinnedInput: parsed })` fires
|
||||
- Whitespace-only textarea → `onPatch({ pinnedInput: null })` fires
|
||||
3. `customNode.test.tsx` — new file OR add tests: pin icon renders when either pinned field is set.
|
||||
|
||||
**Playwright smoke:** extend `frontend/e2e/smoke.spec.ts` — after saving the workflow, double-click the node, type valid JSON into the Pinned input textarea, save, reload, re-open dialog, assert JSON persisted.
|
||||
|
||||
## 8. Out of scope (follow-ups)
|
||||
|
||||
- Skipping upstream node execution when a downstream node has a pin (dependency analysis).
|
||||
- Pinning both sides on the same action node.
|
||||
- Multi-port pinning.
|
||||
- Pin sharing / templates / bulk copy.
|
||||
- Visual "diff" indicator in the dialog: what the node WOULD have received vs. what the pin says it will receive.
|
||||
- Backend index/column for querying "workflows with any pinned data" (moot until we care about listing them).
|
||||
Reference in New Issue
Block a user