VolcanicCore/docs/superpowers/specs/2026-08-30-pinned-node-data-design.md

VolcanicCore — Pinned Node Data (n8n-style) Design

Status: Approved for planning Date: 2026-08-30

Purpose

Give operators n8n-style "pinned data" on individual nodes so a manual test run substitutes real received/emitted data with an operator-provided JSON blob. Speeds up iteration on downstream nodes without re-running slow upstream steps, and lets operators reproduce a specific past execution's data.

The pin editor lives inside the existing NodeEditorDialog (Plan 5). Pinned data is persisted on the workflow's graph JSON — no new DB table, no new endpoint.

Non-goals

• Production runs. Only manual/test runs honor pins. Webhook events and NATS messages ignore pins. Prevents stale pins from silently corrupting live traffic.
• Skipping upstream node execution. If node B has a pinned_input and A → B, A still runs. Its output is simply discarded for B's byPort. Skipping upstream is a much larger executor rewrite (dependency analysis, dead branches, error propagation) and out of scope.
• Simultaneous input + output pinning on the same node. Action nodes have both ports; this plan only exposes the input pin editor for action/terminal nodes and the output pin editor for trigger nodes. A future extension could add the other side.
• Multi-port pinning. Only port 0 is pinnable. All shipped nodes have a single input port; multi-port pinning is a follow-up.
• Pin sharing / templates. No copy-pin-between-nodes UI. Manual JSON copy-paste is fine.

1. Architecture

Frontend + one small backend change. Pinned values live inside the existing workflows.graph_json blob (piggybacks the PUT /api/workflows/{id} save). Executor consults them on manual runs only.

GraphNode {                    # existing shape, two new fields
  id, type_id, position, config, connection_id,
  pinned_input:  null | JsonValue,   # NEW — action/terminal only
  pinned_output: null | JsonValue,   # NEW — trigger only
}

Executor.RunAsync(triggerKind, graph, triggerItems):
  if trigger.pinned_output && triggerKind == "manual":
    triggerItems = wrap(trigger.pinned_output)
  # ...existing trigger row + BFS setup...

Executor.RunNodeAsync(node, byPort, ...):
  if node.pinned_input && triggerKind == "manual":
    byPort = [ wrap(node.pinned_input) ]
  # ...existing InterpolateStrings → CoerceToSchema → ExecuteAsync...

2. Wire format

Both pinned_input and pinned_output are stored as raw JSON on GraphNode. Absent / null → no pin. Present value's shape determines how the executor wraps it into port items:

Pin value	Wrapped as (port 0)
[{a:1}, {b:2}] (array)	2 items: [{json:{a:1}}, {json:{b:2}}]
{a:1} (object)	1 item: [{json:{a:1}}]
"hello" / 42 / true (scalar)	1 item: [{json:"hello"}] etc.
[] (empty array)	0 items: [] — valid; means "empty port"

binary is always null on wrapped items (pinned data is JSON-only).

Trigger pins bypass the endpoint's triggerData parameter — the endpoint keeps working exactly as today for backward compat, but the pin wins on manual runs when set.

3. UI

NodeEditorDialog gets a new Pinned input OR Pinned output section in the Parameters column, above the WebhookUrlHint / footer buttons area. Which section appears depends on the node's category:

• Trigger nodes → Pinned output editor.
• Action / Terminal nodes → Pinned input editor.

Section contents:

Pinned <input|output>
[ Load from execution: [latest ▼] ]        [ Clear ]
┌───────────────────────────────────────────────────────┐
│ { "email": "alice@example.com" }                      │
│                                                       │
│                                                       │
└───────────────────────────────────────────────────────┘
(none — this node runs on real upstream data)   ← placeholder when empty
⚠ Invalid JSON: unexpected token …               ← inline parse error

• Monospace textarea, ~10 rows.
• State model: parent VcNodeData holds the PARSED value (pinnedInput?: unknown | null). The PinnedDataSection child holds LOCAL raw-text state initialized from JSON.stringify(parsed, null, 2). On every keystroke, the section tries JSON.parse(text):
  • Parse succeeds: emit onPatch({ pinnedInput: parsed }) (or pinnedOutput); clear any error.
  • Parse fails: DON'T emit — keep parent state unchanged. Show inline error ⚠ Invalid JSON: <message>.
  • Empty text (whitespace only): emit onPatch({ pinnedInput: null }); clear error. This is the "no pin" state.
• Toolbar Save does NOT need to know about pin parse errors — the parent state only ever has valid parsed values (or null). If the user leaves the dialog with invalid text, the last valid parse is what's persisted; a red banner in the dialog reminds them to fix or clear.
• Load from execution: [dropdown] — lists the last 10 executions for the workflow (timestamp + status + trigger kind). Selecting one writes execution.nodes.find(nodeId===n).<inputs|outputs>[0] into the textarea as pretty-printed JSON (same parse-and-emit flow — this always parses cleanly since it came from the backend). The dropdown resets to "latest" after selection. Uses inputs[0] for action/terminal, outputs[0] for trigger.
• Clear empties the textarea (equivalent to save-with-empty → null).

Canvas indicator (customNode.tsx): a small pin/anchor icon in the node's header when either pinnedInput != null OR pinnedOutput != null. Same icon regardless of which side.

Dialog Run button: unchanged behavior. Kicks a manual run from the first trigger. That run will honor pins.

4. Executor behavior

Executor.RunAsync (see src/VolcanicCore.Api/Engine/Executor.cs) takes triggerKind today. Two changes:

1. Trigger pin override. After graph.Nodes.FirstOrDefault(n => n.Id == triggerNodeId), if the trigger's PinnedOutputJson is non-null AND triggerKind == "manual", replace triggerItems with WrapPin(pinnedOutput) before recording the trigger row.
2. Per-node input pin. Inside the main scheduler loop, right before the running[RunNodeAsync(...)] = nid line, if the node's PinnedInputJson is non-null AND triggerKind == "manual", override byPort with [ WrapPin(pinnedInput) ] (single-port). Then call RunNodeAsync as normal.

WrapPin(jsonElement):

private static IReadOnlyList<Item> WrapPin(JsonElement pin) => pin.ValueKind switch {
    JsonValueKind.Array  => pin.EnumerateArray()
                              .Select(el => new Item(el.Clone()))
                              .ToArray(),
    _                    => new[] { new Item(pin.Clone()) },
};

Interpolation and coercion still apply after pinning. InterpolateStrings uses byPort[0][0].Json as the template input, so pinned-input templates evaluate against the pinned value (correct — that IS what the node "sees"). CoerceToSchema runs unchanged.

Failure propagation unchanged. If an upstream node fails, downstream skipped_due_to_failure behavior is untouched. Pins only affect Data arrivals, never Failure arrivals.

5. Persistence

WorkflowGraph.Parse currently reads nodes[].config, type_id, connection_id, position. Extend to also read pinned_input and pinned_output as raw JSON text (like config). Store on WorkflowNode as PinnedInputJson: string? and PinnedOutputJson: string? (nullable when absent or explicit null).

GET /api/workflows/{id} returns the graph as stored; if the SPA never populates the fields, they stay absent — round-trip safe.

PUT /api/workflows/{id} accepts the graph as sent; new fields flow through the existing JSON serializer without ceremony.

No migration.

6. Frontend flow

• frontend/src/api/types.ts — GraphNode gains pinned_input?: unknown | null; pinned_output?: unknown | null.
• frontend/src/editor/graphAdapter.ts — fromGraph copies fields into VcNodeData; toGraph writes them back. coerceConfigForSave runs only against config, not the pin fields.
• frontend/src/editor/NodeEditorDialog.tsx — new PinnedDataSection child renders per node category. Owns local textarea state (raw string) and parse-error state. Emits onPatch on every successful keystroke parse.
• frontend/src/pages/EditorPage.tsx — no change to the save mutation. Parent state always holds valid parsed values (or null) so toGraph serializes them without ceremony.
• frontend/src/editor/customNode.tsx — add Pin icon (lucide-react Pin or similar) when data.pinnedInput != null || data.pinnedOutput != null.

7. Tests

Backend (xUnit, tests/VolcanicCore.Tests/ExecutorTests.cs):

1. Trigger_pinned_output_overrides_trigger_items_on_manual_run — trigger has pinned_output {a:1}; manual run's downstream receives {a:1} on port 0.
2. Trigger_pinned_output_is_ignored_on_non_manual_run — same graph, triggerKind webhook; downstream receives the triggerItems passed to the endpoint.
3. Node_pinned_input_overrides_upstream_output_on_manual_run — chain t → a → sink; a has pinned_input {x:9}; sink receives {x:9} (not a's real output).
4. Node_pinned_input_is_ignored_on_non_manual_run — same chain, triggerKind webhook; sink receives a's real output.
5. Pinned_input_array_wraps_to_multiple_items — array-shaped pin becomes multiple items on port 0.
6. Pinned_input_object_wraps_to_single_item — object-shaped pin becomes single item.

Frontend (Vitest):

1. graphAdapter.test.ts — new tests: fromGraph/toGraph round-trips pinned_input and pinned_output unchanged.
2. NodeEditorDialog.test.tsx — new tests:
   • Trigger nodes render "Pinned output" editor (not "Pinned input")
   • Action nodes render "Pinned input" editor (not "Pinned output")
   • Load-from-execution dropdown fills the textarea with the right side (inputs[0] for action, outputs[0] for trigger)
   • Clear button empties the textarea
   • Invalid JSON in textarea → inline error visible on the field, onPatch not called (parent state unchanged), last valid parsed value preserved
   • Valid JSON in textarea → inline error clears, onPatch({ pinnedInput: parsed }) fires
   • Whitespace-only textarea → onPatch({ pinnedInput: null }) fires
3. customNode.test.tsx — new file OR add tests: pin icon renders when either pinned field is set.

Playwright smoke: extend frontend/e2e/smoke.spec.ts — after saving the workflow, double-click the node, type valid JSON into the Pinned input textarea, save, reload, re-open dialog, assert JSON persisted.

8. Out of scope (follow-ups)

• Skipping upstream node execution when a downstream node has a pin (dependency analysis).
• Pinning both sides on the same action node.
• Multi-port pinning.
• Pin sharing / templates / bulk copy.
• Visual "diff" indicator in the dialog: what the node WOULD have received vs. what the pin says it will receive.
• Backend index/column for querying "workflows with any pinned data" (moot until we care about listing them).