feat(executor): {input.foo} template interpolation on node config strings

Every action/terminal node's config string fields are now run through
TemplateEngine.Render before the node sees them, so operators can pull
values from the incoming item into any config field.

- {input}, {input.a.b}, {input.tags[0]}, {input["weird.key"]}
- {{ / }} escape to literal braces
- Missing paths / null input → empty string (no 500s on absent query params)
- Unrecognised / malformed placeholders pass through so typos are visible
- Strings render raw; numbers/bools stringified; objects/arrays compact JSON

Interpolation runs executor-side (Executor.RunNodeAsync walks ConfigJson
and rewrites string leaves), so every node opts in for free. 20 new tests
(17 TemplateEngine + 2 Executor + 1 end-to-end webhook→WebResponse).
This commit is contained in:
Claude
2026-08-29 18:44:03 +00:00
parent bc682d8c9f
commit 46372e8e38
6 changed files with 465 additions and 1 deletions

View File

@@ -39,6 +39,23 @@ What shipped in addition to Plans 1 + 2 + 3:
What's next: **initial launch scope complete.** Post-launch work goes to Plan 5 (CI + observability + DataProtection key persistence) — see `docs/superpowers/followups.md`.
## Config-field templates (`{input.foo}` interpolation)
**Shipped.** Every action/terminal node's config string fields are run through `VolcanicCore.Core.TemplateEngine.Render` before the node sees them. The interpolation happens executor-side in `Executor.RunNodeAsync` (see `src/VolcanicCore.Api/Engine/Executor.cs` — `InterpolateStrings` / `WriteInterpolated`), so every node gets it for free without touching node code.
**Supported syntax:**
- `{input}` → whole input rendered as compact JSON
- `{input.foo}` / `{input.a.b}` → nested property lookup on `inputsByPort[0][0].Json`
- `{input.tags[0]}` → array index
- `{input["weird.key"]}` → bracket-form property lookup (for keys with dots/dashes)
- `{{` / `}}` → literal `{` / `}`
**Rendering rules:** strings render raw (no JSON quotes); numbers/bool render as their string form; `null` → empty; objects/arrays → compact JSON. Missing paths render as empty string (a webhook with no `?test=` should not 500). Unrecognised placeholders like `{node.foo}` and malformed/unclosed braces pass through verbatim so operators see the typo.
**Non-goals (still):** filters/pipes (`{input.foo | upper}`), cross-node references (`{nodes.http1.output.body}`), conditionals. Path lookups on the current input item only. Add to `docs/superpowers/followups.md` if you want more.
**Tests:** `TemplateEngineTests` (unit, 17 cases) + `ExecutorTests.Config_string_fields_are_templated_from_first_input_item_before_node_runs` + `WebhookEndpointTests.End_to_end_webhook_to_web_response_interpolates_template_from_query_string`.
## Invariants (do NOT relearn from scratch)
- **Git commit identity.** The repo has no configured identity. Every commit must pass `-c user.email="claude@volcanic" -c user.name="Claude"`:
@@ -62,6 +79,7 @@ What's next: **initial launch scope complete.** Post-launch work goes to Plan 5
- **`UseForwardedHeaders` runs BEFORE `UseAuthentication`.** Cookie `SecurePolicy.SameAsRequest` only flips the `Secure` flag when `Request.IsHttps == true`, which only happens once the forwarded-headers middleware has rewritten the scheme. Moving `UseAuthentication` above `UseForwardedHeaders` silently breaks HTTPS cookies in prod.
- **`SERVICE_FQDN_APP_8080` (not `SERVICE_FQDN_APP`).** Coolify v4 requires the port suffix in the env-var name AND in the `docker_compose_domains[svc].domain` string (`https://core.tes.gd:8080`). Without either, Traefik has no idea which container port to route to and returns HTTP 500.
- **`deploy.json` is gitignored** — it carries secret app env values in `.env`. Never commit; if it ever gets committed, rotate every value in it.
- **Config templating is executor-side, not per-node.** `Executor.RunNodeAsync` walks the node's `ConfigJson` and runs every string leaf through `TemplateEngine.Render` against `byPort[0][0].Json` before building `ExecutionContext`. Nodes read `ctx.Config` and get interpolated strings for free. Do NOT push the substitution back into individual nodes ("hmm, this node needs templating too") — it already has it. If a node needs to opt OUT (e.g. a raw-code field that must not touch `{...}`), that's the exception that carries the burden, not the rule.
## Working style

View File

@@ -240,8 +240,14 @@ public sealed class Executor
{
var inputsJson = SerializeInputs(byPort);
using var cfgDoc = JsonDocument.Parse(node.ConfigJson);
// Templating pass: every string value inside the node's config JSON
// gets `{input.foo}` interpolation against the first item on port 0
// BEFORE the node sees it, so every action node gets template support
// for free. See TemplateEngine + CLAUDE.md "config-field templates".
var firstInput = byPort.Count > 0 && byPort[0].Count > 0 ? byPort[0][0].Json : (JsonElement?)null;
var templatedConfig = InterpolateStrings(cfgDoc.RootElement, firstInput);
var ctx = new ExecutionContext(
executionId, workflowId, node.Id, cfgDoc.RootElement.Clone(),
executionId, workflowId, node.Id, templatedConfig,
answerChannelId, _lf.CreateLogger($"node.{node.Id}"),
node.ConnectionId, _instances);
@@ -257,6 +263,52 @@ public sealed class Executor
}
}
// Walks the config JSON, running every string leaf through TemplateEngine.
// Objects/arrays are rebuilt recursively; non-string leaves pass through.
// Fast-path: if the raw text contains no `{`, no template can fire, so we
// just clone the element.
private static JsonElement InterpolateStrings(JsonElement source, JsonElement? input)
{
if (source.ValueKind is not (JsonValueKind.Object or JsonValueKind.Array or JsonValueKind.String))
return source.Clone();
if (source.GetRawText().IndexOf('{') < 0)
return source.Clone();
var buffer = new System.IO.MemoryStream();
using (var writer = new Utf8JsonWriter(buffer))
WriteInterpolated(source, input, writer);
var doc = JsonDocument.Parse(buffer.ToArray());
return doc.RootElement.Clone();
}
private static void WriteInterpolated(JsonElement el, JsonElement? input, Utf8JsonWriter w)
{
switch (el.ValueKind)
{
case JsonValueKind.Object:
w.WriteStartObject();
foreach (var p in el.EnumerateObject())
{
w.WritePropertyName(p.Name);
WriteInterpolated(p.Value, input, w);
}
w.WriteEndObject();
break;
case JsonValueKind.Array:
w.WriteStartArray();
foreach (var item in el.EnumerateArray())
WriteInterpolated(item, input, w);
w.WriteEndArray();
break;
case JsonValueKind.String:
w.WriteStringValue(TemplateEngine.Render(el.GetString() ?? "", input));
break;
default:
el.WriteTo(w);
break;
}
}
private static string SerializeInputs(IReadOnlyList<IReadOnlyList<Item>> ports)
=> JsonSerializer.Serialize(ports.Select(port => port.Select(i => new
{

View File

@@ -0,0 +1,188 @@
using System.Text;
using System.Text.Json;
namespace VolcanicCore.Core;
// Very small `{input.foo.bar}` interpolator for node config strings.
//
// {input} → whole input rendered as compact JSON (or "" if null)
// {input.foo} → property `foo` on the input
// {input.a.b} → nested property lookup
// {input.tags[0]} → array index
// {input["weird.key"]} → property whose name has non-ident characters
// {{ / }} → literal `{` / `}`
//
// Leaves at string / number / bool render as their raw string form (no JSON
// quotes on strings, so `{input.msg}` inside `hello {input.msg}` reads as
// `hello world` rather than `hello "world"`). Objects and arrays render as
// compact JSON. Nulls render as the empty string.
//
// Missing paths (property not present, array index out of range, wrong type
// mid-path) render as the empty string — an operator wiring a webhook that
// forgets a query param should get an empty string, not a 500.
//
// Unrecognised placeholders (not `input`-rooted) and malformed placeholders
// (unclosed brace, bad syntax) are left in the output verbatim so the operator
// sees their typo rather than silently getting empty output.
public static class TemplateEngine
{
public static string Render(string template, JsonElement? input)
{
if (string.IsNullOrEmpty(template) || template.IndexOf('{') < 0)
return template;
var sb = new StringBuilder(template.Length);
var i = 0;
while (i < template.Length)
{
var c = template[i];
if (c == '{')
{
if (i + 1 < template.Length && template[i + 1] == '{')
{
sb.Append('{');
i += 2;
continue;
}
var end = template.IndexOf('}', i + 1);
if (end < 0)
{
sb.Append(template, i, template.Length - i);
break;
}
var expr = template.AsSpan(i + 1, end - i - 1);
if (TryEvaluate(expr, input, out var value))
{
sb.Append(value);
}
else
{
sb.Append('{').Append(expr).Append('}');
}
i = end + 1;
continue;
}
if (c == '}' && i + 1 < template.Length && template[i + 1] == '}')
{
sb.Append('}');
i += 2;
continue;
}
sb.Append(c);
i++;
}
return sb.ToString();
}
private static bool TryEvaluate(ReadOnlySpan<char> expr, JsonElement? input, out string value)
{
value = "";
expr = expr.Trim();
if (!expr.StartsWith("input"))
return false;
var rest = expr[5..];
if (input is not JsonElement current)
{
// input is null → any well-formed `input`-rooted expression renders empty
return rest.Length == 0 || IsWellFormedTail(rest);
}
var walker = current;
var have = true;
while (rest.Length > 0)
{
if (rest[0] == '.')
{
var j = 1;
while (j < rest.Length && (char.IsLetterOrDigit(rest[j]) || rest[j] == '_'))
j++;
if (j == 1) return false;
var name = rest[1..j].ToString();
if (have && walker.ValueKind == JsonValueKind.Object && walker.TryGetProperty(name, out var next))
walker = next;
else
have = false;
rest = rest[j..];
continue;
}
if (rest[0] == '[')
{
var close = rest.IndexOf(']');
if (close < 0) return false;
var inside = rest[1..close].Trim();
if (inside.Length >= 2 && inside[0] == '"' && inside[^1] == '"')
{
var key = inside[1..^1].ToString();
if (have && walker.ValueKind == JsonValueKind.Object && walker.TryGetProperty(key, out var next))
walker = next;
else
have = false;
}
else if (int.TryParse(inside, out var idx))
{
if (have && walker.ValueKind == JsonValueKind.Array && idx >= 0 && idx < walker.GetArrayLength())
walker = walker[idx];
else
have = false;
}
else
{
return false;
}
rest = rest[(close + 1)..];
continue;
}
// Unknown character after `input` → not our syntax; refuse.
return false;
}
value = have ? RenderValue(walker) : "";
return true;
}
// Used when input is null but we still need to know if the placeholder
// shape is one we own (so we render empty) vs foreign (so we pass through).
private static bool IsWellFormedTail(ReadOnlySpan<char> tail)
{
var i = 0;
while (i < tail.Length)
{
if (tail[i] == '.')
{
var j = i + 1;
while (j < tail.Length && (char.IsLetterOrDigit(tail[j]) || tail[j] == '_')) j++;
if (j == i + 1) return false;
i = j;
}
else if (tail[i] == '[')
{
var close = tail[i..].IndexOf(']');
if (close < 0) return false;
i += close + 1;
}
else
{
return false;
}
}
return true;
}
private static string RenderValue(JsonElement el) => el.ValueKind switch
{
JsonValueKind.String => el.GetString() ?? "",
JsonValueKind.Number => el.GetRawText(),
JsonValueKind.True => "true",
JsonValueKind.False => "false",
JsonValueKind.Null => "",
JsonValueKind.Undefined => "",
_ => el.GetRawText(),
};
}

View File

@@ -222,6 +222,58 @@ public class ExecutorTests
Assert.Single(recDownstreamOfSuccess.Calls);
}
[Fact]
public async Task Config_string_fields_are_templated_from_first_input_item_before_node_runs()
{
var rec = new Recorder { Id = "sink.template" };
var (er, nr, exec, _) = Build(new TestTrigger(), rec);
var graph = new WorkflowGraph
{
Nodes = new[] {
new WorkflowNode("t", "test.trigger", null, "{}"),
new WorkflowNode("s", "sink.template", null,
"""{"greeting":"hi {input.name}","meta":{"deep":"v={input.tags[0]}"},"list":["a","b {input.name}"]}"""),
},
Edges = new[] { new WorkflowEdge("t", 0, "s", 0) },
};
var trig = new Core.Item(JsonSerializer.SerializeToElement(new { name = "world", tags = new[] { "x", "y" } }));
var eid = await exec.StartExecutionAsync("wf", graph, "t", "manual",
new[] { trig }, null, null, CancellationToken.None);
Assert.Equal("completed", er.Get(eid)!.Status);
var cfg = rec.Calls.Single().Config;
Assert.Equal("hi world", cfg.GetProperty("greeting").GetString());
Assert.Equal("v=x", cfg.GetProperty("meta").GetProperty("deep").GetString());
var list = cfg.GetProperty("list");
Assert.Equal("a", list[0].GetString());
Assert.Equal("b world", list[1].GetString());
}
[Fact]
public async Task Missing_input_property_renders_as_empty_in_templated_config()
{
var rec = new Recorder { Id = "sink.template2" };
var (er, _, exec, _) = Build(new TestTrigger(), rec);
var graph = new WorkflowGraph
{
Nodes = new[] {
new WorkflowNode("t", "test.trigger", null, "{}"),
new WorkflowNode("s", "sink.template2", null, """{"msg":"value: {input.missing}"}"""),
},
Edges = new[] { new WorkflowEdge("t", 0, "s", 0) },
};
var trig = new Core.Item(JsonSerializer.SerializeToElement(new { other = 1 }));
var eid = await exec.StartExecutionAsync("wf", graph, "t", "manual",
new[] { trig }, null, null, CancellationToken.None);
Assert.Equal("completed", er.Get(eid)!.Status);
Assert.Equal("value: ", rec.Calls.Single().Config.GetProperty("msg").GetString());
}
[Fact]
public async Task All_downstream_failed_rolls_up_to_failed()
{

View File

@@ -0,0 +1,118 @@
using System.Text.Json;
using VolcanicCore.Core;
namespace VolcanicCore.Tests;
public class TemplateEngineTests
{
private static JsonElement J(string json) => JsonDocument.Parse(json).RootElement;
[Fact]
public void No_placeholders_returns_the_template_unchanged()
{
Assert.Equal("hello world", TemplateEngine.Render("hello world", J("""{"x":1}""")));
}
[Fact]
public void Whole_input_renders_as_compact_json()
{
var input = J("""{"a":1,"b":"x"}""");
Assert.Equal("""got: {"a":1,"b":"x"}""", TemplateEngine.Render("got: {input}", input));
}
[Fact]
public void Top_level_string_field_renders_without_json_quotes()
{
Assert.Equal("value: hello", TemplateEngine.Render("value: {input.test}", J("""{"test":"hello"}""")));
}
[Fact]
public void Nested_path_lookup()
{
Assert.Equal("42", TemplateEngine.Render("{input.a.b.c}", J("""{"a":{"b":{"c":42}}}""")));
}
[Fact]
public void Array_index_lookup()
{
Assert.Equal("y", TemplateEngine.Render("{input.tags[1]}", J("""{"tags":["x","y","z"]}""")));
}
[Fact]
public void Missing_property_renders_as_empty_string()
{
Assert.Equal("value: ", TemplateEngine.Render("value: {input.missing}", J("""{"other":1}""")));
}
[Fact]
public void Missing_nested_path_renders_as_empty_string()
{
Assert.Equal("[]", TemplateEngine.Render("[{input.a.b.c}]", J("""{"a":{"b":{}}}""")));
}
[Fact]
public void Array_index_out_of_range_renders_as_empty_string()
{
Assert.Equal("<>", TemplateEngine.Render("<{input.tags[9]}>", J("""{"tags":["only"]}""")));
}
[Fact]
public void Numeric_and_boolean_and_null_leaves_render_as_their_string_form()
{
var input = J("""{"n":42,"f":1.5,"b":true,"z":null}""");
Assert.Equal("42|1.5|true|", TemplateEngine.Render("{input.n}|{input.f}|{input.b}|{input.z}", input));
}
[Fact]
public void Object_and_array_leaves_render_as_compact_json()
{
var input = J("""{"o":{"k":1},"a":[1,2]}""");
Assert.Equal("""{"k":1}/[1,2]""", TemplateEngine.Render("{input.o}/{input.a}", input));
}
[Fact]
public void Doubled_braces_render_as_literal_single_braces()
{
Assert.Equal("{not a template}", TemplateEngine.Render("{{not a template}}", J("{}")));
}
[Fact]
public void Doubled_braces_around_a_real_placeholder()
{
Assert.Equal("{hello}", TemplateEngine.Render("{{{input.x}}}", J("""{"x":"hello"}""")));
}
[Fact]
public void Null_input_renders_placeholders_as_empty_string()
{
Assert.Equal("hi ", TemplateEngine.Render("hi {input.name}", input: null));
}
[Fact]
public void Whole_input_placeholder_with_null_input_renders_empty()
{
Assert.Equal("", TemplateEngine.Render("{input}", input: null));
}
[Fact]
public void Unrecognised_placeholder_is_left_as_literal_text()
{
// Only `input`-rooted expressions are supported; anything else is passed
// through unchanged so operators notice their typo instead of silent-emptying.
Assert.Equal("{node.foo}", TemplateEngine.Render("{node.foo}", J("""{"foo":1}""")));
}
[Fact]
public void Unclosed_brace_is_left_as_literal_text()
{
Assert.Equal("start {input.x and more", TemplateEngine.Render("start {input.x and more", J("""{"x":1}""")));
}
[Fact]
public void Bracketed_string_key_lookup()
{
// Bracket form also works for property names, useful for keys with dots
// or dashes (`weird.key` can't be reached via dot syntax).
Assert.Equal("v", TemplateEngine.Render("""{input["weird.key"]}""", J("""{"weird.key":"v"}""")));
}
}

View File

@@ -130,6 +130,42 @@ public class WebhookEndpointTests : IClassFixture<WebFactory>
Assert.NotEqual(Guid.Empty, ch);
}
[Fact]
public async Task End_to_end_webhook_to_web_response_interpolates_template_from_query_string()
{
// Real workflow: GET /wh/{wf}/hook?test=hello → executor runs
// WebResponseNode with body "value: {input.test}" → the templating
// pass in Executor.RunNodeAsync should render "value: hello" before
// the node sees it.
var c = _factory.CreateClient(new() { HandleCookies = true, AllowAutoRedirect = false });
var login = await c.PostAsJsonAsync("/api/session", new { password = _factory.AdminPassword });
Assert.Equal(HttpStatusCode.NoContent, login.StatusCode);
var graph = new
{
nodes = new object[]
{
new { id = "t", type_id = "http.webhook", config = new { path = "tpl" } },
new { id = "r", type_id = "http.response",
config = new { status = 200, body = "value: {input.test}", contentType = "text/plain" } },
},
edges = new object[]
{
new { from_node = "t", from_port = 0, to_node = "r", to_port = 0 },
},
};
var create = await c.PostAsJsonAsync("/api/workflows", new { name = "tpl-e2e", graph });
Assert.Equal(HttpStatusCode.Created, create.StatusCode);
var wfId = (await create.Content.ReadFromJsonAsync<JsonElement>()).GetProperty("id").GetString()!;
var activate = await c.PostAsync($"/api/workflows/{wfId}/activate", null);
Assert.Equal(HttpStatusCode.NoContent, activate.StatusCode);
var res = await c.GetAsync($"/wh/{wfId}/tpl?test=hello");
Assert.Equal(HttpStatusCode.OK, res.StatusCode);
Assert.Equal("value: hello", await res.Content.ReadAsStringAsync());
}
[Fact]
public async Task Registered_callback_that_resolves_yields_the_supplied_status_body()
{