admin/VolcanicCore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: close out Plan 2 — workflow engine live

Browse Source
This commit is contained in:
Claude
2026-08-28 15:39:36 +00:00
parent 6065f0c91e
commit bc56e9edf1
2 changed files with 25 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
CLAUDE.md
View File

@@ -6,32 +6,37 @@ VolcanicCore is a self-hosted N8N-style workflow engine (C# backend + React fron
- **Design spec** — `docs/superpowers/specs/2026-08-28-volcaniccore-workflow-engine-design.md`. The whole architecture (execution model, extension contracts, webhook mechanics, persistence, deploy shape). Read this before writing plans or code.
- **Plan 1** (backend foundation, executed) — `docs/superpowers/plans/2026-08-28-plan-1-backend-foundation.md`. Shows the working style and the file structure conventions.
- **Plan 2** (workflow engine, executed) — `docs/superpowers/plans/2026-08-29-plan-2-workflow-engine.md`. Executor, WebhookRouter, TriggerHost, five nodes, NATS provider, retention.
- **Follow-ups** — `docs/superpowers/followups.md`. Cross-plan items surfaced during review that later plans must handle. Read before starting a plan; extend as new items surface.
## Current status
**Plan 1 (backend foundation) shipped.** Head commit at plan close: `86bf173`. `dotnet build` clean, `dotnet test` 20/20.
**Plan 2 (workflow engine) shipped.** Head commit at plan close: `6065f0c`. `dotnet build` clean, `dotnet test` 70/70.
What's done:
- ASP.NET Core 9 monolith. Four projects: `VolcanicCore.Core` (contracts), `.Persistence` (SQLite, migrator, encryption, repos), `.Api` (endpoints, auth, registries), `VolcanicCore.Tests` (xUnit + WebApplicationFactory).
- ASP.NET Core 9 monolith. Four projects: `VolcanicCore.Core` (contracts + `WebhookRegistry`/`WebhookRouter`), `.Persistence` (SQLite, migrator, encryption, repos), `.Providers` (`NatsConnectionProvider`), `.Nodes` (five node implementations), `.Api` (endpoints, auth, registries, engine host), `VolcanicCore.Tests` (xUnit + WebApplicationFactory).
- Admin cookie auth (BCrypt, `ADMIN_PASSWORD` env), sessions endpoints.
- SQLite with hand-rolled versioned migrator (3 migrations: `admin_config`, `connections`, `workflows`).
- AES-GCM encryption for connection secrets (key from `ENCRYPTION_KEY` env or `/data/.enc-key`).
- SQLite with hand-rolled versioned migrator (5 migrations: `admin_config`, `connections`, `workflows`, `executions`, `node_executions`).
- AES-GCM encryption for connection secrets and for the `admin_config` password-source row.
- Extension contracts: `INodeType` / `IActionNode` / `ITriggerNode` / `IConnectionProvider` in `.Core`.
- DI-collected `NodeRegistry` and `ConnectionProviderRegistry`.
- REST endpoints: session, discovery (empty in Plan 1), connections CRUD, workflows CRUD + activate/deactivate.
- DI-collected `NodeRegistry` and `ConnectionProviderRegistry`; 5 nodes + NATS provider registered.
- `Executor` — in-degree scheduler, fan-out, failure propagation, per-node persistence per §4 of the design spec.
- `WebhookRouter` + `WebhookRegistry` in `VolcanicCore.Core`; `POST /wh/{workflow-id}/{*path}` endpoint bridging HTTP to the executor via `answerChannelId` TCS.
- `TriggerHost` `IHostedService` — activates triggers on workflow.active flips; re-hydrates from DB on boot; disposes on deactivate.
- `TriggerActivationSink.EmitAsync` — fire-and-forget `Task.Run` per event; does not block the trigger pump.
- Five nodes: NATS Hook (trigger), NATS Publish (terminal), Webhook (trigger), WebResponse (terminal), PythonTransform (IronPython 3).
- NATS `IConnectionProvider` (userpass + TLS URL).
- `SchemaSecretPolicy` — reads `secretFields[]` off each provider's `ConfigSchema`.
- REST endpoints: session, discovery (returns 5 nodes + `nats` provider), connections CRUD, workflows CRUD + activate/deactivate, executions list + detail, manual test-run.
- `RetentionService` `BackgroundService` — daily prune of executions older than `EXECUTION_RETENTION_DAYS` (default 30).
What's next: **Plan 2** — the workflow engine itself. Not yet written. When authored, it will cover:
- Executions + node_executions tables (Migration 004 + 005).
- `Executor` — in-degree scheduler, fan-out, failure propagation per §4 of the design spec.
- `WebhookRouter` + `POST /wh/{workflow-id}/{*path}` — see §5 of the spec.
- `TriggerHost` lifecycle — start triggers on activation, dispose on deactivation.
- NATS `IConnectionProvider` + all five nodes: NATS Hook, Webhook, PythonTransform (IronPython 3), WebResponse, NATS Publish.
- Manual test-run endpoint.
- Retention `BackgroundService`.
- Discovery endpoints will start returning real node types + providers once these are DI-registered.
What's next: **Plan 3** — React SPA. Not yet written. When authored, it will cover:
- React 18 + Vite + TypeScript + Tailwind + React Flow + TanStack Query SPA under `frontend/`.
- Login screen; workflows list + editor; connections list + form; executions viewer.
- SPA build lands in `wwwroot/` and is served by the `.Api` project via `UseStaticFiles` + fallback-to-index.
- No auth-token work — cookie session already in place; the SPA runs same-origin.
Plans 3 (React SPA) and 4 (Coolify Dockerfile + deploy) follow.
Plan 4 (Coolify Dockerfile + deploy) follows.
## Invariants (do NOT relearn from scratch)
@@ -43,9 +48,10 @@ Plans 3 (React SPA) and 4 (Coolify Dockerfile + deploy) follow.
- **Test parallelism is disabled at the assembly level.** See the `[assembly: CollectionBehavior(DisableTestParallelization = true)]` attribute at the top of `tests/VolcanicCore.Tests/WebFactory.cs`. Multiple `WebFactory` instances share the process; pre-`Configuration.Bind` env-var reads made parallel classes flaky. Do not re-enable parallelism without a redesign.
- **Cookie authentication is in the shared framework.** Do NOT add `Microsoft.AspNetCore.Authentication.Cookies` as a `PackageReference`. The last standalone version on NuGet is 2.3.0 (ASP.NET Core 2.x, 2019). .NET 9's `Microsoft.AspNetCore.App` shared framework provides the cookie-auth types with no PackageReference.
- **`Program.cs` env-var loop honors pre-set config.** In tests, `WebFactory.ConfigureWebHost` uses `builder.UseSetting(...)` to inject `DataDirectory` and `AdminPassword`. The env-var loop in `Program.cs` skips keys already set by upstream sources (`if (!string.IsNullOrEmpty(builder.Configuration[prop])) continue;`). Do not remove this check.
- **`Migration001Initial`, `002Connections`, `003Workflows`** are registered in the `Migrator` factory in `Program.cs`. Add new migrations to the same array in version order.
- **`AdminBootstrap`** stores `admin.password_hash_source` in plaintext by design (see the comment in `AdminBootstrap.cs`) — it is the change-detector for the env-provided `ADMIN_PASSWORD`. Do not "fix" this by hashing the source. Encryption at rest for `admin_config` is a Plan 2 follow-up.
- **`IConnectionSecretPolicy`** in Plan 1 is `HardcodedSecretPolicy` (fixed field-name set). Plan 2 replaces it with a schema-driven implementation that reads `secretFields` off each registered provider's `ConfigSchema`. Do not extend the hardcoded set further — replace instead.
- **`Migration001Initial`, `002Connections`, `003Workflows`, `004Executions`, `005NodeExecutions`** are registered in the `Migrator` factory in `Program.cs`. Add new migrations to the same array in version order.
- **`AdminBootstrap`** stores `admin.password_hash_source` in the `admin_config` table as the change-detector for the env-provided `ADMIN_PASSWORD` (see the comment in `AdminBootstrap.cs`). Do not "fix" this by hashing the source — it must be reversible so we can compare against the current env var on boot. The row is AES-GCM encrypted at rest as of Plan 2.
- **Fire-and-forget trigger sinks.** `TriggerActivationSink.EmitAsync` returns `Task.CompletedTask` after spinning the execution into a background `Task.Run`. This prevents NATS pumps from blocking per-event on execution completion. `WebhookRouter.Register` (not the sink) is what the `/wh` endpoint awaits for the reply.
- **Cross-project cycle-avoidance for `WebhookRegistry` and `WebhookRouter`.** Both types live in `VolcanicCore.Core`, NOT `VolcanicCore.Api.Engine`, because `WebhookNode` and `WebResponseNode` in `.Nodes` need them and `.Api` already references `.Nodes` (adding a back-reference would cycle). Do not move them back.
## Working style