admin/Experiment_ReinforcementLearning
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
dff43288e215ba3554ecde1ecf751ffdebfd6c30
Experiment_ReinforcementLea…/docs/superpowers/plans/2026-07-17-03-backend.md
meelstorm caf0376323 Add Plan 3: ASP.NET backend plan
2026-07-17 18:23:16 +00:00

1096 lines
35 KiB
Markdown
Raw Blame History

# ASP.NET Backend Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the ASP.NET Core Web API that sits between the React frontend and the Game CLI. Per connected browser: accept a WebSocket at `/ws/game`, spawn one `GameCli` subprocess, run a 50 Hz tick loop that asks the trained PPO policy (loaded from ONNX) for an action and steps the CLI, and forward each new state to the browser. Reference spec at `docs/superpowers/specs/2026-07-17-cursor-following-lander-design.md` §3.
**Architecture:** One `Backend/` .NET 8 web project. Each WebSocket connection owns a `GameSession` that owns a `GameProcess` (CLI subprocess). A single `PolicyRunner` (registered as a singleton) holds the loaded ONNX inference session and is called by all sessions. No SignalR — raw WebSocket at `/ws/game`. The frontend build will eventually be served as static files from the same host; that wiring is out of scope for this plan.
**Tech Stack:** .NET 8 SDK, ASP.NET Core, `Microsoft.ML.OnnxRuntime`, `System.Threading.Channels`, xUnit for tests.
---
## Prerequisites
- **Plan 1 complete.** The `GameCli` binary must be publishable at `publish/GameCli/GameCli`.
- **Plan 2 complete.** A trained model must exist at `models/ppo_lander.onnx` (even a smoke-quality one is fine — the backend just needs a valid ONNX file to load).
---
## File structure
```
Experiment_ReinforcementLearning/
├── Backend/
│ ├── Backend.csproj
│ ├── Program.cs # startup + WebSocket routing
│ ├── LanderConfig.cs # strongly-typed config (paths)
│ ├── GameProcess.cs # CLI subprocess wrapper
│ ├── PolicyRunner.cs # ONNX inference wrapper (singleton)
│ ├── GameSession.cs # per-connection tick loop
│ ├── WireMessages.cs # WebSocket JSON DTOs (client↔server)
│ └── appsettings.json # default Lander config
└── Backend.Tests/
├── Backend.Tests.csproj
├── GameProcessTests.cs
├── PolicyRunnerTests.cs
└── EndToEndSmokeTests.cs
```
`Program.cs` is the only file that touches ASP.NET plumbing. `GameProcess`/`PolicyRunner`/`GameSession` are testable in isolation.
---
## Configuration convention
Both the CLI and ONNX paths are resolved at startup relative to a "repo root" anchor computed by walking up from `AppContext.BaseDirectory` until we find `GameCli.sln`. This keeps `dotnet run --project Backend` working from any CWD.
- `Lander:CliPath` (default: `publish/GameCli/GameCli` — relative to repo root)
- `Lander:ModelPath` (default: `models/ppo_lander.onnx` — relative to repo root)
- Env vars `LANDER__CliPath` and `LANDER__ModelPath` override (ASP.NET double-underscore convention; case-insensitive so `LANDER__CLIPATH` also works).
---
## Task 1: Scaffold Backend project
**Files:**
- Create: `Backend/Backend.csproj`
- Create: `Backend/Program.cs` (stub — replaced in Task 6)
- Create: `Backend/appsettings.json`
- Create: `Backend.Tests/Backend.Tests.csproj`
- Modify: `GameCli.sln` (add both projects)
- [ ] **Step 1: Create the projects and add to solution**
Run from the repo root:
```bash
dotnet new webapi -n Backend -o Backend -f net8.0 --no-https --use-controllers false
dotnet new xunit -n Backend.Tests -o Backend.Tests -f net8.0
dotnet sln add Backend/Backend.csproj Backend.Tests/Backend.Tests.csproj
dotnet add Backend.Tests/Backend.Tests.csproj reference Backend/Backend.csproj
dotnet add Backend/Backend.csproj package Microsoft.ML.OnnxRuntime --version 1.19.*
dotnet add Backend.Tests/Backend.Tests.csproj package Microsoft.AspNetCore.Mvc.Testing --version 8.0.*
```
- [ ] **Step 2: Delete boilerplate**
The webapi template creates `Backend/WeatherForecast.cs` (or similar sample) and puts a full sample in `Program.cs`. Delete or replace:
```bash
rm -f Backend/WeatherForecast.cs
rm -f Backend.Tests/UnitTest1.cs
```
Overwrite `Backend/Program.cs` with a stub (Task 6 rewrites it):
```csharp
// Placeholder. Rewritten in Task 6.
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/", () => "GameCli Backend placeholder");
app.Run();
```
- [ ] **Step 3: Replace `Backend/appsettings.json`**
Overwrite with:
```json
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"AllowedHosts": "*",
"Lander": {
"CliPath": "publish/GameCli/GameCli",
"ModelPath": "models/ppo_lander.onnx"
}
}
```
Delete `Backend/appsettings.Development.json` if it contains sample logging that duplicates the above.
- [ ] **Step 4: Verify it builds and starts**
```bash
dotnet build
```
Expected: 0 errors.
Quick manual check (optional but recommended):
```bash
dotnet run --project Backend --urls http://localhost:5100 &
BACKEND_PID=$!
sleep 3
curl -sf http://localhost:5100/ && echo
kill $BACKEND_PID
wait $BACKEND_PID 2>/dev/null
```
Expected: `GameCli Backend placeholder`.
- [ ] **Step 5: Verify empty test suite runs**
```bash
dotnet test --filter FullyQualifiedName~Backend.Tests
```
Expected: exit code 0 (either "Passed: 0" or "No test is available" — both fine).
- [ ] **Step 6: Commit**
```bash
git add Backend/ Backend.Tests/ GameCli.sln
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(backend): scaffold ASP.NET Core Web API + xunit test project"
```
---
## Task 2: `WireMessages.cs` — WebSocket JSON DTOs
**Files:**
- Create: `Backend/WireMessages.cs`
The backend talks JSON with the browser. Same source-gen pattern as GameCli's Protocol.cs.
- [ ] **Step 1: Write `Backend/WireMessages.cs`**
```csharp
using System.Text.Json.Serialization;
namespace Backend;
// -------- Client → Server --------
/// <summary>Cursor position update from the browser.</summary>
public sealed class CursorMessage
{
[JsonPropertyName("type")] public string Type { get; set; } = "cursor";
[JsonPropertyName("x")] public float X { get; set; }
[JsonPropertyName("y")] public float Y { get; set; }
}
// -------- Server → Client --------
/// <summary>Sent once on WebSocket open. Tells the client the world dimensions.</summary>
public sealed class InitFrame
{
[JsonPropertyName("type")] public string Type { get; set; } = "init";
[JsonPropertyName("world")] public float[] World { get; set; } = new[] { 1f, 1f };
}
/// <summary>Sent every physics tick with the current ship pose and target.</summary>
public sealed class StateFrame
{
[JsonPropertyName("type")] public string Type { get; set; } = "state";
[JsonPropertyName("x")] public float X { get; set; }
[JsonPropertyName("y")] public float Y { get; set; }
[JsonPropertyName("angle")] public float Angle { get; set; }
[JsonPropertyName("engine")] public int Engine { get; set; }
[JsonPropertyName("target")] public float[] Target { get; set; } = new[] { 0.5f, 0.5f };
[JsonPropertyName("step")] public int Step { get; set; }
}
/// <summary>Source-gen JSON contracts.</summary>
[JsonSourceGenerationOptions(WriteIndented = false)]
[JsonSerializable(typeof(CursorMessage))]
[JsonSerializable(typeof(InitFrame))]
[JsonSerializable(typeof(StateFrame))]
internal partial class WireJsonContext : System.Text.Json.Serialization.JsonSerializerContext
{
}
```
- [ ] **Step 2: Add `InternalsVisibleTo` for tests**
Edit `Backend/Backend.csproj`, add before `</Project>`:
```xml
<ItemGroup>
<InternalsVisibleTo Include="Backend.Tests" />
</ItemGroup>
```
- [ ] **Step 3: Verify build**
```bash
dotnet build
```
Expected: 0 errors.
- [ ] **Step 4: Commit**
```bash
git add Backend/WireMessages.cs Backend/Backend.csproj
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(backend): add WebSocket wire message DTOs"
```
---
## Task 3: `GameProcess` — CLI subprocess wrapper
**Files:**
- Create: `Backend/GameProcess.cs`
- Create: `Backend.Tests/GameProcessTests.cs`
`GameProcess` wraps `System.Diagnostics.Process` running the `GameCli` binary. It reads the init handshake once on start, exposes async `StepAsync` and `ResetAsync`, and disposes cleanly. It reuses the DTOs from the CLI's `Protocol.cs` — but those are `internal` to the `GameCli` assembly. Rather than take a project reference to the whole CLI (its `Program.cs` runs on load-only under top-level statements), we declare local DTOs in `GameProcess.cs` that mirror the wire format.
- [ ] **Step 1: Write the failing tests**
Create `Backend.Tests/GameProcessTests.cs`:
```csharp
using System.IO;
using Backend;
using Xunit;
namespace Backend.Tests;
public class GameProcessTests
{
private static string LocateCliBinary()
{
// Walk up from the test assembly's location to find publish/GameCli/GameCli.
var dir = AppContext.BaseDirectory;
while (dir is not null)
{
var candidate = Path.Combine(dir, "publish", "GameCli", "GameCli");
if (File.Exists(candidate)) return candidate;
var parent = Directory.GetParent(dir);
if (parent is null) break;
dir = parent.FullName;
}
throw new FileNotFoundException(
"publish/GameCli/GameCli not found — run `dotnet publish GameCli -c Release -o publish/GameCli`");
}
[Fact]
public async Task StartsAndReadsInitHandshake()
{
await using var proc = new GameProcess(LocateCliBinary());
await proc.StartAsync();
Assert.Equal(0.02f, proc.Dt);
Assert.Equal(7, proc.ObsDim);
Assert.Equal(4, proc.NActions);
}
[Fact]
public async Task StepReturnsSevenDimObservation()
{
await using var proc = new GameProcess(LocateCliBinary());
await proc.StartAsync();
var obs = await proc.StepAsync(action: 0, target: (0.5f, 0.5f));
Assert.Equal(7, obs.Observation.Length);
Assert.Equal(1, obs.Step);
}
[Fact]
public async Task ResetClearsStepCounter()
{
await using var proc = new GameProcess(LocateCliBinary());
await proc.StartAsync();
for (int i = 0; i < 3; i++)
await proc.StepAsync(action: 0, target: (0.5f, 0.5f));
var afterReset = await proc.ResetAsync(seed: 42);
Assert.Equal(0, afterReset.Step);
}
}
```
- [ ] **Step 2: Run tests to verify they fail**
```bash
dotnet publish GameCli/GameCli.csproj -c Release -o publish/GameCli
dotnet test --filter FullyQualifiedName~GameProcessTests
```
Expected: build error — `GameProcess` does not exist.
- [ ] **Step 3: Write `Backend/GameProcess.cs`**
```csharp
using System.Diagnostics;
using System.Text.Json;
using System.Text.Json.Serialization;
namespace Backend;
/// <summary>
/// One instance == one <c>GameCli</c> subprocess. Not thread-safe.
/// Callers must sequence <see cref="StepAsync"/> and <see cref="ResetAsync"/>.
/// </summary>
public sealed class GameProcess : IAsyncDisposable
{
private readonly string _binaryPath;
private Process? _proc;
private StreamReader? _stdout;
private StreamWriter? _stdin;
public float Dt { get; private set; }
public int ObsDim { get; private set; }
public int NActions { get; private set; }
public GameProcess(string binaryPath)
{
_binaryPath = binaryPath;
}
public async Task StartAsync()
{
if (_proc is not null) throw new InvalidOperationException("already started");
if (!File.Exists(_binaryPath))
throw new FileNotFoundException($"GameCli binary not found at {_binaryPath}");
var psi = new ProcessStartInfo
{
FileName = _binaryPath,
RedirectStandardInput = true,
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false,
};
_proc = Process.Start(psi)
?? throw new InvalidOperationException("failed to start GameCli");
_stdout = _proc.StandardOutput;
_stdin = _proc.StandardInput;
var line = await _stdout.ReadLineAsync()
?? throw new IOException("GameCli died before init handshake");
var init = JsonSerializer.Deserialize(line, ProcessJsonContext.Default.InitEnvelope)
?? throw new IOException($"malformed init handshake: {line}");
Dt = init.Init.Dt;
ObsDim = init.Init.ObsDim;
NActions = init.Init.NActions;
}
public async Task<StepResult> StepAsync(int action, (float X, float Y) target)
{
var input = new StepIn
{
Action = action,
Target = new[] { target.X, target.Y },
};
await WriteAsync(input, ProcessJsonContext.Default.StepIn);
return await ReadStepResultAsync();
}
public async Task<StepResult> ResetAsync(int? seed = null)
{
var input = new StepIn { Cmd = "reset", Seed = seed };
await WriteAsync(input, ProcessJsonContext.Default.StepIn);
return await ReadStepResultAsync();
}
private async Task WriteAsync<T>(T value, System.Text.Json.Serialization.Metadata.JsonTypeInfo<T> ctx)
{
if (_stdin is null) throw new InvalidOperationException("not started");
var json = JsonSerializer.Serialize(value, ctx);
await _stdin.WriteLineAsync(json);
await _stdin.FlushAsync();
}
private async Task<StepResult> ReadStepResultAsync()
{
if (_stdout is null) throw new InvalidOperationException("not started");
var line = await _stdout.ReadLineAsync()
?? throw new IOException("GameCli closed stdout unexpectedly");
var msg = JsonSerializer.Deserialize(line, ProcessJsonContext.Default.StepOut)
?? throw new IOException($"malformed step output: {line}");
return new StepResult(msg.Obs, msg.State, msg.Reward, msg.Done, msg.Step);
}
public async ValueTask DisposeAsync()
{
try { _stdin?.Close(); } catch { }
if (_proc is not null)
{
try
{
var exited = _proc.WaitForExit(2000);
if (!exited) _proc.Kill(entireProcessTree: true);
}
catch { }
_proc.Dispose();
_proc = null;
}
await ValueTask.CompletedTask;
}
}
// -------- Result --------
public readonly record struct StepResult(
float[] Observation,
ShipStatePayload State,
float Reward,
bool Done,
int Step);
// -------- Wire DTOs (local mirror of GameCli's Protocol.cs) --------
public sealed class StepIn
{
[JsonPropertyName("action")] public int? Action { get; set; }
[JsonPropertyName("target")] public float[]? Target { get; set; }
[JsonPropertyName("cmd")] public string? Cmd { get; set; }
[JsonPropertyName("seed")] public int? Seed { get; set; }
}
public sealed class StepOut
{
[JsonPropertyName("obs")] public float[] Obs { get; set; } = Array.Empty<float>();
[JsonPropertyName("state")] public ShipStatePayload State { get; set; } = new();
[JsonPropertyName("reward")] public float Reward { get; set; }
[JsonPropertyName("done")] public bool Done { get; set; }
[JsonPropertyName("step")] public int Step { get; set; }
}
public sealed class ShipStatePayload
{
[JsonPropertyName("x")] public float X { get; set; }
[JsonPropertyName("y")] public float Y { get; set; }
[JsonPropertyName("angle")] public float Angle { get; set; }
[JsonPropertyName("engine")] public int Engine { get; set; }
}
public sealed class InitEnvelope
{
[JsonPropertyName("init")] public InitPayload Init { get; set; } = new();
}
public sealed class InitPayload
{
[JsonPropertyName("world")] public float[] World { get; set; } = new[] { 1f, 1f };
[JsonPropertyName("dt")] public float Dt { get; set; }
[JsonPropertyName("obs_dim")] public int ObsDim { get; set; }
[JsonPropertyName("n_actions")] public int NActions { get; set; }
}
[JsonSourceGenerationOptions(WriteIndented = false)]
[JsonSerializable(typeof(StepIn))]
[JsonSerializable(typeof(StepOut))]
[JsonSerializable(typeof(ShipStatePayload))]
[JsonSerializable(typeof(InitEnvelope))]
[JsonSerializable(typeof(InitPayload))]
internal partial class ProcessJsonContext : JsonSerializerContext
{
}
```
- [ ] **Step 4: Run tests to verify they pass**
```bash
dotnet test --filter FullyQualifiedName~GameProcessTests
```
Expected: 3 passed, 0 failed.
- [ ] **Step 5: Commit**
```bash
git add Backend/GameProcess.cs Backend.Tests/GameProcessTests.cs
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(backend): GameProcess subprocess wrapper with async step/reset"
```
---
## Task 4: `PolicyRunner` — ONNX inference wrapper
**Files:**
- Create: `Backend/PolicyRunner.cs`
- Create: `Backend.Tests/PolicyRunnerTests.cs`
- [ ] **Step 1: Write the failing tests**
Create `Backend.Tests/PolicyRunnerTests.cs`:
```csharp
using System.IO;
using Backend;
using Xunit;
namespace Backend.Tests;
public class PolicyRunnerTests
{
private static string LocateOnnxModel()
{
var dir = AppContext.BaseDirectory;
while (dir is not null)
{
var candidate = Path.Combine(dir, "models", "ppo_lander.onnx");
if (File.Exists(candidate)) return candidate;
var parent = Directory.GetParent(dir);
if (parent is null) break;
dir = parent.FullName;
}
throw new FileNotFoundException(
"models/ppo_lander.onnx not found — run `python Training/export_onnx.py ...`");
}
[Fact]
public void LoadsWithoutError()
{
using var runner = new PolicyRunner(LocateOnnxModel());
}
[Fact]
public void SelectAction_ReturnsValidActionForZeroObs()
{
using var runner = new PolicyRunner(LocateOnnxModel());
var obs = new float[7]; // all zeros
int action = runner.SelectAction(obs);
Assert.InRange(action, 0, 3);
}
[Fact]
public void SelectAction_IsDeterministicForSameInput()
{
using var runner = new PolicyRunner(LocateOnnxModel());
var obs = new float[] { 0.1f, -0.2f, 0.05f, 0.0f, 0.0f, 1.0f, 0.0f };
int a1 = runner.SelectAction(obs);
int a2 = runner.SelectAction(obs);
Assert.Equal(a1, a2);
}
[Fact]
public void SelectAction_ThrowsIfObsLengthWrong()
{
using var runner = new PolicyRunner(LocateOnnxModel());
Assert.Throws<ArgumentException>(() => runner.SelectAction(new float[3]));
}
}
```
- [ ] **Step 2: Run tests to verify they fail**
```bash
dotnet test --filter FullyQualifiedName~PolicyRunnerTests
```
Expected: build error — `PolicyRunner` does not exist.
- [ ] **Step 3: Write `Backend/PolicyRunner.cs`**
```csharp
using Microsoft.ML.OnnxRuntime;
using Microsoft.ML.OnnxRuntime.Tensors;
namespace Backend;
/// <summary>
/// Loads a PPO policy ONNX model once and provides deterministic action selection.
/// Thread-safe: <see cref="InferenceSession"/> is safe for concurrent Run() calls.
/// </summary>
public sealed class PolicyRunner : IDisposable
{
public const int ObsDim = 7;
public const int NActions = 4;
private readonly InferenceSession _session;
private readonly string _inputName;
public PolicyRunner(string onnxPath)
{
if (!File.Exists(onnxPath))
throw new FileNotFoundException($"ONNX model not found at {onnxPath}");
_session = new InferenceSession(onnxPath);
_inputName = _session.InputMetadata.Keys.First();
}
/// <summary>Run one inference and return the argmax action index.</summary>
public int SelectAction(ReadOnlySpan<float> obs)
{
if (obs.Length != ObsDim)
throw new ArgumentException($"expected obs of length {ObsDim}, got {obs.Length}", nameof(obs));
var tensor = new DenseTensor<float>(new[] { 1, ObsDim });
for (int i = 0; i < ObsDim; i++) tensor[0, i] = obs[i];
using var results = _session.Run(new[]
{
NamedOnnxValue.CreateFromTensor(_inputName, tensor)
});
var logits = results.First().AsEnumerable<float>().ToArray();
// argmax
int best = 0;
float bestVal = logits[0];
for (int i = 1; i < logits.Length; i++)
{
if (logits[i] > bestVal) { best = i; bestVal = logits[i]; }
}
return best;
}
public void Dispose() => _session.Dispose();
}
```
- [ ] **Step 4: Run tests to verify they pass**
```bash
dotnet test --filter FullyQualifiedName~PolicyRunnerTests
```
Expected: 4 passed, 0 failed.
- [ ] **Step 5: Commit**
```bash
git add Backend/PolicyRunner.cs Backend.Tests/PolicyRunnerTests.cs
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(backend): PolicyRunner ONNX inference wrapper"
```
---
## Task 5: `GameSession` — per-connection tick loop
**Files:**
- Create: `Backend/GameSession.cs`
`GameSession` owns one WebSocket + one `GameProcess`. It runs two concurrent loops (reader + ticker) both driven by a single `CancellationToken`. The ticker fires at 50 Hz with a `PeriodicTimer`.
- [ ] **Step 1: Write `Backend/GameSession.cs`**
```csharp
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Threading.Channels;
namespace Backend;
/// <summary>
/// One <see cref="GameSession"/> per connected browser. Runs the physics tick loop
/// and the WebSocket reader loop concurrently until either side disconnects.
/// </summary>
public sealed class GameSession : IAsyncDisposable
{
private readonly WebSocket _socket;
private readonly GameProcess _game;
private readonly PolicyRunner _policy;
private readonly ILogger<GameSession> _logger;
// Mailbox: cursor updates from the browser. Bounded to 1 with drop-newest
// isn't quite right — we want drop-oldest so the ticker always reads the
// latest cursor. Channels' DropWrite behavior keeps the first, so we do
// it manually by draining the reader on each tick.
private readonly Channel<(float X, float Y)> _cursorInbox =
Channel.CreateUnbounded<(float, float)>();
private (float X, float Y) _currentCursor = (0.5f, 0.5f);
private float[] _lastObs = new float[PolicyRunner.ObsDim];
public GameSession(WebSocket socket, GameProcess game, PolicyRunner policy,
ILogger<GameSession> logger)
{
_socket = socket;
_game = game;
_policy = policy;
_logger = logger;
}
public async Task RunAsync(CancellationToken cancellation)
{
await _game.StartAsync();
// First step (noop) to get an initial observation for the policy.
var initial = await _game.ResetAsync();
_lastObs = initial.Observation;
// Send init frame to the browser.
await SendJsonAsync(new InitFrame { World = new[] { 1f, 1f } },
WireJsonContext.Default.InitFrame, cancellation);
var readerTask = ReaderLoopAsync(cancellation);
var tickerTask = TickerLoopAsync(cancellation);
// First loop to complete cancels the other.
var done = await Task.WhenAny(readerTask, tickerTask);
try { await done; }
catch (OperationCanceledException) { /* expected */ }
catch (Exception ex) { _logger.LogWarning(ex, "session loop ended"); }
}
private async Task ReaderLoopAsync(CancellationToken cancellation)
{
var buffer = new byte[4 * 1024];
while (!cancellation.IsCancellationRequested)
{
var result = await _socket.ReceiveAsync(buffer, cancellation);
if (result.MessageType == WebSocketMessageType.Close) break;
if (result.MessageType != WebSocketMessageType.Text) continue;
var text = Encoding.UTF8.GetString(buffer, 0, result.Count);
CursorMessage? msg;
try
{
msg = JsonSerializer.Deserialize(text, WireJsonContext.Default.CursorMessage);
}
catch (JsonException) { continue; }
if (msg is null || msg.Type != "cursor") continue;
await _cursorInbox.Writer.WriteAsync(
(Clamp01(msg.X), Clamp01(msg.Y)), cancellation);
}
}
private async Task TickerLoopAsync(CancellationToken cancellation)
{
using var timer = new PeriodicTimer(TimeSpan.FromMilliseconds(20)); // 50 Hz
while (await timer.WaitForNextTickAsync(cancellation))
{
// Drain the cursor mailbox — keep only the latest update.
while (_cursorInbox.Reader.TryRead(out var next)) _currentCursor = next;
int action = _policy.SelectAction(_lastObs);
var step = await _game.StepAsync(action, _currentCursor);
_lastObs = step.Observation;
var frame = new StateFrame
{
X = step.State.X,
Y = step.State.Y,
Angle = step.State.Angle,
Engine = step.State.Engine,
Target = new[] { _currentCursor.X, _currentCursor.Y },
Step = step.Step,
};
await SendJsonAsync(frame, WireJsonContext.Default.StateFrame, cancellation);
}
}
private async Task SendJsonAsync<T>(
T value,
System.Text.Json.Serialization.Metadata.JsonTypeInfo<T> ctx,
CancellationToken cancellation)
{
var bytes = JsonSerializer.SerializeToUtf8Bytes(value, ctx);
await _socket.SendAsync(bytes, WebSocketMessageType.Text,
endOfMessage: true, cancellation);
}
private static float Clamp01(float v) => v < 0f ? 0f : (v > 1f ? 1f : v);
public async ValueTask DisposeAsync()
{
try
{
if (_socket.State == WebSocketState.Open)
await _socket.CloseAsync(WebSocketCloseStatus.NormalClosure,
"session-end", CancellationToken.None);
}
catch { }
await _game.DisposeAsync();
}
}
```
- [ ] **Step 2: Verify build**
```bash
dotnet build
```
Expected: 0 errors. If warnings appear about nullable-analysis on the `msg.Type` check, ignore them.
- [ ] **Step 3: Commit**
```bash
git add Backend/GameSession.cs
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(backend): GameSession runs reader + ticker loops per WebSocket connection"
```
---
## Task 6: `Program.cs` — startup wiring
**Files:**
- Modify: `Backend/Program.cs` (overwrite the Task 1 stub)
- Create: `Backend/LanderConfig.cs`
- [ ] **Step 1: Write `Backend/LanderConfig.cs`**
```csharp
namespace Backend;
public sealed class LanderConfig
{
public string CliPath { get; set; } = "publish/GameCli/GameCli";
public string ModelPath { get; set; } = "models/ppo_lander.onnx";
}
/// <summary>
/// Walks up from <see cref="AppContext.BaseDirectory"/> to find the repo root
/// (marker file: <c>GameCli.sln</c>) and resolves the configured paths against it.
/// </summary>
public static class PathResolver
{
public static string RepoRoot()
{
var dir = AppContext.BaseDirectory;
while (dir is not null)
{
if (File.Exists(Path.Combine(dir, "GameCli.sln"))) return dir;
var parent = Directory.GetParent(dir);
if (parent is null) break;
dir = parent.FullName;
}
throw new InvalidOperationException(
"could not locate repo root (no GameCli.sln found in any ancestor)");
}
public static string Resolve(string relativeOrAbsolute)
{
if (Path.IsPathRooted(relativeOrAbsolute)) return relativeOrAbsolute;
return Path.GetFullPath(Path.Combine(RepoRoot(), relativeOrAbsolute));
}
}
```
- [ ] **Step 2: Overwrite `Backend/Program.cs`**
```csharp
using System.Net.WebSockets;
using Backend;
var builder = WebApplication.CreateBuilder(args);
// -------- Config --------
builder.Services.Configure<LanderConfig>(builder.Configuration.GetSection("Lander"));
// -------- Singletons --------
builder.Services.AddSingleton<PolicyRunner>(sp =>
{
var cfg = sp.GetRequiredService<Microsoft.Extensions.Options.IOptions<LanderConfig>>().Value;
var absolute = PathResolver.Resolve(cfg.ModelPath);
return new PolicyRunner(absolute);
});
builder.Services.AddLogging();
var app = builder.Build();
app.UseWebSockets();
// -------- Health --------
app.MapGet("/", () => "GameCli Backend");
// -------- WebSocket endpoint --------
app.Map("/ws/game", async (HttpContext ctx,
PolicyRunner policy,
Microsoft.Extensions.Options.IOptions<LanderConfig> cfg,
ILoggerFactory loggerFactory) =>
{
if (!ctx.WebSockets.IsWebSocketRequest)
{
ctx.Response.StatusCode = 400;
return;
}
using var socket = await ctx.WebSockets.AcceptWebSocketAsync();
var cliPath = PathResolver.Resolve(cfg.Value.CliPath);
var proc = new GameProcess(cliPath);
var logger = loggerFactory.CreateLogger<GameSession>();
await using var session = new GameSession(socket, proc, policy, logger);
try
{
await session.RunAsync(ctx.RequestAborted);
}
catch (WebSocketException) { /* client disconnected */ }
catch (OperationCanceledException) { /* shutdown */ }
});
app.Run();
// -------- Public program class so Backend.Tests can use WebApplicationFactory --------
public partial class Program { }
```
- [ ] **Step 3: Verify it builds and starts**
```bash
dotnet build
dotnet run --project Backend --urls http://localhost:5100 &
BACKEND_PID=$!
sleep 3
curl -sf http://localhost:5100/ && echo
kill $BACKEND_PID
wait $BACKEND_PID 2>/dev/null
```
Expected:
- Build succeeds.
- The GET returns `GameCli Backend`.
- Backend logs on startup that it loaded `models/ppo_lander.onnx` (visible in the console output before curl).
- [ ] **Step 4: Commit**
```bash
git add Backend/Program.cs Backend/LanderConfig.cs
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(backend): wire ASP.NET startup with WebSocket endpoint and DI"
```
---
## Task 7: End-to-end smoke test — WebSocket round-trip
**Files:**
- Create: `Backend.Tests/EndToEndSmokeTests.cs`
Uses `WebApplicationFactory` + `ClientWebSocket` to test the whole stack:
1. Start the backend in-process.
2. Open a WebSocket to `/ws/game`.
3. Receive `init` frame.
4. Send a cursor update.
5. Receive a `state` frame within a reasonable timeout.
6. Verify shape.
- [ ] **Step 1: Write the failing test**
Create `Backend.Tests/EndToEndSmokeTests.cs`:
```csharp
using System.IO;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using Backend;
using Microsoft.AspNetCore.Mvc.Testing;
using Xunit;
namespace Backend.Tests;
public class EndToEndSmokeTests : IClassFixture<WebApplicationFactory<Program>>
{
private readonly WebApplicationFactory<Program> _factory;
public EndToEndSmokeTests(WebApplicationFactory<Program> factory)
{
_factory = factory.WithWebHostBuilder(b =>
{
// Prevent MVC picking up assemblies; force env vars if needed.
b.UseEnvironment("Development");
});
}
[Fact]
public async Task WebSocket_ReceivesInitAndStateFrames()
{
// Prerequisites: publish/GameCli/GameCli and models/ppo_lander.onnx must exist.
var repoRoot = FindRepoRoot();
Assert.True(File.Exists(Path.Combine(repoRoot, "publish", "GameCli", "GameCli")),
"run `dotnet publish GameCli -c Release -o publish/GameCli` before this test");
Assert.True(File.Exists(Path.Combine(repoRoot, "models", "ppo_lander.onnx")),
"run `python Training/export_onnx.py ...` before this test");
var client = _factory.Server.CreateWebSocketClient();
var baseUri = _factory.Server.BaseAddress;
var wsUri = new UriBuilder(baseUri) { Scheme = "ws", Path = "/ws/game" }.Uri;
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15));
using var ws = await client.ConnectAsync(wsUri, cts.Token);
var initText = await ReceiveTextAsync(ws, cts.Token);
using var initDoc = JsonDocument.Parse(initText);
Assert.Equal("init", initDoc.RootElement.GetProperty("type").GetString());
// Send a cursor and receive at least one state frame.
var cursorPayload = """{"type":"cursor","x":0.3,"y":0.7}""";
await ws.SendAsync(Encoding.UTF8.GetBytes(cursorPayload),
WebSocketMessageType.Text, true, cts.Token);
// Read up to N frames looking for a state frame; the ticker runs at 50 Hz.
for (int i = 0; i < 100; i++)
{
var text = await ReceiveTextAsync(ws, cts.Token);
using var doc = JsonDocument.Parse(text);
if (doc.RootElement.GetProperty("type").GetString() == "state")
{
Assert.True(doc.RootElement.TryGetProperty("x", out _));
Assert.True(doc.RootElement.TryGetProperty("y", out _));
Assert.True(doc.RootElement.TryGetProperty("angle", out _));
Assert.True(doc.RootElement.TryGetProperty("engine", out _));
Assert.True(doc.RootElement.TryGetProperty("target", out _));
Assert.True(doc.RootElement.TryGetProperty("step", out _));
return;
}
}
Assert.Fail("did not receive a state frame in 100 messages");
}
private static async Task<string> ReceiveTextAsync(WebSocket ws, CancellationToken ct)
{
var buffer = new byte[16 * 1024];
var sb = new StringBuilder();
WebSocketReceiveResult result;
do
{
result = await ws.ReceiveAsync(buffer, ct);
sb.Append(Encoding.UTF8.GetString(buffer, 0, result.Count));
} while (!result.EndOfMessage);
return sb.ToString();
}
private static string FindRepoRoot()
{
var dir = AppContext.BaseDirectory;
while (dir is not null)
{
if (File.Exists(Path.Combine(dir, "GameCli.sln"))) return dir;
var parent = Directory.GetParent(dir);
if (parent is null) break;
dir = parent.FullName;
}
throw new InvalidOperationException("repo root not found");
}
}
```
- [ ] **Step 2: Publish CLI and confirm ONNX exists**
```bash
dotnet publish GameCli/GameCli.csproj -c Release -o publish/GameCli
ls -la models/ppo_lander.onnx
```
- [ ] **Step 3: Run the smoke test**
```bash
dotnet test --filter FullyQualifiedName~EndToEndSmokeTests
```
Expected: 1 passed. The test should complete within ~5 seconds (init frame + ~1 tick).
- [ ] **Step 4: Run the full Backend test suite**
```bash
dotnet test --filter FullyQualifiedName~Backend.Tests
```
Expected: 3 GameProcess + 4 PolicyRunner + 1 smoke = 8 passed, 0 failed.
- [ ] **Step 5: Commit**
```bash
git add Backend.Tests/EndToEndSmokeTests.cs
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "test(backend): end-to-end WebSocket smoke test via WebApplicationFactory"
```
---
## Definition of done for Plan 3
- `dotnet test --filter FullyQualifiedName~Backend.Tests` reports 8 passed, 0 failed.
- `dotnet run --project Backend` starts, logs the ONNX model load, and responds to `GET /` with `GameCli Backend`.
- Connecting a WebSocket client to `ws://localhost:5100/ws/game` receives an `init` frame followed by 50 Hz `state` frames.
- No file exceeds ~250 lines; every source file has a single clear responsibility.
The next plan (Plan 4) will build the React + Vite frontend that opens this WebSocket, streams the mouse cursor, and renders the ship on a canvas.
Reference in New Issue View Git Blame Copy Permalink