admin/Experiment_ReinforcementLearning
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Plan 4: React frontend plan

Browse Source
This commit is contained in:
meelstorm
2026-07-17 17:50:13 +00:00
committed by EugeneTes
parent f5020726ec
commit b6a0709c3e
1 changed files with 825 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

825
docs/superpowers/plans/2026-07-17-04-frontend.md Normal file
View File

@@ -0,0 +1,825 @@
# React Frontend Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the browser-side of the design: a React app that opens a WebSocket to `ws://localhost:5100/ws/game`, streams mouse-position updates to the backend at ~50 Hz, receives ship state each tick, and renders it on an HTML canvas. Reference the design spec at `docs/superpowers/specs/2026-07-17-cursor-following-lander-design.md` §4.
**Architecture:** Vite + React + TypeScript. Three focused hooks (`useGameSocket`, `useCursorSender`, `useAnimationLoop`) + one canvas component. No routing, no state management library, no game engine — just the platform.
**Tech Stack:** Node.js 20+, Vite, React 18+, TypeScript. Runtime browser API only (`WebSocket`, `Canvas 2D`, `requestAnimationFrame`, `MouseEvent`).
---
## Prerequisites
- **Node.js 20+** available as `node`. Verify with `node --version`.
- **Plan 3 backend** works — this frontend expects `ws://localhost:5100/ws/game` to reply. That backend must be startable on demand for manual smoke.
---
## File structure
```
Experiment_ReinforcementLearning/
├── Frontend/
│ ├── package.json
│ ├── tsconfig.json
│ ├── tsconfig.node.json
│ ├── vite.config.ts
│ ├── index.html
│ └── src/
│ ├── main.tsx
│ ├── App.tsx
│ ├── LanderCanvas.tsx
│ ├── hooks/
│ │ ├── useGameSocket.ts
│ │ ├── useCursorSender.ts
│ │ └── useAnimationLoop.ts
│ ├── protocol.ts # TS types for wire messages
│ └── render.ts # pure canvas draw functions
```
Each hook has one responsibility. `render.ts` is pure functions — testable without a DOM.
---
## Task 1: Scaffold the Vite + React + TypeScript project
**Files:**
- Create: `Frontend/package.json` (via Vite template)
- Create: `Frontend/vite.config.ts`
- Create: `Frontend/tsconfig.json`
- Create: `Frontend/index.html`
- Create: `Frontend/src/main.tsx`
- [ ] **Step 1: Check Node is available**
Run: `node --version`
Expected: `v20.x` or higher. If missing, STOP and report BLOCKED.
- [ ] **Step 2: Scaffold with Vite**
Run from the repo root:
```bash
npm create vite@latest Frontend -- --template react-ts
```
This may prompt "Ok to proceed? (y)" — pass `--yes` or reply `y`.
- [ ] **Step 3: Install dependencies**
```bash
cd Frontend
npm install
cd ..
```
Expected: `Frontend/node_modules/` created, no errors.
- [ ] **Step 4: Clean out template placeholders**
The Vite template creates a demo counter component. Delete files we won't use:
```bash
rm -f Frontend/src/App.css
rm -f Frontend/src/index.css
rm -f Frontend/src/assets/react.svg
rm -f Frontend/public/vite.svg
```
Overwrite `Frontend/src/App.tsx` with a placeholder (Task 6 rewrites it):
```tsx
export function App() {
return <div style={{ padding: 20 }}>Frontend placeholder</div>;
}
```
Overwrite `Frontend/src/main.tsx` (removing CSS imports the template added):
```tsx
import { StrictMode } from 'react';
import { createRoot } from 'react-dom/client';
import { App } from './App';
const root = document.getElementById('root')!;
createRoot(root).render(
<StrictMode>
<App />
</StrictMode>
);
```
Overwrite `Frontend/index.html`:
```html
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Cursor-Following Lander</title>
<style>
html, body, #root { margin: 0; padding: 0; height: 100%; overflow: hidden; background: #000; }
body { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; color: #eee; }
</style>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
```
- [ ] **Step 5: Configure Vite dev proxy for the backend WebSocket**
Overwrite `Frontend/vite.config.ts`:
```ts
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
// https://vitejs.dev/config/
export default defineConfig({
plugins: [react()],
server: {
port: 5173,
proxy: {
'/ws': {
target: 'ws://localhost:5100',
ws: true,
changeOrigin: true,
},
},
},
});
```
This lets the frontend reach the backend via `ws://localhost:5173/ws/game` (same origin as the dev server), which Vite forwards to `localhost:5100`.
- [ ] **Step 6: Verify it builds and serves**
```bash
cd Frontend
npm run build
```
Expected: `dist/` directory created, no TypeScript errors.
Dev server smoke (in background):
```bash
npm run dev &
DEV_PID=$!
sleep 3
curl -sf http://localhost:5173/ | head -5
kill $DEV_PID
wait $DEV_PID 2>/dev/null
cd ..
```
Expected: the served HTML contains `<div id="root">` and the placeholder script tag.
- [ ] **Step 7: Commit**
```bash
git add Frontend/package.json Frontend/package-lock.json Frontend/vite.config.ts \
Frontend/tsconfig.json Frontend/tsconfig.node.json Frontend/tsconfig.app.json \
Frontend/index.html Frontend/src/
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(frontend): scaffold Vite + React + TypeScript project"
```
---
## Task 2: `protocol.ts` — wire message TypeScript types
**Files:**
- Create: `Frontend/src/protocol.ts`
Mirror the backend's WireMessages exactly. This is the source of truth for what the frontend expects on the wire.
- [ ] **Step 1: Write `Frontend/src/protocol.ts`**
```ts
// Wire messages exchanged with the backend over WebSocket.
// Mirror of Backend/WireMessages.cs.
export interface CursorMessage {
type: 'cursor';
x: number;
y: number;
}
export interface InitFrame {
type: 'init';
world: [number, number];
}
export interface StateFrame {
type: 'state';
x: number;
y: number;
angle: number;
engine: number;
target: [number, number];
step: number;
}
export type ServerFrame = InitFrame | StateFrame;
export function isInit(frame: ServerFrame): frame is InitFrame {
return frame.type === 'init';
}
export function isState(frame: ServerFrame): frame is StateFrame {
return frame.type === 'state';
}
```
- [ ] **Step 2: Verify TypeScript compiles**
```bash
cd Frontend && npx tsc --noEmit && cd ..
```
Expected: no errors.
- [ ] **Step 3: Commit**
```bash
git add Frontend/src/protocol.ts
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(frontend): wire message TypeScript types"
```
---
## Task 3: `useGameSocket` hook
**Files:**
- Create: `Frontend/src/hooks/useGameSocket.ts`
Opens the WebSocket on mount, parses inbound frames, exposes `{init, state, connected, send}` as React state.
- [ ] **Step 1: Write `Frontend/src/hooks/useGameSocket.ts`**
```ts
import { useEffect, useRef, useState } from 'react';
import type { InitFrame, ServerFrame, StateFrame, CursorMessage } from '../protocol';
import { isInit, isState } from '../protocol';
export interface GameSocketState {
connected: boolean;
init: InitFrame | null;
state: StateFrame | null;
sendCursor: (x: number, y: number) => void;
}
/**
* Opens a WebSocket to the given URL, tracks the latest init/state frames.
* Auto-reconnects with exponential backoff on close.
*/
export function useGameSocket(url: string): GameSocketState {
const [connected, setConnected] = useState(false);
const [init, setInit] = useState<InitFrame | null>(null);
const [state, setState] = useState<StateFrame | null>(null);
const wsRef = useRef<WebSocket | null>(null);
useEffect(() => {
let closed = false;
let backoffMs = 500;
let reconnectTimer: number | null = null;
const open = () => {
const ws = new WebSocket(url);
wsRef.current = ws;
ws.onopen = () => {
setConnected(true);
backoffMs = 500; // reset backoff on successful connect
};
ws.onclose = () => {
setConnected(false);
wsRef.current = null;
if (!closed) {
reconnectTimer = window.setTimeout(open, backoffMs);
backoffMs = Math.min(backoffMs * 2, 8000);
}
};
ws.onerror = () => { /* let onclose handle it */ };
ws.onmessage = (ev) => {
let frame: ServerFrame;
try {
frame = JSON.parse(ev.data) as ServerFrame;
} catch {
return;
}
if (isInit(frame)) setInit(frame);
else if (isState(frame)) setState(frame);
};
};
open();
return () => {
closed = true;
if (reconnectTimer !== null) window.clearTimeout(reconnectTimer);
wsRef.current?.close();
};
}, [url]);
const sendCursor = (x: number, y: number) => {
const ws = wsRef.current;
if (!ws || ws.readyState !== WebSocket.OPEN) return;
const msg: CursorMessage = { type: 'cursor', x, y };
ws.send(JSON.stringify(msg));
};
return { connected, init, state, sendCursor };
}
```
- [ ] **Step 2: Verify TypeScript compiles**
```bash
cd Frontend && npx tsc --noEmit && cd ..
```
Expected: no errors.
- [ ] **Step 3: Commit**
```bash
git add Frontend/src/hooks/useGameSocket.ts
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(frontend): useGameSocket hook with auto-reconnect"
```
---
## Task 4: `useCursorSender` hook
**Files:**
- Create: `Frontend/src/hooks/useCursorSender.ts`
Attaches a `mousemove` listener to a target element (the canvas). Converts pixel coords → world coords in `[0,1] × [0,1]` using letterbox mapping. Throttles sends to ~50 Hz using a `requestAnimationFrame` timestamp gate.
- [ ] **Step 1: Write `Frontend/src/hooks/useCursorSender.ts`**
```ts
import { useEffect, useRef } from 'react';
/**
* Convert a viewport point to world coords in [0,1] × [0,1] using
* letterbox mapping (preserves world aspect ratio 1:1).
*/
export function viewportToWorld(
viewportPx: { x: number; y: number },
viewportSize: { w: number; h: number },
): { x: number; y: number } {
// World is 1×1. Fit inside viewport with black bars on the wider axis.
const scale = Math.min(viewportSize.w, viewportSize.h);
const offsetX = (viewportSize.w - scale) / 2;
const offsetY = (viewportSize.h - scale) / 2;
return {
x: Math.max(0, Math.min(1, (viewportPx.x - offsetX) / scale)),
y: Math.max(0, Math.min(1, (viewportPx.y - offsetY) / scale)),
};
}
const SEND_INTERVAL_MS = 20; // 50 Hz cap
export function useCursorSender(
targetRef: React.RefObject<HTMLElement>,
onSend: (x: number, y: number) => void,
) {
const lastCursorRef = useRef<{ x: number; y: number } | null>(null);
const lastSentAtRef = useRef<number>(0);
const rafRef = useRef<number | null>(null);
useEffect(() => {
const el = targetRef.current;
if (!el) return;
const handleMove = (ev: MouseEvent) => {
const rect = el.getBoundingClientRect();
const world = viewportToWorld(
{ x: ev.clientX - rect.left, y: ev.clientY - rect.top },
{ w: rect.width, h: rect.height },
);
lastCursorRef.current = world;
};
const tick = (now: number) => {
const c = lastCursorRef.current;
if (c && now - lastSentAtRef.current >= SEND_INTERVAL_MS) {
onSend(c.x, c.y);
lastSentAtRef.current = now;
}
rafRef.current = requestAnimationFrame(tick);
};
el.addEventListener('mousemove', handleMove);
rafRef.current = requestAnimationFrame(tick);
return () => {
el.removeEventListener('mousemove', handleMove);
if (rafRef.current !== null) cancelAnimationFrame(rafRef.current);
};
}, [targetRef, onSend]);
}
```
- [ ] **Step 2: Verify TypeScript compiles**
```bash
cd Frontend && npx tsc --noEmit && cd ..
```
Expected: no errors.
- [ ] **Step 3: Commit**
```bash
git add Frontend/src/hooks/useCursorSender.ts
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(frontend): useCursorSender hook with letterbox mapping and 50 Hz throttle"
```
---
## Task 5: `render.ts` — pure canvas draw functions
**Files:**
- Create: `Frontend/src/render.ts`
Pure functions that draw the scene on a `CanvasRenderingContext2D`. Testable without a DOM (Vitest is not being set up in this plan — this is more about keeping the module isolated for correctness).
- [ ] **Step 1: Write `Frontend/src/render.ts`**
```ts
import type { StateFrame } from './protocol';
/** Ship as a filled triangle, drawn at (x,y) rotated by angle. Size in canvas px. */
export function drawShip(
ctx: CanvasRenderingContext2D,
worldX: number,
worldY: number,
angle: number,
engine: number,
scale: number,
offsetX: number,
offsetY: number,
) {
const px = offsetX + worldX * scale;
const py = offsetY + worldY * scale;
const size = scale * 0.04;
ctx.save();
ctx.translate(px, py);
ctx.rotate(angle);
// Body: triangle pointing up (angle=0 → up).
ctx.fillStyle = '#eee';
ctx.beginPath();
ctx.moveTo(0, -size);
ctx.lineTo(size * 0.7, size * 0.6);
ctx.lineTo(-size * 0.7, size * 0.6);
ctx.closePath();
ctx.fill();
// Flame if an engine is firing.
if (engine !== 0) {
ctx.fillStyle = '#ff9d3d';
ctx.beginPath();
if (engine === 2) {
// Main engine: flame under the ship.
const flame = size * 0.9 + Math.random() * size * 0.4;
ctx.moveTo(-size * 0.4, size * 0.6);
ctx.lineTo(size * 0.4, size * 0.6);
ctx.lineTo(0, size * 0.6 + flame);
} else if (engine === 1) {
// Left thruster: flame on right side of body.
ctx.moveTo(size * 0.7, -size * 0.2);
ctx.lineTo(size * 0.7, size * 0.2);
ctx.lineTo(size * 1.3, 0);
} else if (engine === 3) {
// Right thruster: flame on left side of body.
ctx.moveTo(-size * 0.7, -size * 0.2);
ctx.lineTo(-size * 0.7, size * 0.2);
ctx.lineTo(-size * 1.3, 0);
}
ctx.closePath();
ctx.fill();
}
ctx.restore();
}
/** Draw a crosshair at the target world position. */
export function drawTarget(
ctx: CanvasRenderingContext2D,
worldX: number,
worldY: number,
scale: number,
offsetX: number,
offsetY: number,
) {
const px = offsetX + worldX * scale;
const py = offsetY + worldY * scale;
const r = 8;
ctx.strokeStyle = '#5cf';
ctx.lineWidth = 1.5;
ctx.beginPath();
ctx.arc(px, py, r, 0, Math.PI * 2);
ctx.stroke();
ctx.beginPath();
ctx.moveTo(px - r * 1.5, py);
ctx.lineTo(px + r * 1.5, py);
ctx.moveTo(px, py - r * 1.5);
ctx.lineTo(px, py + r * 1.5);
ctx.stroke();
}
/** Layout: fit the 1×1 world into the canvas with black letterbox bars. */
export function layout(canvas: HTMLCanvasElement) {
const scale = Math.min(canvas.width, canvas.height);
const offsetX = (canvas.width - scale) / 2;
const offsetY = (canvas.height - scale) / 2;
return { scale, offsetX, offsetY };
}
/** Render one frame: clear + world background + target + ship. */
export function renderFrame(
canvas: HTMLCanvasElement,
state: StateFrame | null,
) {
const ctx = canvas.getContext('2d');
if (!ctx) return;
// Clear
ctx.fillStyle = '#000';
ctx.fillRect(0, 0, canvas.width, canvas.height);
const { scale, offsetX, offsetY } = layout(canvas);
// World background (subtle dark band so the play area is visible).
ctx.fillStyle = '#0a0a12';
ctx.fillRect(offsetX, offsetY, scale, scale);
if (!state) return;
drawTarget(ctx, state.target[0], state.target[1], scale, offsetX, offsetY);
drawShip(ctx, state.x, state.y, state.angle, state.engine, scale, offsetX, offsetY);
}
```
- [ ] **Step 2: Verify TypeScript compiles**
```bash
cd Frontend && npx tsc --noEmit && cd ..
```
Expected: no errors.
- [ ] **Step 3: Commit**
```bash
git add Frontend/src/render.ts
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(frontend): pure canvas draw functions (ship + flame + target + layout)"
```
---
## Task 6: `useAnimationLoop`, `LanderCanvas`, `App` — wire it all up
**Files:**
- Create: `Frontend/src/hooks/useAnimationLoop.ts`
- Create: `Frontend/src/LanderCanvas.tsx`
- Modify: `Frontend/src/App.tsx`
- [ ] **Step 1: Write `Frontend/src/hooks/useAnimationLoop.ts`**
```ts
import { useEffect, useRef } from 'react';
/** Calls `callback` once per browser frame. Runs while mounted. */
export function useAnimationLoop(callback: (now: number) => void) {
const cbRef = useRef(callback);
cbRef.current = callback;
useEffect(() => {
let rafId = 0;
const loop = (now: number) => {
cbRef.current(now);
rafId = requestAnimationFrame(loop);
};
rafId = requestAnimationFrame(loop);
return () => cancelAnimationFrame(rafId);
}, []);
}
```
- [ ] **Step 2: Write `Frontend/src/LanderCanvas.tsx`**
```tsx
import { useEffect, useRef } from 'react';
import type { StateFrame } from './protocol';
import { renderFrame } from './render';
import { useAnimationLoop } from './hooks/useAnimationLoop';
import { useCursorSender } from './hooks/useCursorSender';
export interface LanderCanvasProps {
state: StateFrame | null;
onCursor: (x: number, y: number) => void;
}
/**
* Full-viewport canvas. Redraws every animation frame reading the latest
* `state` prop (no interpolation). Mouse movement is captured on the canvas
* and forwarded through `onCursor`.
*/
export function LanderCanvas({ state, onCursor }: LanderCanvasProps) {
const canvasRef = useRef<HTMLCanvasElement>(null);
const stateRef = useRef<StateFrame | null>(state);
stateRef.current = state;
// Keep the canvas's backing store in sync with its CSS size + devicePixelRatio.
useEffect(() => {
const canvas = canvasRef.current;
if (!canvas) return;
const resize = () => {
const dpr = window.devicePixelRatio ?? 1;
const rect = canvas.getBoundingClientRect();
canvas.width = Math.floor(rect.width * dpr);
canvas.height = Math.floor(rect.height * dpr);
};
resize();
window.addEventListener('resize', resize);
return () => window.removeEventListener('resize', resize);
}, []);
useAnimationLoop(() => {
const canvas = canvasRef.current;
if (!canvas) return;
renderFrame(canvas, stateRef.current);
});
useCursorSender(canvasRef as React.RefObject<HTMLElement>, onCursor);
return (
<canvas
ref={canvasRef}
style={{ display: 'block', width: '100vw', height: '100vh', cursor: 'crosshair' }}
/>
);
}
```
- [ ] **Step 3: Overwrite `Frontend/src/App.tsx`**
```tsx
import { useGameSocket } from './hooks/useGameSocket';
import { LanderCanvas } from './LanderCanvas';
// Vite dev server proxies `/ws` → `ws://localhost:5100`. In prod the backend
// serves the built static files, so same-origin works there too.
function buildWsUrl(): string {
const proto = window.location.protocol === 'https:' ? 'wss:' : 'ws:';
return `${proto}//${window.location.host}/ws/game`;
}
export function App() {
const { connected, state, sendCursor } = useGameSocket(buildWsUrl());
return (
<>
<LanderCanvas state={state} onCursor={sendCursor} />
<StatusBar connected={connected} step={state?.step ?? 0} />
</>
);
}
interface StatusBarProps { connected: boolean; step: number; }
function StatusBar({ connected, step }: StatusBarProps) {
return (
<div style={{
position: 'fixed', top: 8, left: 8, padding: '4px 10px',
background: 'rgba(0,0,0,0.5)', borderRadius: 4, fontSize: 12,
pointerEvents: 'none',
}}>
<span style={{ color: connected ? '#5f5' : '#f55' }}>●</span>{' '}
{connected ? 'connected' : 'disconnected'} · step {step}
</div>
);
}
```
- [ ] **Step 4: Verify TypeScript compiles and build passes**
```bash
cd Frontend && npx tsc --noEmit && npm run build && cd ..
```
Expected: no TypeScript errors, `dist/` produced.
- [ ] **Step 5: Commit**
```bash
git add Frontend/src/hooks/useAnimationLoop.ts Frontend/src/LanderCanvas.tsx Frontend/src/App.tsx
git -c user.email=meelstorm@gmail.com -c user.name=meelstorm commit -m "feat(frontend): LanderCanvas + App wiring cursor and state to WebSocket"
```
---
## Task 7: End-to-end manual verification
**Files:** (none — verification only)
This task launches everything at once (backend + frontend) and confirms the ship renders and moves.
- [ ] **Step 1: Kill any stale processes**
```bash
pkill -f "dotnet.*Backend" 2>/dev/null; sleep 1
pkill -f "vite" 2>/dev/null; sleep 1
```
- [ ] **Step 2: Publish CLI (idempotent)**
```bash
dotnet publish GameCli/GameCli.csproj -c Release -o publish/GameCli
```
- [ ] **Step 3: Start the backend in the background**
```bash
ASPNETCORE_URLS=http://localhost:5100 dotnet run --project Backend --no-launch-profile > /tmp/backend.log 2>&1 &
BACKEND_PID=$!
sleep 5
grep -q "Loaded PPO policy" /tmp/backend.log && echo "backend loaded ONNX" || (echo "backend startup failed"; cat /tmp/backend.log; exit 1)
curl -sf http://localhost:5100/ && echo
```
Expected: `backend loaded ONNX` and health returns `GameCli Backend`.
- [ ] **Step 4: Start the Vite dev server**
```bash
cd Frontend
npm run dev > /tmp/vite.log 2>&1 &
VITE_PID=$!
cd ..
sleep 5
curl -sf http://localhost:5173/ | grep -q "Cursor-Following Lander" && echo "vite serving index"
```
Expected: `vite serving index`.
- [ ] **Step 5: WebSocket handshake test through Vite's proxy**
Use a small Node script or curl to prove the WS proxies through. Node one-liner (assumes `ws` is available; if not, skip this step and rely on the browser check below).
Simpler: just use `curl` to hit the frontend and confirm no 500 errors. The real WS test is the browser check.
- [ ] **Step 6: Browser check (manual — the deliverable)**
Open `http://localhost:5173/` in a browser. You should see:
- Black background with a slightly-lighter square (the world).
- A blue crosshair follows your mouse.
- A small white triangular ship appears somewhere in the world, twitching / firing engines as the (smoke-trained) PPO policy tries to control it.
- Top-left corner shows `● connected · step N` with N ticking up.
If the ship never appears or the socket says `disconnected`, capture what's on screen and the browser DevTools Network → WS tab for the `ws/game` connection to diagnose.
- [ ] **Step 7: Teardown**
```bash
kill $VITE_PID $BACKEND_PID 2>/dev/null
wait $VITE_PID $BACKEND_PID 2>/dev/null
```
- [ ] **Step 8: No commit — verification only**
---
## Definition of done for Plan 4
- `npx tsc --noEmit` in `Frontend/` reports 0 errors.
- `npm run build` produces `Frontend/dist/`.
- With backend running on 5100 and Vite dev server on 5173, opening `http://localhost:5173/` shows a canvas with a ship that responds to the cursor.
- Every source file has one clear responsibility; no file exceeds ~150 lines.
## Notes on real training vs the smoke model
The ship's motion quality directly reflects how well-trained the PPO policy is. The current `models/ppo_lander.onnx` was trained for only 10k steps in Plan 2 — it's essentially random with a slight bias. To see actual cursor-following behavior:
```bash
source .venv/bin/activate
python Training/train.py --steps 2000000 --n-envs 8
python Training/export_onnx.py \
--checkpoint checkpoints/ppo_lander_final.zip \
--out models/ppo_lander.onnx
```
Then restart the backend so it re-loads the new ONNX. Training takes on the order of an hour on CPU.