leerform recipe subsystem created

2026-03-31 13:26:26 +02:00
parent 4bd117af47
commit 1c7a4abd5c
20 changed files with 823 additions and 70 deletions
--- a/docs/RECOGNITION_CONTROL.md
+++ b/docs/RECOGNITION_CONTROL.md
@@ -0,0 +1,543 @@
+# Recognition Control System
+
+The recognition control system manages the image processing lifecycle: acquiring images, running them through a processing pipeline, and emitting results as events. Each camera gets its own `IRecognitionControl` instance, registered in the per-camera Ninject child kernel.
+
+## Architecture Overview
+
+```
+IRecognitionControl (interface)
+    ^
+    |
+BaseRecognitionControl (abstract base - lifecycle, threading, events)
+    ^
+    |--- HawkeyeRecognitionControl (workflow-based pipeline processing)
+    |--- CandyboxImageProcessingControl (histogram-based processing)
+    |--- TestImageProcessingControl (minimal test stub)
+    |--- [Your custom implementation]
+```
+
+```
+BaseRecognitionControlSettings (abstract base - shared config)
+    ^
+    |--- HawkeyeRecognitionSettings
+    |--- CandyboxRecognitionControlSettings
+    |--- TestRecognitionControlSettings
+    |--- [Your custom settings]
+```
+
+## IRecognitionControl Interface
+
+Defined in `VisionBuilder.UI.Common/Processing/IRecognitionControl.cs`.
+
+```csharp
+public interface IRecognitionControl
+{
+    bool IsRunning { get; }
+    List<RecipeData> GetRecipesData();
+    void SetRecipe(RecipeData recipe);
+    void Start();
+    void Stop();
+    void Pause();
+    void Resume();
+    event Action<ImageProcessedEvent> ImageProcessed;
+    event Action<SessionStartedEvent> SessionStarted;
+    event Action<SessionEndedEvent> SessionEnded;
+    event Action<ErrorsInSequenceEvent> ErrorsInSequenceAlarm;
+    void HotReload();
+    event Action RecipeFileChanged;
+}
+```
+
+| Member | Description |
+|---|---|
+| `IsRunning` | Whether the processing loop is active. |
+| `GetRecipesData()` | Returns available recipes (name + optional preview image). |
+| `SetRecipe(recipe)` | Selects a recipe. Throws if `IsRunning` is true. |
+| `Start()` | Initializes the recipe, warms up, then starts the processing loop on a background task. |
+| `Stop()` | Cancels the processing loop, waits for it to finish, runs cleanup. |
+| `Pause()` / `Resume()` | Temporarily suspends/resumes image processing without stopping the session. |
+| `HotReload()` | Reloads recipe parameters without stopping the session. |
+| `ImageProcessed` | Fired after every processed image with full result data. |
+| `SessionStarted` / `SessionEnded` | Fired when a session starts/stops. |
+| `ErrorsInSequenceAlarm` | Fired when consecutive errors reach the configured threshold. |
+| `RecipeFileChanged` | Fired when the recipe file on disk changes (for hot-reload UI). |
+
+## BaseRecognitionControl
+
+Defined in `VisionBuilder.UI.Common/Processing/BaseRecognitionControl.cs`. This abstract class implements all of `IRecognitionControl` and provides the processing loop, threading, pause logic, error-in-sequence tracking, and event plumbing. Subclasses only need to implement four methods.
+
+### Abstract Methods to Implement
+
+```csharp
+// Load/prepare resources for the given recipe
+protected abstract void Initialize(RecipeData currentRecipe);
+
+// Run a throwaway processing pass to warm up caches, JIT, models, etc.
+protected abstract void WarmUp();
+
+// Acquire and process one image. Return null to stop the loop.
+protected abstract (
+    Mat originalImage,
+    Mat analysisImage,
+    TimeSpan processingTime,
+    TimeSpan acquisitionTime,
+    string[] errorNames
+)? ProcessImage(CancellationToken token);
+
+// Return the list of available recipes
+public abstract List<RecipeData> GetRecipesData();
+```
+
+### Virtual Methods (Optional Overrides)
+
+```csharp
+// Release resources when the session stops (called from Stop())
+protected virtual void Cleanup() { }
+
+// Reload recipe parameters without restarting the session
+public virtual void HotReload() { }
+```
+
+### Protected Helper
+
+```csharp
+// Call this to raise the RecipeFileChanged event (e.g. from a file watcher)
+protected void RaiseRecipeFileChanged();
+```
+
+### Processing Loop Behavior
+
+The `Start()` method:
+1. Validates that a recipe is set and the control is not already running.
+2. Shows a loading indicator via `ILoadingService`.
+3. Calls `Initialize(currentRecipe)`.
+4. Calls `WarmUp()` (errors are logged, not thrown).
+5. Launches `Loop()` on `Task.Run` with a `CancellationToken`.
+6. Fires `SessionStarted`.
+7. Hides the loading indicator.
+
+The `Loop()` runs continuously until cancellation:
+1. If paused, sleeps 100ms and continues.
+2. Calls `ProcessImage(token)`. If it returns `null`, the loop exits.
+3. If processing time is below `MinimumProcessingTime`, sleeps the difference.
+4. Tracks consecutive errors. If they reach `ErrorsInSequenceAlarm`, fires the alarm event.
+5. Builds an `ImageProcessedEvent` and fires it.
+
+The `Stop()` method:
+1. Cancels the token, waits for the loop task to finish.
+2. Calls `Cleanup()`.
+3. Fires `SessionEnded`.
+
+### Thread Safety
+
+All public lifecycle methods (`Start`, `Stop`, `Pause`, `Resume`, `SetRecipe`) are protected by a shared `lock`. The processing loop runs on a background `Task`. `ProcessImage` receives a `CancellationToken` and should respect it for timely shutdown.
+
+## BaseRecognitionControlSettings
+
+Defined in `VisionBuilder.UI.Common/Processing/BaseRecognitionControlSettings.cs`. Provides the shared configuration for all recognition controls.
+
+```csharp
+public abstract class BaseRecognitionControlSettings(string cameraName) : ISettings
+{
+    public string CameraName { get; }                    // Injected camera identifier
+    public int MinimumProcessingTime { get; set; }       // Floor for loop iteration (ms)
+    public string RecipeNameFilter { get; set; } = "*";  // Glob filter for recipe discovery
+    public int ErrorsInSequenceAlarm { get; set; } = 5;  // Consecutive errors before alarm
+}
+```
+
+All properties are registered in the `Inspectron.Settings` system under `{CameraName}/Recognition` by the `RegisterSettings` method. Override it to add custom settings (always call `base.RegisterSettings` first).
+
+## Events
+
+All event classes are in `VisionBuilder.UI.Common/Commands/`.
+
+### ImageProcessedEvent
+
+Fired after every image. Contains everything needed to update the UI and record statistics.
+
+```csharp
+public class ImageProcessedEvent : IEvent
+{
+    public DateTime SessionStart { get; set; }
+    public string ImageSource { get; set; }       // Camera name
+    public string RecipeName { get; set; }
+    public Mat ImageOriginal { get; set; }        // Raw camera image
+    public Mat ImageAnalysis { get; set; }        // Annotated/processed image
+    public bool HasError { get; set; }
+    public List<string> ErrorNames { get; set; }  // Names of failed operations
+    public TimeSpan AnalysisTime { get; set; }    // Processing minus acquisition
+    public TimeSpan AcquisitionTime { get; set; } // Camera capture time
+    public TimeSpan SleepTime { get; set; }       // MinimumProcessingTime padding
+}
+```
+
+### SessionStartedEvent / SessionEndedEvent
+
+```csharp
+public class SessionStartedEvent : IEvent
+{
+    public string RecipeName { get; set; }
+    public DateTime SessionStarted { get; set; }
+}
+
+public class SessionEndedEvent : IEvent
+{
+    public DateTime SessionEnded { get; set; }
+}
+```
+
+### ErrorsInSequenceEvent
+
+```csharp
+public class ErrorsInSequenceEvent
+{
+    public int Errors { get; set; }  // Current consecutive error count
+}
+```
+
+## RecipeData
+
+Defined in `VisionBuilder.UI.Common/ViewModel/Classes/RecipeData.cs`.
+
+```csharp
+public class RecipeData
+{
+    public Mat? Image { get; set; }        // Optional preview thumbnail
+    public string RecipeName { get; set; } // Recipe identifier (typically filename without extension)
+}
+```
+
+## ILoadingService
+
+Defined in `VisionBuilder.UI.Common/Processing/ILoadingService.cs`. Called by `BaseRecognitionControl` during `Start()` and `Stop()` to show/hide a loading overlay in the UI.
+
+```csharp
+public interface ILoadingService
+{
+    void StartLoading(string title);
+    void StopLoading(string title);
+}
+```
+
+The loading service is injected into `BaseRecognitionControl` via the constructor and exposed as the `LoadingService` protected property, so subclasses can use it in `GetRecipesData()` or elsewhere.
+
+## How to Implement a Custom Recognition Control
+
+### Step 1: Create Settings
+
+Extend `BaseRecognitionControlSettings` with any additional configuration your processing needs.
+
+```csharp
+using Inspectron.Settings;
+using VisionBuilder.UI.Common.Processing;
+
+public class MyRecognitionSettings(string cameraName) : BaseRecognitionControlSettings(cameraName)
+{
+    public double Threshold { get; set; } = 0.5;
+
+    public override void RegisterSettings(InspectronSettings settings)
+    {
+        base.RegisterSettings(settings);
+        settings.RegisterSimple(this, () => Threshold,
+            CameraName + "/Recognition", nameof(Threshold));
+    }
+}
+```
+
+The `RegisterSettings` call makes the property editable in the application's settings UI. Always call `base.RegisterSettings(settings)` first to register the base properties (`MinimumProcessingTime`, `RecipeNameFilter`, `ErrorsInSequenceAlarm`).
+
+### Step 2: Create the Recognition Control
+
+Extend `BaseRecognitionControl` and implement the four abstract members.
+
+```csharp
+using System.Diagnostics;
+using OpenCvSharp;
+using VisionBuilder.UI.Common.Processing;
+using VisionBuilder.UI.Common.ViewModel.Classes;
+
+public class MyRecognitionControl : BaseRecognitionControl
+{
+    private readonly MyRecognitionSettings _settings;
+    private readonly IImageSource _imageSource;
+    private MyProcessor _processor;  // your processing logic
+
+    public MyRecognitionControl(
+        MyRecognitionSettings settings,
+        IImageSource imageSource,
+        ILoadingService loadingService)
+        : base(settings, loadingService)
+    {
+        _settings = settings;
+        _imageSource = imageSource;
+    }
+
+    public override List<RecipeData> GetRecipesData()
+    {
+        // Return available recipes. Use _settings.RecipeNameFilter if applicable.
+        var recipes = new List<RecipeData>();
+        foreach (var file in Directory.GetFiles(@"..\Data\Recipes", "*.myrecipe"))
+        {
+            recipes.Add(new RecipeData
+            {
+                RecipeName = Path.GetFileNameWithoutExtension(file),
+                Image = null  // or load a preview thumbnail
+            });
+        }
+        return recipes;
+    }
+
+    protected override void Initialize(RecipeData currentRecipe)
+    {
+        // Load the recipe and prepare your processing pipeline.
+        // Called once when Start() is invoked.
+        var path = Path.Combine(@"..\Data\Recipes", currentRecipe.RecipeName + ".myrecipe");
+        _processor = MyProcessor.Load(path, _settings.Threshold);
+    }
+
+    protected override void WarmUp()
+    {
+        // Optional: run a throwaway pass to warm up JIT, GPU, ONNX models, etc.
+        // Errors here are caught and logged, they won't prevent startup.
+        var dummy = new Mat(100, 100, MatType.CV_8UC3, Scalar.All(0));
+        _processor.Process(dummy);
+        dummy.Dispose();
+    }
+
+    protected override (Mat originalImage, Mat analysisImage,
+        TimeSpan processingTime, TimeSpan acquisitionTime,
+        string[] errorNames)?
+        ProcessImage(CancellationToken token)
+    {
+        // 1. Acquire image
+        var swTotal = Stopwatch.StartNew();
+        var swAcquire = Stopwatch.StartNew();
+        var image = _imageSource.GetImage(token).Result;
+        swAcquire.Stop();
+
+        if (image == null)
+            return null;  // null signals "stop the loop"
+
+        // 2. Process
+        var result = _processor.Process(image);
+        swTotal.Stop();
+
+        // 3. Build annotated image
+        var annotated = image.Clone();
+        // ... draw overlays on annotated ...
+
+        // 4. Collect error names (empty array = pass)
+        string[] errors = result.Defects
+            .Select(d => d.Name)
+            .ToArray();
+
+        return (image, annotated, swTotal.Elapsed, swAcquire.Elapsed, errors);
+    }
+
+    protected override void Cleanup()
+    {
+        // Release resources when the session stops.
+        _processor?.Dispose();
+        _processor = null;
+    }
+}
+```
+
+### Key implementation notes for ProcessImage
+
+- **Return `null`** to signal the loop should exit (e.g. when the image source is exhausted or cancelled).
+- **Respect the `CancellationToken`** — pass it to blocking calls like `GetImage()`. Catch `OperationCanceledException` if you need cleanup before returning null.
+- **`errorNames`** drives the pass/fail logic. An empty array means the image passed. Each string becomes a named error in statistics and UI.
+- **`originalImage`** is the raw camera image; **`analysisImage`** is the annotated version shown in the preview UI.
+- **`processingTime`** is the total wall-clock time including acquisition; **`acquisitionTime`** is just the camera capture. The base class computes `AnalysisTime = processingTime - acquisitionTime` for the event.
+
+### Step 3: Register via DI
+
+Register your settings and control in the per-camera Ninject child kernel. This is typically done in a plugin's `RegisterCameraModules` or in an extension method.
+
+**Option A: In a plugin**
+
+```csharp
+public class MyPlugin : IPlugin
+{
+    public void RegisterGlobalModules(IKernel kernel) { }
+
+    public void RegisterCameraModules(IKernel kernel, string cameraName)
+    {
+        kernel.Bind<MyRecognitionSettings, BaseRecognitionControlSettings, ISettings>()
+            .ToConstant(new MyRecognitionSettings(cameraName));
+
+        kernel.Rebind<IRecognitionControl>()
+            .To<MyRecognitionControl>()
+            .InSingletonScope();
+    }
+}
+```
+
+**Option B: As an extension method (like HawkeyeRecipe)**
+
+```csharp
+public static class ModuleExtensions
+{
+    public static IChildKernel UseMyRecipes(this IChildKernel self, string cameraName)
+    {
+        self.Bind<MyRecognitionSettings, BaseRecognitionControlSettings, ISettings>()
+            .ToConstant(new MyRecognitionSettings(cameraName));
+        self.Bind<IRecognitionControl>()
+            .To<MyRecognitionControl>()
+            .InSingletonScope();
+        return self;
+    }
+}
+```
+
+**Important DI details:**
+- Bind your settings class to **three types**: itself, `BaseRecognitionControlSettings`, and `ISettings`. The `ISettings` binding makes `RegisterSettings` get called by the settings system. The `BaseRecognitionControlSettings` binding lets the base class constructor receive it.
+- Use `Rebind<IRecognitionControl>` (not `Bind`) if replacing an existing binding from another recipe provider.
+- Always use `InSingletonScope()` — there should be exactly one recognition control per camera.
+
+### Step 4: Hot Reload (Optional)
+
+To support live recipe parameter updates without stopping:
+
+1. Set up a `FileSystemWatcher` in `Initialize()` to monitor the recipe file.
+2. When the file changes, call `RaiseRecipeFileChanged()` to notify the UI.
+3. Override `HotReload()` to apply the new parameters on the next loop iteration.
+
+Example from `HawkeyeRecognitionControl`:
+
+```csharp
+private FileSystemWatcher? _fileWatcher;
+private System.Timers.Timer? _debounceTimer;
+private string? _pendingHotReloadFile;
+private string? _currentRecipeFilePath;
+
+protected override void Initialize(RecipeData currentRecipe)
+{
+    _currentRecipeFilePath = Path.GetFullPath(/* recipe path */);
+    _workflow = LoadRecipe(_currentRecipeFilePath);
+    StartFileWatcher(_currentRecipeFilePath);
+}
+
+private void StartFileWatcher(string filePath)
+{
+    var directory = Path.GetDirectoryName(filePath)!;
+    var fileName = Path.GetFileName(filePath);
+
+    _debounceTimer = new System.Timers.Timer(500) { AutoReset = false };
+    _debounceTimer.Elapsed += (_, _) => RaiseRecipeFileChanged();
+
+    _fileWatcher = new FileSystemWatcher(directory, fileName)
+    {
+        NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.Size,
+        EnableRaisingEvents = true
+    };
+    _fileWatcher.Changed += (_, _) =>
+    {
+        _debounceTimer.Stop();
+        _debounceTimer.Start();  // debounce to 500ms
+    };
+}
+
+public override void HotReload()
+{
+    // Store path; ProcessImage picks it up on the next iteration
+    Interlocked.Exchange(ref _pendingHotReloadFile, _currentRecipeFilePath);
+}
+
+protected override void Cleanup()
+{
+    _fileWatcher?.Dispose();
+    _debounceTimer?.Dispose();
+}
+```
+
+The UI flow: `RecipeFileChanged` event -> `SingleCameraVM` sets `IsHotReloadAvailable = true` -> user clicks hot-reload button -> calls `HotReload()` -> next `ProcessImage` iteration applies new parameters.
+
+## How the UI Consumes IRecognitionControl
+
+`SingleCameraVM` (in `VisionBuilder.UI.Common/ViewModel/SingleCameraVM.cs`) is the primary consumer:
+
+**Commands:**
+```csharp
+// Start/Stop run on background tasks to avoid blocking the UI
+await Task.Run(() => _recognitionControl.Start());
+await Task.Run(() => _recognitionControl.Stop());
+
+// Pause/Resume are synchronous (just flip a flag)
+_recognitionControl.Pause();
+_recognitionControl.Resume();
+
+// Recipe selection
+_recognitionControl.SetRecipe(selectedRecipe);
+
+// Hot reload
+_recognitionControl.HotReload();
+```
+
+**Event subscriptions (in constructor):**
+```csharp
+recognitionControl.ImageProcessed += Handle;         // -> PreviewVm, StatisticsVm, ErrorsVm
+recognitionControl.SessionStarted += StatisticsVm.Handle;
+recognitionControl.SessionStarted += ErrorsVm.Handle;
+recognitionControl.RecipeFileChanged += () =>
+    SynchronizationContext?.Post(_ => IsHotReloadAvailable = true, null);
+```
+
+**Other modules subscribing to events:**
+- `VisionBuilderStatistics` — writes CSV logs per session, tracks pass/fail counts.
+- `VisionBuilderRingbuffer` — saves images to ringbuffers (separate good/bad).
+- `IOCommanderCameraModule` — emits GPIO signals on session start/end, image result, and error alarms.
+- Plugin modules (e.g. `PralinenModule`) — custom PLC integration based on results.
+
+## Existing Implementations Reference
+
+### HawkeyeRecognitionControl
+
+**Project:** `VisionBuilder.UI.Recipes.HawkeyeRecipe`
+
+The primary implementation for workflow-based image processing.
+
+- **GetRecipesData**: Loads `.jhrcp` files from `Data/Recipes/`, deserializes each to extract recipe name and preview image.
+- **Initialize**: Loads the workflow JSON file, starts a file watcher for hot-reload.
+- **WarmUp**: Executes the workflow once with an `EmptyImageSource`.
+- **ProcessImage**: Runs the full workflow pipeline, collects errors from operations where `Result == false`, draws graphics overlays on the analysis image using `OpenCVCanvas`.
+- **HotReload**: Uses `Interlocked.Exchange` to pass the file path; `ProcessImage` checks for pending reload via `_workflow.HotReloadParameters()`.
+- **Cleanup**: Disposes the file watcher and debounce timer.
+- **Settings**: `HawkeyeRecognitionSettings` adds `AlwaysError` (bool) for testing.
+
+### CandyboxImageProcessingControl
+
+**Project:** `Plugins/CandyboxPlugin`
+
+Histogram-based image processing for candy box quality control.
+
+- **GetRecipesData**: Loads `.json` files from `Data/Recipes/`, finds matching sample `.bmp` images for previews.
+- **Initialize**: Creates a `HistoRecipe`, applies camera width settings to HawkEye hardware.
+- **WarmUp**: No-op.
+- **ProcessImage**: Acquires image, processes via `HistoRecipe`, collects errors from `ErrorPoints` and `ErrorReason`.
+- **Settings**: `CandyboxRecognitionControlSettings` adds `AlwaysError` with `[SettingDescription]` attribute.
+- **DI**: Plugin replaces the existing `BaseRecognitionControlSettings` binding before adding its own.
+
+### TestImageProcessingControl
+
+**Project:** `Plugins/TestPlugin`
+
+Minimal stub for testing and development.
+
+- **GetRecipesData**: Returns two hardcoded recipes ("Hello", "World") with red 100x100 Mat images.
+- **Initialize / WarmUp**: No-op.
+- **ProcessImage**: Returns a fixed red image with `"TestError"`.
+- **Settings**: `TestRecognitionControlSettings` with no extra properties.
+
+## Checklist for New Implementations
+
+1. Create a settings class extending `BaseRecognitionControlSettings`.
+2. Override `RegisterSettings` and call `base.RegisterSettings(settings)` first.
+3. Create a recognition control class extending `BaseRecognitionControl`.
+4. Implement `GetRecipesData()`, `Initialize()`, `WarmUp()`, `ProcessImage()`.
+5. Override `Cleanup()` to release resources.
+6. (Optional) Override `HotReload()` and set up file watching with `RaiseRecipeFileChanged()`.
+7. Register both classes in the DI container (bind settings to 3 types, bind control as singleton).
+8. Ensure `ProcessImage` respects `CancellationToken` and returns `null` when cancelled/exhausted.