HawkeyeVision/docs/RECOGNITION_CONTROL.md

# Recognition Control System

The recognition control system manages the image processing lifecycle: acquiring images, running them through a processing pipeline, and emitting results as events. Each camera gets its own `IRecognitionControl` instance, registered in the per-camera Ninject child kernel.

## Architecture Overview

```
IRecognitionControl (interface)
    ^
    |
BaseRecognitionControl (abstract base - lifecycle, threading, events)
    ^
    |--- HawkeyeRecognitionControl (workflow-based pipeline processing)
    |--- CandyboxImageProcessingControl (histogram-based processing)
    |--- TestImageProcessingControl (minimal test stub)
    |--- [Your custom implementation]
```

```
BaseRecognitionControlSettings (abstract base - shared config)
    ^
    |--- HawkeyeRecognitionSettings
    |--- CandyboxRecognitionControlSettings
    |--- TestRecognitionControlSettings
    |--- [Your custom settings]
```

## IRecognitionControl Interface

Defined in `VisionBuilder.UI.Common/Processing/IRecognitionControl.cs`.

```csharp
public interface IRecognitionControl
{
    bool IsRunning { get; }
    List<RecipeData> GetRecipesData();
    void SetRecipe(RecipeData recipe);
    void Start();
    void Stop();
    void Pause();
    void Resume();
    event Action<ImageProcessedEvent> ImageProcessed;
    event Action<SessionStartedEvent> SessionStarted;
    event Action<SessionEndedEvent> SessionEnded;
    event Action<ErrorsInSequenceEvent> ErrorsInSequenceAlarm;
    void HotReload();
    event Action RecipeFileChanged;
}
```

| Member | Description |
|---|---|
| `IsRunning` | Whether the processing loop is active. |
| `GetRecipesData()` | Returns available recipes (name + optional preview image). |
| `SetRecipe(recipe)` | Selects a recipe. Throws if `IsRunning` is true. |
| `Start()` | Initializes the recipe, warms up, then starts the processing loop on a background task. |
| `Stop()` | Cancels the processing loop, waits for it to finish, runs cleanup. |
| `Pause()` / `Resume()` | Temporarily suspends/resumes image processing without stopping the session. |
| `HotReload()` | Reloads recipe parameters without stopping the session. |
| `ImageProcessed` | Fired after every processed image with full result data. |
| `SessionStarted` / `SessionEnded` | Fired when a session starts/stops. |
| `ErrorsInSequenceAlarm` | Fired when consecutive errors reach the configured threshold. |
| `RecipeFileChanged` | Fired when the recipe file on disk changes (for hot-reload UI). |

## BaseRecognitionControl

Defined in `VisionBuilder.UI.Common/Processing/BaseRecognitionControl.cs`. This abstract class implements all of `IRecognitionControl` and provides the processing loop, threading, pause logic, error-in-sequence tracking, and event plumbing. Subclasses only need to implement four methods.

### Abstract Methods to Implement

```csharp
// Load/prepare resources for the given recipe
protected abstract void Initialize(RecipeData currentRecipe);

// Run a throwaway processing pass to warm up caches, JIT, models, etc.
protected abstract void WarmUp();

// Acquire and process one image. Return null to stop the loop.
protected abstract (
    Mat originalImage,
    Mat analysisImage,
    TimeSpan processingTime,
    TimeSpan acquisitionTime,
    string[] errorNames
)? ProcessImage(CancellationToken token);

// Return the list of available recipes
public abstract List<RecipeData> GetRecipesData();
```

### Virtual Methods (Optional Overrides)

```csharp
// Release resources when the session stops (called from Stop())
protected virtual void Cleanup() { }

// Reload recipe parameters without restarting the session
public virtual void HotReload() { }
```

### Protected Helper

```csharp
// Call this to raise the RecipeFileChanged event (e.g. from a file watcher)
protected void RaiseRecipeFileChanged();
```

### Processing Loop Behavior

The `Start()` method:
1. Validates that a recipe is set and the control is not already running.
2. Shows a loading indicator via `ILoadingService`.
3. Calls `Initialize(currentRecipe)`.
4. Calls `WarmUp()` (errors are logged, not thrown).
5. Launches `Loop()` on `Task.Run` with a `CancellationToken`.
6. Fires `SessionStarted`.
7. Hides the loading indicator.

The `Loop()` runs continuously until cancellation:
1. If paused, sleeps 100ms and continues.
2. Calls `ProcessImage(token)`. If it returns `null`, the loop exits.
3. If processing time is below `MinimumProcessingTime`, sleeps the difference.
4. Tracks consecutive errors. If they reach `ErrorsInSequenceAlarm`, fires the alarm event.
5. Builds an `ImageProcessedEvent` and fires it.

The `Stop()` method:
1. Cancels the token, waits for the loop task to finish.
2. Calls `Cleanup()`.
3. Fires `SessionEnded`.

### Thread Safety

All public lifecycle methods (`Start`, `Stop`, `Pause`, `Resume`, `SetRecipe`) are protected by a shared `lock`. The processing loop runs on a background `Task`. `ProcessImage` receives a `CancellationToken` and should respect it for timely shutdown.

## BaseRecognitionControlSettings

Defined in `VisionBuilder.UI.Common/Processing/BaseRecognitionControlSettings.cs`. Provides the shared configuration for all recognition controls.

```csharp
public abstract class BaseRecognitionControlSettings(string cameraName) : ISettings
{
    public string CameraName { get; }                    // Injected camera identifier
    public int MinimumProcessingTime { get; set; }       // Floor for loop iteration (ms)
    public string RecipeNameFilter { get; set; } = "*";  // Glob filter for recipe discovery
    public int ErrorsInSequenceAlarm { get; set; } = 5;  // Consecutive errors before alarm
}
```

All properties are registered in the `Inspectron.Settings` system under `{CameraName}/Recognition` by the `RegisterSettings` method. Override it to add custom settings (always call `base.RegisterSettings` first).

## Events

All event classes are in `VisionBuilder.UI.Common/Commands/`.

### ImageProcessedEvent

Fired after every image. Contains everything needed to update the UI and record statistics.

```csharp
public class ImageProcessedEvent : IEvent
{
    public DateTime SessionStart { get; set; }
    public string ImageSource { get; set; }       // Camera name
    public string RecipeName { get; set; }
    public Mat ImageOriginal { get; set; }        // Raw camera image
    public Mat ImageAnalysis { get; set; }        // Annotated/processed image
    public bool HasError { get; set; }
    public List<string> ErrorNames { get; set; }  // Names of failed operations
    public TimeSpan AnalysisTime { get; set; }    // Processing minus acquisition
    public TimeSpan AcquisitionTime { get; set; } // Camera capture time
    public TimeSpan SleepTime { get; set; }       // MinimumProcessingTime padding
}
```

### SessionStartedEvent / SessionEndedEvent

```csharp
public class SessionStartedEvent : IEvent
{
    public string RecipeName { get; set; }
    public DateTime SessionStarted { get; set; }
}

public class SessionEndedEvent : IEvent
{
    public DateTime SessionEnded { get; set; }
}
```

### ErrorsInSequenceEvent

```csharp
public class ErrorsInSequenceEvent
{
    public int Errors { get; set; }  // Current consecutive error count
}
```

## RecipeData

Defined in `VisionBuilder.UI.Common/ViewModel/Classes/RecipeData.cs`.

```csharp
public class RecipeData
{
    public Mat? Image { get; set; }        // Optional preview thumbnail
    public string RecipeName { get; set; } // Recipe identifier (typically filename without extension)
}
```

## ILoadingService

Defined in `VisionBuilder.UI.Common/Processing/ILoadingService.cs`. Called by `BaseRecognitionControl` during `Start()` and `Stop()` to show/hide a loading overlay in the UI.

```csharp
public interface ILoadingService
{
    void StartLoading(string title);
    void StopLoading(string title);
}
```

The loading service is injected into `BaseRecognitionControl` via the constructor and exposed as the `LoadingService` protected property, so subclasses can use it in `GetRecipesData()` or elsewhere.

## How to Implement a Custom Recognition Control

### Step 1: Create Settings

Extend `BaseRecognitionControlSettings` with any additional configuration your processing needs.

```csharp
using Inspectron.Settings;
using VisionBuilder.UI.Common.Processing;

public class MyRecognitionSettings(string cameraName) : BaseRecognitionControlSettings(cameraName)
{
    public double Threshold { get; set; } = 0.5;

    public override void RegisterSettings(InspectronSettings settings)
    {
        base.RegisterSettings(settings);
        settings.RegisterSimple(this, () => Threshold,
            CameraName + "/Recognition", nameof(Threshold));
    }
}
```

The `RegisterSettings` call makes the property editable in the application's settings UI. Always call `base.RegisterSettings(settings)` first to register the base properties (`MinimumProcessingTime`, `RecipeNameFilter`, `ErrorsInSequenceAlarm`).

### Step 2: Create the Recognition Control

Extend `BaseRecognitionControl` and implement the four abstract members.

```csharp
using System.Diagnostics;
using OpenCvSharp;
using VisionBuilder.UI.Common.Processing;
using VisionBuilder.UI.Common.ViewModel.Classes;

public class MyRecognitionControl : BaseRecognitionControl
{
    private readonly MyRecognitionSettings _settings;
    private readonly IImageSource _imageSource;
    private MyProcessor _processor;  // your processing logic

    public MyRecognitionControl(
        MyRecognitionSettings settings,
        IImageSource imageSource,
        ILoadingService loadingService)
        : base(settings, loadingService)
    {
        _settings = settings;
        _imageSource = imageSource;
    }

    public override List<RecipeData> GetRecipesData()
    {
        // Return available recipes. Use _settings.RecipeNameFilter if applicable.
        var recipes = new List<RecipeData>();
        foreach (var file in Directory.GetFiles(@"..\Data\Recipes", "*.myrecipe"))
        {
            recipes.Add(new RecipeData
            {
                RecipeName = Path.GetFileNameWithoutExtension(file),
                Image = null  // or load a preview thumbnail
            });
        }
        return recipes;
    }

    protected override void Initialize(RecipeData currentRecipe)
    {
        // Load the recipe and prepare your processing pipeline.
        // Called once when Start() is invoked.
        var path = Path.Combine(@"..\Data\Recipes", currentRecipe.RecipeName + ".myrecipe");
        _processor = MyProcessor.Load(path, _settings.Threshold);
    }

    protected override void WarmUp()
    {
        // Optional: run a throwaway pass to warm up JIT, GPU, ONNX models, etc.
        // Errors here are caught and logged, they won't prevent startup.
        var dummy = new Mat(100, 100, MatType.CV_8UC3, Scalar.All(0));
        _processor.Process(dummy);
        dummy.Dispose();
    }

    protected override (Mat originalImage, Mat analysisImage,
        TimeSpan processingTime, TimeSpan acquisitionTime,
        string[] errorNames)?
        ProcessImage(CancellationToken token)
    {
        // 1. Acquire image
        var swTotal = Stopwatch.StartNew();
        var swAcquire = Stopwatch.StartNew();
        var image = _imageSource.GetImage(token).Result;
        swAcquire.Stop();

        if (image == null)
            return null;  // null signals "stop the loop"

        // 2. Process
        var result = _processor.Process(image);
        swTotal.Stop();

        // 3. Build annotated image
        var annotated = image.Clone();
        // ... draw overlays on annotated ...

        // 4. Collect error names (empty array = pass)
        string[] errors = result.Defects
            .Select(d => d.Name)
            .ToArray();

        return (image, annotated, swTotal.Elapsed, swAcquire.Elapsed, errors);
    }

    protected override void Cleanup()
    {
        // Release resources when the session stops.
        _processor?.Dispose();
        _processor = null;
    }
}
```

### Key implementation notes for ProcessImage

- **Return `null`** to signal the loop should exit (e.g. when the image source is exhausted or cancelled).
- **Respect the `CancellationToken`** — pass it to blocking calls like `GetImage()`. Catch `OperationCanceledException` if you need cleanup before returning null.
- **`errorNames`** drives the pass/fail logic. An empty array means the image passed. Each string becomes a named error in statistics and UI.
- **`originalImage`** is the raw camera image; **`analysisImage`** is the annotated version shown in the preview UI.
- **`processingTime`** is the total wall-clock time including acquisition; **`acquisitionTime`** is just the camera capture. The base class computes `AnalysisTime = processingTime - acquisitionTime` for the event.

### Step 3: Register via DI

Register your settings and control in the per-camera Ninject child kernel. This is typically done in a plugin's `RegisterCameraModules` or in an extension method.

**Option A: In a plugin**

```csharp
public class MyPlugin : IPlugin
{
    public void RegisterGlobalModules(IKernel kernel) { }

    public void RegisterCameraModules(IKernel kernel, string cameraName)
    {
        kernel.Bind<MyRecognitionSettings, BaseRecognitionControlSettings, ISettings>()
            .ToConstant(new MyRecognitionSettings(cameraName));

        kernel.Rebind<IRecognitionControl>()
            .To<MyRecognitionControl>()
            .InSingletonScope();
    }
}
```

**Option B: As an extension method (like HawkeyeRecipe)**

```csharp
public static class ModuleExtensions
{
    public static IChildKernel UseMyRecipes(this IChildKernel self, string cameraName)
    {
        self.Bind<MyRecognitionSettings, BaseRecognitionControlSettings, ISettings>()
            .ToConstant(new MyRecognitionSettings(cameraName));
        self.Bind<IRecognitionControl>()
            .To<MyRecognitionControl>()
            .InSingletonScope();
        return self;
    }
}
```

**Important DI details:**
- Bind your settings class to **three types**: itself, `BaseRecognitionControlSettings`, and `ISettings`. The `ISettings` binding makes `RegisterSettings` get called by the settings system. The `BaseRecognitionControlSettings` binding lets the base class constructor receive it.
- Use `Rebind<IRecognitionControl>` (not `Bind`) if replacing an existing binding from another recipe provider.
- Always use `InSingletonScope()` — there should be exactly one recognition control per camera.

### Step 4: Hot Reload (Optional)

To support live recipe parameter updates without stopping:

1. Set up a `FileSystemWatcher` in `Initialize()` to monitor the recipe file.
2. When the file changes, call `RaiseRecipeFileChanged()` to notify the UI.
3. Override `HotReload()` to apply the new parameters on the next loop iteration.

Example from `HawkeyeRecognitionControl`:

```csharp
private FileSystemWatcher? _fileWatcher;
private System.Timers.Timer? _debounceTimer;
private string? _pendingHotReloadFile;
private string? _currentRecipeFilePath;

protected override void Initialize(RecipeData currentRecipe)
{
    _currentRecipeFilePath = Path.GetFullPath(/* recipe path */);
    _workflow = LoadRecipe(_currentRecipeFilePath);
    StartFileWatcher(_currentRecipeFilePath);
}

private void StartFileWatcher(string filePath)
{
    var directory = Path.GetDirectoryName(filePath)!;
    var fileName = Path.GetFileName(filePath);

    _debounceTimer = new System.Timers.Timer(500) { AutoReset = false };
    _debounceTimer.Elapsed += (_, _) => RaiseRecipeFileChanged();

    _fileWatcher = new FileSystemWatcher(directory, fileName)
    {
        NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.Size,
        EnableRaisingEvents = true
    };
    _fileWatcher.Changed += (_, _) =>
    {
        _debounceTimer.Stop();
        _debounceTimer.Start();  // debounce to 500ms
    };
}

public override void HotReload()
{
    // Store path; ProcessImage picks it up on the next iteration
    Interlocked.Exchange(ref _pendingHotReloadFile, _currentRecipeFilePath);
}

protected override void Cleanup()
{
    _fileWatcher?.Dispose();
    _debounceTimer?.Dispose();
}
```

The UI flow: `RecipeFileChanged` event -> `SingleCameraVM` sets `IsHotReloadAvailable = true` -> user clicks hot-reload button -> calls `HotReload()` -> next `ProcessImage` iteration applies new parameters.

## How the UI Consumes IRecognitionControl

`SingleCameraVM` (in `VisionBuilder.UI.Common/ViewModel/SingleCameraVM.cs`) is the primary consumer:

**Commands:**
```csharp
// Start/Stop run on background tasks to avoid blocking the UI
await Task.Run(() => _recognitionControl.Start());
await Task.Run(() => _recognitionControl.Stop());

// Pause/Resume are synchronous (just flip a flag)
_recognitionControl.Pause();
_recognitionControl.Resume();

// Recipe selection
_recognitionControl.SetRecipe(selectedRecipe);

// Hot reload
_recognitionControl.HotReload();
```

**Event subscriptions (in constructor):**
```csharp
recognitionControl.ImageProcessed += Handle;         // -> PreviewVm, StatisticsVm, ErrorsVm
recognitionControl.SessionStarted += StatisticsVm.Handle;
recognitionControl.SessionStarted += ErrorsVm.Handle;
recognitionControl.RecipeFileChanged += () =>
    SynchronizationContext?.Post(_ => IsHotReloadAvailable = true, null);
```

**Other modules subscribing to events:**
- `VisionBuilderStatistics` — writes CSV logs per session, tracks pass/fail counts.
- `VisionBuilderRingbuffer` — saves images to ringbuffers (separate good/bad).
- `IOCommanderCameraModule` — emits GPIO signals on session start/end, image result, and error alarms.
- Plugin modules (e.g. `PralinenModule`) — custom PLC integration based on results.

## Existing Implementations Reference

### HawkeyeRecognitionControl

**Project:** `VisionBuilder.UI.Recipes.HawkeyeRecipe`

The primary implementation for workflow-based image processing.

- **GetRecipesData**: Loads `.jhrcp` files from `Data/Recipes/`, deserializes each to extract recipe name and preview image.
- **Initialize**: Loads the workflow JSON file, starts a file watcher for hot-reload.
- **WarmUp**: Executes the workflow once with an `EmptyImageSource`.
- **ProcessImage**: Runs the full workflow pipeline, collects errors from operations where `Result == false`, draws graphics overlays on the analysis image using `OpenCVCanvas`.
- **HotReload**: Uses `Interlocked.Exchange` to pass the file path; `ProcessImage` checks for pending reload via `_workflow.HotReloadParameters()`.
- **Cleanup**: Disposes the file watcher and debounce timer.
- **Settings**: `HawkeyeRecognitionSettings` adds `AlwaysError` (bool) for testing.

### CandyboxImageProcessingControl

**Project:** `Plugins/CandyboxPlugin`

Histogram-based image processing for candy box quality control.

- **GetRecipesData**: Loads `.json` files from `Data/Recipes/`, finds matching sample `.bmp` images for previews.
- **Initialize**: Creates a `HistoRecipe`, applies camera width settings to HawkEye hardware.
- **WarmUp**: No-op.
- **ProcessImage**: Acquires image, processes via `HistoRecipe`, collects errors from `ErrorPoints` and `ErrorReason`.
- **Settings**: `CandyboxRecognitionControlSettings` adds `AlwaysError` with `[SettingDescription]` attribute.
- **DI**: Plugin replaces the existing `BaseRecognitionControlSettings` binding before adding its own.

### TestImageProcessingControl

**Project:** `Plugins/TestPlugin`

Minimal stub for testing and development.

- **GetRecipesData**: Returns two hardcoded recipes ("Hello", "World") with red 100x100 Mat images.
- **Initialize / WarmUp**: No-op.
- **ProcessImage**: Returns a fixed red image with `"TestError"`.
- **Settings**: `TestRecognitionControlSettings` with no extra properties.

## Checklist for New Implementations

1. Create a settings class extending `BaseRecognitionControlSettings`.
2. Override `RegisterSettings` and call `base.RegisterSettings(settings)` first.
3. Create a recognition control class extending `BaseRecognitionControl`.
4. Implement `GetRecipesData()`, `Initialize()`, `WarmUp()`, `ProcessImage()`.
5. Override `Cleanup()` to release resources.
6. (Optional) Override `HotReload()` and set up file watching with `RaiseRecipeFileChanged()`.
7. Register both classes in the DI container (bind settings to 3 types, bind control as singleton).
8. Ensure `ProcessImage` respects `CancellationToken` and returns `null` when cancelled/exhausted.