HawkeyeVision/docs/PLUGIN_SYSTEM.md

# VisionBuilder Plugin System

This document provides in-depth documentation of the VisionBuilder plugin system architecture and step-by-step instructions for implementing new plugins.

## Table of Contents

- [Overview](#overview)
- [Architecture](#architecture)
  - [Plugin Lifecycle](#plugin-lifecycle)
  - [Dependency Injection Scoping](#dependency-injection-scoping)
  - [Module Initialization Order](#module-initialization-order)
- [Core Interfaces](#core-interfaces)
  - [IPlugin](#iplugin)
  - [IVisionBuilderModule](#ivisionbuildermodule)
  - [ISettings](#isettings)
  - [IRecognitionControl](#irecognitioncontrol)
  - [BaseRecognitionControl](#baserecognitioncontrol)
  - [IImageSource](#iimagesource)
  - [ILoadingService](#iloadingservice)
  - [IRecipeCreationTool](#irecipecreationtool)
  - [ILearningTool](#ilearningtool)
- [Plugin Discovery and Loading](#plugin-discovery-and-loading)
  - [Directory Structure](#directory-structure)
  - [enabled_plugins.txt](#enabled_pluginstxt)
  - [PluginLoader](#pluginloader)
  - [PluginLoadContext (Assembly Isolation)](#pluginloadcontext-assembly-isolation)
  - [Plugin Profiles](#plugin-profiles)
- [Event System](#event-system)
- [Extension Methods Reference](#extension-methods-reference)
- [How to Implement a Plugin](#how-to-implement-a-plugin)
  - [Step 1: Create the Project](#step-1-create-the-project)
  - [Step 2: Implement IPlugin](#step-2-implement-iplugin)
  - [Step 3: Create Modules](#step-3-create-modules)
  - [Step 4: Add Settings](#step-4-add-settings)
  - [Step 5: Deploy and Enable](#step-5-deploy-and-enable)
- [Common Plugin Patterns](#common-plugin-patterns)
  - [Pattern: Add a Background Module](#pattern-add-a-background-module)
  - [Pattern: Replace the Recognition Control](#pattern-replace-the-recognition-control)
  - [Pattern: React to Image Processing Events](#pattern-react-to-image-processing-events)
  - [Pattern: External Hardware Integration (PLC/Serial)](#pattern-external-hardware-integration-plcserial)
  - [Pattern: Register Custom Type Converters](#pattern-register-custom-type-converters)
  - [Pattern: Override UI Tools](#pattern-override-ui-tools)
  - [Pattern: Add Dynamic UI Buttons](#pattern-add-dynamic-ui-buttons)
- [Existing Plugin Examples](#existing-plugin-examples)
- [Key Source File Locations](#key-source-file-locations)

---

## Overview

The plugin system allows extending VisionBuilder with external assemblies that are loaded at runtime. Plugins can:

- Register app-wide (global) services and modules
- Register per-camera services and modules
- Replace core services like `IRecognitionControl`, `ILearningTool`, `IRecipeCreationTool`
- Subscribe to image processing events and camera lifecycle events
- Add configurable settings that persist via `InspectronSettings`
- Register custom type converters for settings serialization
- Integrate with external hardware (PLCs, barcode readers, serial devices)

Plugins are loaded from the `../Data/Plugins/` directory and are enabled/disabled via a text configuration file.

## Architecture

### Plugin Lifecycle

```
Application Startup
    |
    v
1. Main kernel created (StandardKernel)
    |
    v
2. kernel.UsePlugins(profile)
    |--- PluginLoader registered as IVisionBuilderModule
    |--- PluginLoader.InitializeModule(profile) called
    |--- Scans ../Data/Plugins/ for enabled plugin folders
    |--- Loads assemblies, discovers IPlugin implementations
    |--- Stores plugins in PluginLoader.Plugins list
    |
    v
3. kernel.RegisterGlobalPlugins()
    |--- For each IPlugin: calls RegisterGlobalModules(mainKernel)
    |--- Plugins register global modules, type converters, rebind global tools
    |
    v
4. For each camera:
    |
    |--- Create ChildKernel(mainKernel)
    |--- Register camera-specific bindings (CameraSettings, etc.)
    |--- kernel.RegisterCameraPlugins(cameraName)
    |    |--- For each IPlugin: calls RegisterCameraModules(childKernel, cameraName)
    |    |--- Plugins register camera modules, settings, rebind per-camera services
    |
    |--- kernel.RegisterSettings()
    |    |--- Finds all ISettings in kernel, calls RegisterSettings(InspectronSettings)
    |
    |--- kernel.InitializeModules()
    |    |--- Finds all IVisionBuilderModule in kernel
    |    |--- Sorts by [ModulePriority] descending (higher = earlier)
    |    |--- Calls InitializeModule() on each, skipping already-initialized
    |
    v
5. Application running — modules active, events flowing
```

### Dependency Injection Scoping

The system uses two levels of Ninject kernels:

**Main Kernel (StandardKernel)** — Global scope:
- Shared across all cameras
- Holds `InspectronSettings`, `UIConfiguration`, `PluginLoader`
- Global modules registered here are singletons for the entire app
- `RegisterGlobalModules()` receives this kernel

**Child Kernels (ChildKernel)** — Per-camera scope:
- One child kernel per camera
- Inherits bindings from the main kernel (can resolve global services)
- Camera-specific bindings: `CameraSettings`, `SingleCameraVM`, `IRecognitionControl`, `IImageSource`
- `RegisterCameraModules()` receives the child kernel
- Services bound here are isolated per camera

```
MainKernel (StandardKernel)
  |--- InspectronSettings (singleton)
  |--- PluginLoader (singleton)
  |--- IRecipeCreationTool (singleton, rebindable)
  |--- Global IVisionBuilderModule instances
  |
  |--- ChildKernel (Camera "Left")
  |      |--- CameraSettings
  |      |--- SingleCameraVM (singleton)
  |      |--- IRecognitionControl (singleton)
  |      |--- IImageSource
  |      |--- Camera-specific IVisionBuilderModule instances
  |      |--- Camera-specific ISettings instances
  |
  |--- ChildKernel (Camera "Right")
         |--- (same structure, independent instances)
```

### Module Initialization Order

Modules are initialized via `kernel.InitializeModules()`, which:

1. Collects all `IVisionBuilderModule` instances from the kernel
2. Filters out already-initialized modules (tracked per kernel)
3. Sorts by `[ModulePriority(n)]` attribute in **descending** order (higher priority = initialized first)
4. Calls `InitializeModule()` on each in order

Default priority is `0` if no attribute is present. The `PluginLoader` itself uses priority `1000` to ensure it runs first.

```csharp
[ModulePriority(100)]  // Initialized before modules with lower priority
public class MyCriticalModule : IVisionBuilderModule { ... }

public class MyRegularModule : IVisionBuilderModule { ... }  // Priority 0 (default)
```

## Core Interfaces

### IPlugin

**Location:** `VisionBuilder.UI.Common/Plugins/IPlugin.cs`

The entry point for all plugins. Each plugin assembly must contain at least one class implementing this interface.

```csharp
public interface IPlugin
{
    void RegisterGlobalModules(IKernel kernel);
    void RegisterCameraModules(IKernel kernel, string cameraName);
}
```

- `RegisterGlobalModules(IKernel kernel)` — Called **once** at startup with the main kernel. Register app-wide services, type converters, and rebind global tool interfaces here.
- `RegisterCameraModules(IKernel kernel, string cameraName)` — Called **once per camera** with the camera's child kernel. Register camera-specific modules, settings, and service overrides here.

### IVisionBuilderModule

**Location:** `VisionBuilder.UI.Common/IVisionBuilderModule.cs`

Modules are lifecycle-managed components that perform deferred initialization after the DI container is fully configured.

```csharp
public interface IVisionBuilderModule
{
    void InitializeModule();
}
```

Register modules using the extension method:

```csharp
kernel.RegisterModule<MyModule>();
// Equivalent to:
// kernel.Bind<MyModule, IVisionBuilderModule>().To<MyModule>().InSingletonScope();
```

Modules receive their dependencies via **constructor injection** from Ninject. `InitializeModule()` is the place to start background tasks, subscribe to events, open connections, etc.

### ISettings

**Location:** `VisionBuilder.UI.Common/ISettings.cs`

Settings classes register configurable properties with the `InspectronSettings` system.

```csharp
public interface ISettings
{
    void RegisterSettings(InspectronSettings settings);
}
```

Settings are bound as constants (since they hold state) and must also be bound to `ISettings` so the framework discovers them:

```csharp
kernel.Bind<MyPluginSettings, ISettings>().ToConstant(new MyPluginSettings(cameraName));
```

Inside `RegisterSettings`, use `InspectronSettings.RegisterSimple()` to expose properties:

```csharp
public class MyPluginSettings : ISettings
{
    public string CameraName { get; }
    public int PollingInterval { get; set; } = 1000;
    public string ServerAddress { get; set; } = "127.0.0.1";

    public MyPluginSettings(string cameraName) => CameraName = cameraName;

    public void RegisterSettings(InspectronSettings settings)
    {
        settings.RegisterSimple(this, () => PollingInterval, CameraName + "/MyPlugin", "Polling interval (ms)");
        settings.RegisterSimple(this, () => ServerAddress, CameraName + "/MyPlugin", nameof(ServerAddress));
    }
}
```

### IRecognitionControl

**Location:** `VisionBuilder.UI.Common/Processing/IRecognitionControl.cs`

The core interface for image processing control. One instance per camera.

```csharp
public interface IRecognitionControl
{
    bool IsRunning { get; }
    List<RecipeData> GetRecipesData();
    void SetRecipe(RecipeData recipe);
    void Start();
    void Stop();
    void Pause();
    void Resume();
    event Action<ImageProcessedEvent> ImageProcessed;
    event Action<SessionStartedEvent> SessionStarted;
    event Action<SessionEndedEvent> SessionEnded;
    event Action<ErrorsInSequenceEvent> ErrorsInSequenceAlarm;
    void HotReload();
    event Action RecipeFileChanged;
}
```

Events fired during the processing loop:
- `ImageProcessed` — After each image is processed (contains result, errors, timing)
- `SessionStarted` — When `Start()` is called successfully
- `SessionEnded` — When `Stop()` completes
- `ErrorsInSequenceAlarm` — When consecutive errors exceed the configured threshold

### BaseRecognitionControl

**Location:** `VisionBuilder.UI.Common/Processing/BaseRecognitionControl.cs`

Abstract base class that implements the recognition loop. Extend this when your plugin needs a custom image processing pipeline.

```csharp
public abstract class BaseRecognitionControl : IRecognitionControl
{
    // Constructor — requires settings and loading service
    public BaseRecognitionControl(
        BaseRecognitionControlSettings settings,
        ILoadingService loadingService);

    // Must implement:
    public abstract List<RecipeData> GetRecipesData();
    protected abstract void Initialize(RecipeData currentRecipe);
    protected abstract void WarmUp();
    protected abstract (Mat originalImage, Mat analysisImage,
        TimeSpan processingTime, TimeSpan acquisitionTime,
        string[] errorNames)? ProcessImage(CancellationToken token);

    // Optional override:
    protected virtual void Cleanup() { }
    public virtual void HotReload() { }
}
```

The base class handles:
- Thread-safe start/stop/pause/resume lifecycle
- The main processing loop (calls `ProcessImage` repeatedly)
- Minimum processing time enforcement (configurable via settings)
- Consecutive error tracking and alarm firing
- Event dispatch (`ImageProcessed`, `SessionStarted`, `SessionEnded`, `ErrorsInSequenceAlarm`)

**RecipeData:**

```csharp
public class RecipeData
{
    public Mat? Image { get; set; }       // Thumbnail image for UI
    public string RecipeName { get; set; } // Recipe identifier
}
```

**BaseRecognitionControlSettings:**

```csharp
public abstract class BaseRecognitionControlSettings(string cameraName) : ISettings
{
    public string CameraName { get; }
    public int MinimumProcessingTime { get; set; }       // ms, enforced between frames
    public string RecipeNameFilter { get; set; } = "*";  // Glob filter for recipe listing
    public int ErrorsInSequenceAlarm { get; set; } = 5;  // Threshold for alarm event
}
```

### IImageSource

**Location:** `VisionBuilder.UI.Common/Processing/IImageSource.cs`

Provides images to the recognition control.

```csharp
public interface IImageSource
{
    Task<Mat> GetImage(CancellationToken token);
}
```

### ILoadingService

**Location:** `VisionBuilder.UI.Common/Processing/ILoadingService.cs`

Shows/hides loading indicators in the UI during long operations.

```csharp
public interface ILoadingService
{
    void StartLoading(string title);
    void StopLoading(string title);
}
```

### IRecipeCreationTool

**Location:** `VisionBuilder.UI.Common/ViewModel/Interfaces/UI/IRecipeCreationTool.cs`

UI tool for creating new recipes. Default implementation (`NoRecipeCreation`) is disabled. Plugins can `Rebind` this to provide custom recipe creation.

```csharp
public interface IRecipeCreationTool
{
    bool Enabled { get; set; }
    bool CreateRecipe(out string recipeName);
}
```

### ILearningTool

**Location:** `VisionBuilder.UI.Common/ViewModel/Interfaces/UI/ILearningTool.cs`

UI tool for learning/training recipes from images. Default implementation (`NoLearning`) is disabled.

```csharp
public interface ILearningTool
{
    bool IsLearningEnabled(string recipeName);
    void Learn(string recipeName, Mat image);
}
```

## Plugin Discovery and Loading

### Directory Structure

Plugins are deployed to the `../Data/Plugins/` directory (relative to the application executable). Each plugin resides in its own subfolder:

```
Data/
  Plugins/
    enabled_plugins.txt          # Lists enabled plugins (one per line)
    my_custom_plugins.txt        # Alternative profile (optional)
    B24SiemensPlugin/
      B24SiemensPlugin.dll       # Plugin assembly (name must match folder)
      (dependency DLLs)
    CandyboxPlugin/
      CandyboxPlugin.dll
      (dependency DLLs)
    MyNewPlugin/
      MyNewPlugin.dll            # Your plugin
      SomeLibrary.dll            # Any additional dependencies
```

**Critical:** The folder name and the main DLL name (without extension) must match exactly. The loader looks for `<FolderName>/<FolderName>.dll`.

### enabled_plugins.txt

A simple text file listing enabled plugin folder names, one per line:

```
B24SiemensPlugin
CandyboxPlugin
MyNewPlugin
```

Plugins not listed here are skipped during loading. If the file doesn't exist, it's created empty (no plugins enabled).

### PluginLoader

**Location:** `VisionBuilder.UI.Common/Plugins/PluginLoader.cs`

The `PluginLoader` is an `IVisionBuilderModule` with `[ModulePriority(1000)]` (highest built-in priority). It:

1. Reads the enabled plugins list from `../Data/Plugins/<profile>.txt`
2. Iterates over subdirectories in `../Data/Plugins/`
3. Skips folders not in the enabled list
4. Loads the main assembly from each enabled folder using `PluginLoadContext`
5. Scans loaded assembly types for `IPlugin` implementations
6. Creates instances via `Activator.CreateInstance()` and stores them in `Plugins` list

### PluginLoadContext (Assembly Isolation)

**Location:** `VisionBuilder.UI.Common/Plugins/PluginLoadContext.cs`

Each plugin is loaded in its own `AssemblyLoadContext` to isolate dependencies:

- Uses `AssemblyDependencyResolver` to resolve managed assemblies from the plugin folder
- Supports native/unmanaged DLL loading for plugins with native dependencies
- Prevents version conflicts between plugins and the host application

**Important project reference configuration:** Plugin projects must reference `VisionBuilder.UI.Common` (and other host assemblies) with `<Private>false</Private>` and `<ExcludeAssets>runtime</ExcludeAssets>` to avoid duplicating host assemblies in the plugin output.

### Plugin Profiles

The application supports multiple plugin configurations via profiles. The `--profile` / `-p` command-line argument selects which `.txt` file to use:

```bash
# Uses ../Data/Plugins/enabled_plugins.txt (default)
VisionBuilder.exe

# Uses ../Data/Plugins/production.txt
VisionBuilder.exe --profile production

# Uses ../Data/Plugins/testing.txt
VisionBuilder.exe -p testing
```

## Event System

The recognition control fires events as `Action<T>` delegates. Modules subscribe in `InitializeModule()`:

### ImageProcessedEvent

Fired after each image is processed.

```csharp
public class ImageProcessedEvent : IEvent
{
    public DateTime SessionStart { get; set; }
    public string ImageSource { get; set; }        // Camera name
    public string RecipeName { get; set; }
    public Mat ImageOriginal { get; set; }         // Raw camera image
    public Mat ImageAnalysis { get; set; }         // Annotated result image
    public bool HasError { get; set; }
    public List<string> ErrorNames { get; set; }   // Detected defects/errors
    public TimeSpan AnalysisTime { get; set; }
    public TimeSpan AcquisitionTime { get; set; }
    public TimeSpan SleepTime { get; set; }
}
```

### SessionStartedEvent

Fired when recognition starts.

```csharp
public class SessionStartedEvent
{
    public string RecipeName { get; set; }
    public DateTime SessionStarted { get; set; }
}
```

### SessionEndedEvent

Fired when recognition stops.

```csharp
public class SessionEndedEvent
{
    public DateTime SessionEnded { get; set; }
}
```

### ErrorsInSequenceEvent

Fired when consecutive error count reaches the threshold.

```csharp
public class ErrorsInSequenceEvent
{
    public int Errors { get; set; }
}
```

## Extension Methods Reference

**Location:** `VisionBuilder.UI.Common/Extensions.cs`

| Method | Description |
|--------|-------------|
| `kernel.RegisterModule<T>()` | Binds `T` as both itself and `IVisionBuilderModule` in singleton scope |
| `kernel.UsePlugins(profile)` | Registers and initializes the `PluginLoader` with the given profile |
| `kernel.RegisterGlobalPlugins()` | Calls `RegisterGlobalModules()` on all loaded plugins |
| `kernel.RegisterCameraPlugins(cameraName)` | Calls `RegisterCameraModules()` on all loaded plugins (child kernel) |
| `kernel.RegisterSettings()` | Finds all `ISettings` instances and calls `RegisterSettings()` |
| `kernel.InitializeModules()` | Initializes all `IVisionBuilderModule` instances in priority order |

## How to Implement a Plugin

### Step 1: Create the Project

Create a new .NET 8.0 class library project. The critical settings are:

```xml
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    <!-- Required: enables dynamic loading as a plugin -->
    <EnableDynamicLoading>true</EnableDynamicLoading>

    <!-- Output directly to the Plugins directory for development -->
    <OutDir>..\Data\Plugins\MyPlugin</OutDir>
  </PropertyGroup>

  <ItemGroup>
    <!-- Reference the common library — Private=false prevents copying host DLLs -->
    <ProjectReference Include="..\VisionBuilder.UI.Common\VisionBuilder.UI.Common.csproj">
      <Private>false</Private>
      <ExcludeAssets>runtime</ExcludeAssets>
    </ProjectReference>

    <!-- Reference settings framework if needed -->
    <ProjectReference Include="..\framework\Inspectron.Settings\Inspectron.Settings.csproj">
      <Private>false</Private>
      <ExcludeAssets>runtime</ExcludeAssets>
    </ProjectReference>

    <!-- Add any other host references with Private=false -->
  </ItemGroup>

  <ItemGroup>
    <!-- Plugin-specific NuGet packages (these WILL be copied to output) -->
    <!-- <PackageReference Include="SomeLibrary" Version="1.0.0" /> -->
  </ItemGroup>
</Project>
```

**Key project settings explained:**

- `EnableDynamicLoading` — Tells the build system this assembly will be loaded dynamically. Ensures all plugin-specific dependencies are copied to the output directory.
- `Private=false` / `ExcludeAssets=runtime` on host references — Prevents the plugin from copying VisionBuilder.UI.Common.dll and other host assemblies into the plugin folder. The host application already has these loaded; duplicating them causes type identity conflicts.
- `OutDir` — Set to the Plugins directory for easy development. Adjust the relative path based on your project location.

### Step 2: Implement IPlugin

Create a `Plugin.cs` (the class name can be anything):

```csharp
using Ninject;
using VisionBuilder.UI.Common;
using VisionBuilder.UI.Common.Plugins;

namespace MyPlugin;

public class Plugin : IPlugin
{
    public void RegisterGlobalModules(IKernel kernel)
    {
        // Called once at startup with the main kernel.
        // Register app-wide services, type converters, or rebind global tools.

        // Example: Register a custom type converter for settings
        // TypeConverterRegistry.Register<List<MyMapping>>(new MyMappingConverter());

        // Example: Override the recipe creation tool globally
        // kernel.Rebind<IRecipeCreationTool>().To<MyRecipeCreationTool>().InSingletonScope();
    }

    public void RegisterCameraModules(IKernel kernel, string cameraName)
    {
        // Called once per camera with the camera's child kernel.
        // Register camera-specific modules and settings.

        // Register a module (will be initialized later via InitializeModules)
        kernel.RegisterModule<MyCameraModule>();

        // Register settings (bound as constant since they hold state)
        kernel.Bind<MyPluginSettings, ISettings>()
            .ToConstant(new MyPluginSettings(cameraName));
    }
}
```

### Step 3: Create Modules

Modules do the actual work. They receive dependencies via constructor injection and perform initialization in `InitializeModule()`.

```csharp
using Serilog;
using VisionBuilder.UI.Common;
using VisionBuilder.UI.Common.Processing;

namespace MyPlugin;

public class MyCameraModule : IVisionBuilderModule
{
    private readonly IRecognitionControl _recognitionControl;
    private readonly MyPluginSettings _settings;

    // Dependencies are injected by Ninject from the camera's child kernel
    public MyCameraModule(
        IRecognitionControl recognitionControl,
        MyPluginSettings settings)
    {
        _recognitionControl = recognitionControl;
        _settings = settings;
    }

    public void InitializeModule()
    {
        // Subscribe to events
        _recognitionControl.ImageProcessed += OnImageProcessed;
        _recognitionControl.SessionStarted += OnSessionStarted;
        _recognitionControl.SessionEnded += OnSessionEnded;

        Log.Information("MyPlugin initialized for camera {Camera}", _settings.CameraName);
    }

    private void OnImageProcessed(ImageProcessedEvent e)
    {
        if (e.HasError)
        {
            Log.Warning("Errors detected: {Errors}", string.Join(", ", e.ErrorNames));
            // React to errors — send signal, log, notify, etc.
        }
    }

    private void OnSessionStarted(SessionStartedEvent e)
    {
        Log.Information("Session started: {Recipe}", e.RecipeName);
    }

    private void OnSessionEnded(SessionEndedEvent e)
    {
        Log.Information("Session ended");
    }
}
```

**Available services for constructor injection in camera modules:**

| Service | Description |
|---------|-------------|
| `IRecognitionControl` | Image processing control for this camera |
| `SingleCameraVM` | ViewModel for this camera (UI state, commands, `DynamicButtons` for adding custom buttons) |
| `CameraSettings` | Camera configuration (source type, label) |
| `IImageSource` | Image provider for this camera |
| `InspectronSettings` | Global settings manager |
| `ILoadingService` | Loading dialog service |
| Any `ISettings` bound in the kernel | Plugin-specific settings |
| Any `IVisionBuilderModule` bound in the kernel | Other modules (be careful of initialization order) |

### Step 4: Add Settings

Create a settings class to expose configurable properties:

```csharp
using Inspectron.Settings;
using VisionBuilder.UI.Common;

namespace MyPlugin;

public class MyPluginSettings : ISettings
{
    public string CameraName { get; }

    // Configurable properties with defaults
    public string ServerAddress { get; set; } = "127.0.0.1";
    public int Port { get; set; } = 5000;
    public bool Enabled { get; set; } = true;

    public MyPluginSettings(string cameraName)
    {
        CameraName = cameraName;
    }

    public void RegisterSettings(InspectronSettings settings)
    {
        // Register each property under a category path
        // Format: settings.RegisterSimple(owner, propertyExpression, category, displayName)
        settings.RegisterSimple(this, () => ServerAddress, CameraName + "/MyPlugin", "Server address");
        settings.RegisterSimple(this, () => Port, CameraName + "/MyPlugin", "Server port");
        settings.RegisterSimple(this, () => Enabled, CameraName + "/MyPlugin", "Enable plugin");
    }
}
```

Settings are automatically persisted and restored by the `InspectronSettings` infrastructure. The category path (`CameraName + "/MyPlugin"`) determines how settings appear in the UI.

### Step 5: Deploy and Enable

1. **Build the plugin** — output goes to `Data/Plugins/MyPlugin/`
2. **Add to enabled_plugins.txt** — append `MyPlugin` to `../Data/Plugins/enabled_plugins.txt`
3. **Restart the application** — the plugin will be discovered and loaded

```
Data/
  Plugins/
    enabled_plugins.txt    # Add "MyPlugin" line
    MyPlugin/
      MyPlugin.dll         # Folder name must match DLL name
```

## Common Plugin Patterns

### Pattern: Add a Background Module

Register a module that runs background work (e.g., listening on a port, polling a device).

```csharp
// In Plugin.RegisterCameraModules:
kernel.RegisterModule<MyBackgroundModule>();

// Module implementation:
public class MyBackgroundModule : IVisionBuilderModule
{
    private readonly MyPluginSettings _settings;
    private TcpListener _listener;

    public MyBackgroundModule(MyPluginSettings settings)
    {
        _settings = settings;
    }

    public void InitializeModule()
    {
        _listener = new TcpListener(IPAddress.Any, _settings.Port);
        _listener.Start();
        Task.Run(() => AcceptClients());
    }

    private async Task AcceptClients() { /* ... */ }
}
```

### Pattern: Replace the Recognition Control

Provide a completely custom image processing pipeline by replacing `IRecognitionControl`.

```csharp
// In Plugin.RegisterCameraModules:

// 1. Remove existing settings binding (if replacing BaseRecognitionControlSettings)
var existingSettings = kernel.GetBindings(typeof(BaseRecognitionControlSettings)).First();
var toRemove = kernel.GetBindings(typeof(ISettings))
    .First(x => x.ProviderCallback.Target == existingSettings.ProviderCallback.Target);
kernel.RemoveBinding(toRemove);

// 2. Bind new settings
kernel.Bind<MyRecognitionSettings, BaseRecognitionControlSettings, ISettings>()
    .ToConstant(new MyRecognitionSettings(cameraName));

// 3. Rebind the recognition control
kernel.Rebind<IRecognitionControl>().To<MyRecognitionControl>().InSingletonScope();

// Settings class:
public class MyRecognitionSettings : BaseRecognitionControlSettings
{
    public MyRecognitionSettings(string cameraName) : base(cameraName) { }

    public string CustomProperty { get; set; } = "default";

    public override void RegisterSettings(InspectronSettings settings)
    {
        base.RegisterSettings(settings); // Register base properties
        settings.RegisterSimple(this, () => CustomProperty, CameraName + "/MyRecognition", nameof(CustomProperty));
    }
}

// Recognition control:
public class MyRecognitionControl : BaseRecognitionControl
{
    private readonly IImageSource _imageSource;

    public MyRecognitionControl(
        MyRecognitionSettings settings,
        ILoadingService loadingService,
        IImageSource imageSource) : base(settings, loadingService)
    {
        _imageSource = imageSource;
    }

    public override List<RecipeData> GetRecipesData()
    {
        // Return available recipes
        return new List<RecipeData>
        {
            new() { RecipeName = "Recipe1", Image = CreateThumbnail() }
        };
    }

    protected override void Initialize(RecipeData currentRecipe)
    {
        // Called when Start() is invoked — load recipe data, initialize models, etc.
    }

    protected override void WarmUp()
    {
        // Called once after Initialize, before the main loop.
        // Use for warm-up runs (e.g., first inference is slow for AI models).
    }

    protected override (Mat originalImage, Mat analysisImage,
        TimeSpan processingTime, TimeSpan acquisitionTime,
        string[] errorNames)? ProcessImage(CancellationToken token)
    {
        var sw = Stopwatch.StartNew();
        var image = _imageSource.GetImage(token).Result;
        var acquisitionTime = sw.Elapsed;

        // Process image...
        var errors = AnalyzeImage(image);
        sw.Stop();

        return (image, image, sw.Elapsed, acquisitionTime, errors);
        // Return null to stop the loop
    }

    protected override void Cleanup()
    {
        // Called after Stop() — release resources
    }
}
```

### Pattern: React to Image Processing Events

Subscribe to events from the recognition control to react to results.

```csharp
public class ResultReporterModule : IVisionBuilderModule
{
    private readonly IRecognitionControl _recognitionControl;

    public ResultReporterModule(IRecognitionControl recognitionControl)
    {
        _recognitionControl = recognitionControl;
    }

    public void InitializeModule()
    {
        _recognitionControl.ImageProcessed += OnImageProcessed;
        _recognitionControl.ErrorsInSequenceAlarm += OnErrorAlarm;
    }

    private void OnImageProcessed(ImageProcessedEvent e)
    {
        // e.HasError — true if any errors detected
        // e.ErrorNames — list of detected error/defect names
        // e.ImageOriginal — raw image (Mat)
        // e.ImageAnalysis — annotated image (Mat)
        // e.AnalysisTime — processing duration
        // e.RecipeName — active recipe
    }

    private void OnErrorAlarm(ErrorsInSequenceEvent e)
    {
        // e.Errors — number of consecutive errors
        // Trigger alarm, stop line, notify operator, etc.
    }
}
```

### Pattern: External Hardware Integration (PLC/Serial)

Integrate with PLC controllers or serial devices.

```csharp
public class PLCIntegrationModule : IVisionBuilderModule
{
    private readonly IRecognitionControl _recognitionControl;
    private readonly SingleCameraVM _cameraVm;
    private readonly MyPLCSettings _settings;

    public PLCIntegrationModule(
        IRecognitionControl recognitionControl,
        SingleCameraVM cameraVm,
        MyPLCSettings settings)
    {
        _recognitionControl = recognitionControl;
        _cameraVm = cameraVm;
        _settings = settings;
    }

    public void InitializeModule()
    {
        // Start PLC communication
        var server = new TcpListener(IPAddress.Any, _settings.Port);
        server.Start();

        // React to processing results
        _recognitionControl.ImageProcessed += e =>
        {
            SendResultToPLC(e.HasError ? "NOK" : "OK");
        };
    }

    // Use SingleCameraVM to control the camera programmatically
    private void SelectRecipe(string recipeName)
    {
        var vm = _cameraVm.GetRecipeSelectionVm();
        vm.SelectedRecipe = vm.Recipes.FirstOrDefault(r => r.RecipeName == recipeName);
        if (vm.SelectedRecipe != null)
        {
            // Must post to UI thread for SingleCameraVM operations
            _cameraVm.SynchronizationContext!.Post(_ =>
            {
                _cameraVm.ProcessRecipeSelectionVm(vm);
            }, null);
        }
    }
}
```

**Note:** When interacting with `SingleCameraVM` from a background thread, always use `SynchronizationContext.Post()` to marshal calls to the UI thread.

### Pattern: Register Custom Type Converters

If your settings contain complex types that need custom serialization:

```csharp
// In Plugin.RegisterGlobalModules:
public void RegisterGlobalModules(IKernel kernel)
{
    TypeConverterRegistry.Register<List<MyMapping>>(new MyMappingConverter());
}

// Converter implementation:
public class MyMappingConverter : ITypeConverter
{
    public object ConvertFrom(object value)
    {
        // Deserialize from string (e.g., JSON)
        return JsonSerializer.Deserialize<List<MyMapping>>((string)value);
    }

    public object ConvertTo(object value, Type destinationType)
    {
        // Serialize to string
        return JsonSerializer.Serialize((List<MyMapping>)value);
    }
}
```

### Pattern: Override UI Tools

Replace default (disabled) UI tools with functional implementations:

```csharp
public void RegisterGlobalModules(IKernel kernel)
{
    // Enable recipe creation
    kernel.Rebind<IRecipeCreationTool>().To<MyRecipeCreationTool>().InSingletonScope();
}

public void RegisterCameraModules(IKernel kernel, string cameraName)
{
    // Enable learning per camera
    kernel.Rebind<ILearningTool>().To<MyLearningTool>().InSingletonScope();
}
```

### Pattern: Add Dynamic UI Buttons

Add custom buttons to the camera control panel (right side) or the main window bottom bar at runtime.

**Camera-level buttons** (right-side panel per camera):

```csharp
public class MyCameraModule : IVisionBuilderModule
{
    private readonly SingleCameraVM _cameraVm;

    public MyCameraModule(SingleCameraVM cameraVm)
    {
        _cameraVm = cameraVm;
    }

    public void InitializeModule()
    {
        // Add a click button
        _cameraVm.AddButton(new ButtonDefinition(
            "myPlugin.resetCounters", "Reset Counters", () =>
            {
                // Handle click
            }));

        // Add a toggle button
        _cameraVm.AddButton(new ButtonDefinition(
            "myPlugin.autoMode", "Auto Mode", (isToggled) =>
            {
                // Handle toggle state change
            }, initialState: false));

        // Control visibility later
        _cameraVm.SetButtonVisibility("myPlugin.resetCounters", false);

        // Remove a button
        _cameraVm.RemoveButton("myPlugin.autoMode");
    }
}
```

**App-level buttons** (bottom bar):

```csharp
public void RegisterGlobalModules(IKernel kernel)
{
    var mainVm = kernel.Get<MainWindowVM>();
    mainVm.AddButton(new ButtonDefinition(
        "myPlugin.dashboard", "Dashboard", () =>
        {
            // Open dashboard window
        }));
}
```

**ButtonDefinition properties:**

| Property | Type | Description |
|----------|------|-------------|
| `Key` | `string` | Unique identifier for the button (use plugin prefix, e.g., `"myPlugin.action"`) |
| `Title` | `string` | Display text on the button |
| `Type` | `ButtonType` | `Click` or `Toggle` |
| `IsVisible` | `bool` | Controls button visibility (default: `true`). Observable — UI updates automatically |
| `IsToggled` | `bool` | Current toggle state (only for `Toggle` buttons). Observable |

**Thread safety:** When calling `AddButton`, `RemoveButton`, or `SetButtonVisibility` from a background thread, marshal to the UI thread via `SynchronizationContext.Post`:

```csharp
_cameraVm.SynchronizationContext!.Post(_ =>
{
    _cameraVm.AddButton(new ButtonDefinition("key", "Title", () => { }));
}, null);
```

## Existing Plugin Examples

| Plugin | Purpose | Key Patterns Used |
|--------|---------|-------------------|
| **TestPlugin** | Minimal reference implementation | Module registration, custom recognition control |
| **B24SiemensPlugin** | Siemens PLC integration for recipe selection | Type converter, camera module, settings, PLC communication |
| **CandyboxPlugin** | Full-featured candy box inspection | Replace recognition control, barcode reader, recipe creation tool, learning tool |
| **PralinenPLC** | Send pass/fail results to PLC via TCP | Event subscription, network communication |
| **PackstrasseBarcodeReader** | Serial barcode reader for recipe selection | Settings, serial port module, barcode-to-recipe mapping |

## Key Source File Locations

| Component | Path |
|-----------|------|
| `IPlugin` interface | `VisionBuilder.UI.Common/Plugins/IPlugin.cs` |
| `IVisionBuilderModule` interface | `VisionBuilder.UI.Common/IVisionBuilderModule.cs` |
| `ISettings` interface | `VisionBuilder.UI.Common/ISettings.cs` |
| `PluginLoader` | `VisionBuilder.UI.Common/Plugins/PluginLoader.cs` |
| `PluginLoadContext` | `VisionBuilder.UI.Common/Plugins/PluginLoadContext.cs` |
| Extension methods (`RegisterModule`, etc.) | `VisionBuilder.UI.Common/Extensions.cs` |
| `ModulePriorityAttribute` | `VisionBuilder.UI.Common/Attributes/ModulePriorityAttribute.cs` |
| `IRecognitionControl` | `VisionBuilder.UI.Common/Processing/IRecognitionControl.cs` |
| `BaseRecognitionControl` | `VisionBuilder.UI.Common/Processing/BaseRecognitionControl.cs` |
| `BaseRecognitionControlSettings` | `VisionBuilder.UI.Common/Processing/BaseRecognitionControlSettings.cs` |
| `IImageSource` | `VisionBuilder.UI.Common/Processing/IImageSource.cs` |
| `ILoadingService` | `VisionBuilder.UI.Common/Processing/ILoadingService.cs` |
| `IRecipeCreationTool` | `VisionBuilder.UI.Common/ViewModel/Interfaces/UI/IRecipeCreationTool.cs` |
| `ILearningTool` | `VisionBuilder.UI.Common/ViewModel/Interfaces/UI/ILearningTool.cs` |
| `ImageProcessedEvent` | `VisionBuilder.UI.Common/Commands/ImageProcessedEvent.cs` |
| `RecipeData` | `VisionBuilder.UI.Common/ViewModel/Classes/RecipeData.cs` |
| `TypeConverterRegistry` | `framework/Inspectron.Settings/TypeConverterRegistry.cs` |
| `SingleCameraVM` | `VisionBuilder.UI.Common/ViewModel/SingleCameraVM.cs` |
| `ButtonDefinition` | `VisionBuilder.UI.Common/ViewModel/Classes/ButtonDefinition.cs` |
| Existing plugins | `Plugins/` directory |