HawkeyeVision/docs/PLUGIN_SYSTEM.md

VisionBuilder Plugin System

This document provides in-depth documentation of the VisionBuilder plugin system architecture and step-by-step instructions for implementing new plugins.

Table of Contents

• Overview
• Architecture
  • Plugin Lifecycle
  • Dependency Injection Scoping
  • Module Initialization Order
• Core Interfaces
  • IPlugin
  • IVisionBuilderModule
  • ISettings
  • IRecognitionControl
  • BaseRecognitionControl
  • IImageSource
  • ILoadingService
  • IRecipeCreationTool
  • ILearningTool
• Plugin Discovery and Loading
  • Directory Structure
  • enabled_plugins.txt
  • PluginLoader
  • PluginLoadContext (Assembly Isolation)
  • Plugin Profiles
• Event System
• Extension Methods Reference
• How to Implement a Plugin
  • Step 1: Create the Project
  • Step 2: Implement IPlugin
  • Step 3: Create Modules
  • Step 4: Add Settings
  • Step 5: Deploy and Enable
• Common Plugin Patterns
  • Pattern: Add a Background Module
  • Pattern: Replace the Recognition Control
  • Pattern: React to Image Processing Events
  • Pattern: External Hardware Integration (PLC/Serial)
  • Pattern: Register Custom Type Converters
  • Pattern: Override UI Tools
  • Pattern: Add Dynamic UI Buttons
• Existing Plugin Examples
• Key Source File Locations

---

Overview

The plugin system allows extending VisionBuilder with external assemblies that are loaded at runtime. Plugins can:

• Register app-wide (global) services and modules
• Register per-camera services and modules
• Replace core services like IRecognitionControl, ILearningTool, IRecipeCreationTool
• Subscribe to image processing events and camera lifecycle events
• Add configurable settings that persist via InspectronSettings
• Register custom type converters for settings serialization
• Integrate with external hardware (PLCs, barcode readers, serial devices)

Plugins are loaded from the ../Data/Plugins/ directory and are enabled/disabled via a text configuration file.

Architecture

Plugin Lifecycle

Application Startup
    |
    v
1. Main kernel created (StandardKernel)
    |
    v
2. kernel.UsePlugins(profile)
    |--- PluginLoader registered as IVisionBuilderModule
    |--- PluginLoader.InitializeModule(profile) called
    |--- Scans ../Data/Plugins/ for enabled plugin folders
    |--- Loads assemblies, discovers IPlugin implementations
    |--- Stores plugins in PluginLoader.Plugins list
    |
    v
3. kernel.RegisterGlobalPlugins()
    |--- For each IPlugin: calls RegisterGlobalModules(mainKernel)
    |--- Plugins register global modules, type converters, rebind global tools
    |
    v
4. For each camera:
    |
    |--- Create ChildKernel(mainKernel)
    |--- Register camera-specific bindings (CameraSettings, etc.)
    |--- kernel.RegisterCameraPlugins(cameraName)
    |    |--- For each IPlugin: calls RegisterCameraModules(childKernel, cameraName)
    |    |--- Plugins register camera modules, settings, rebind per-camera services
    |
    |--- kernel.RegisterSettings()
    |    |--- Finds all ISettings in kernel, calls RegisterSettings(InspectronSettings)
    |
    |--- kernel.InitializeModules()
    |    |--- Finds all IVisionBuilderModule in kernel
    |    |--- Sorts by [ModulePriority] descending (higher = earlier)
    |    |--- Calls InitializeModule() on each, skipping already-initialized
    |
    v
5. Application running — modules active, events flowing

Dependency Injection Scoping

The system uses two levels of Ninject kernels:

Main Kernel (StandardKernel) — Global scope:

• Shared across all cameras
• Holds InspectronSettings, UIConfiguration, PluginLoader
• Global modules registered here are singletons for the entire app
• RegisterGlobalModules() receives this kernel

Child Kernels (ChildKernel) — Per-camera scope:

• One child kernel per camera
• Inherits bindings from the main kernel (can resolve global services)
• Camera-specific bindings: CameraSettings, SingleCameraVM, IRecognitionControl, IImageSource
• RegisterCameraModules() receives the child kernel
• Services bound here are isolated per camera

MainKernel (StandardKernel)
  |--- InspectronSettings (singleton)
  |--- PluginLoader (singleton)
  |--- IRecipeCreationTool (singleton, rebindable)
  |--- Global IVisionBuilderModule instances
  |
  |--- ChildKernel (Camera "Left")
  |      |--- CameraSettings
  |      |--- SingleCameraVM (singleton)
  |      |--- IRecognitionControl (singleton)
  |      |--- IImageSource
  |      |--- Camera-specific IVisionBuilderModule instances
  |      |--- Camera-specific ISettings instances
  |
  |--- ChildKernel (Camera "Right")
         |--- (same structure, independent instances)

Module Initialization Order

Modules are initialized via kernel.InitializeModules(), which:

1. Collects all IVisionBuilderModule instances from the kernel
2. Filters out already-initialized modules (tracked per kernel)
3. Sorts by [ModulePriority(n)] attribute in descending order (higher priority = initialized first)
4. Calls InitializeModule() on each in order

Default priority is 0 if no attribute is present. The PluginLoader itself uses priority 1000 to ensure it runs first.

[ModulePriority(100)]  // Initialized before modules with lower priority
public class MyCriticalModule : IVisionBuilderModule { ... }

public class MyRegularModule : IVisionBuilderModule { ... }  // Priority 0 (default)

Core Interfaces

IPlugin

Location: VisionBuilder.UI.Common/Plugins/IPlugin.cs

The entry point for all plugins. Each plugin assembly must contain at least one class implementing this interface.

public interface IPlugin
{
    void RegisterGlobalModules(IKernel kernel);
    void RegisterCameraModules(IKernel kernel, string cameraName);
}

• RegisterGlobalModules(IKernel kernel) — Called once at startup with the main kernel. Register app-wide services, type converters, and rebind global tool interfaces here.
• RegisterCameraModules(IKernel kernel, string cameraName) — Called once per camera with the camera's child kernel. Register camera-specific modules, settings, and service overrides here.

IVisionBuilderModule

Location: VisionBuilder.UI.Common/IVisionBuilderModule.cs

Modules are lifecycle-managed components that perform deferred initialization after the DI container is fully configured.

public interface IVisionBuilderModule
{
    void InitializeModule();
}

Register modules using the extension method:

kernel.RegisterModule<MyModule>();
// Equivalent to:
// kernel.Bind<MyModule, IVisionBuilderModule>().To<MyModule>().InSingletonScope();

Modules receive their dependencies via constructor injection from Ninject. InitializeModule() is the place to start background tasks, subscribe to events, open connections, etc.

ISettings

Location: VisionBuilder.UI.Common/ISettings.cs

Settings classes register configurable properties with the InspectronSettings system.

public interface ISettings
{
    void RegisterSettings(InspectronSettings settings);
}

Settings are bound as constants (since they hold state) and must also be bound to ISettings so the framework discovers them:

kernel.Bind<MyPluginSettings, ISettings>().ToConstant(new MyPluginSettings(cameraName));

Inside RegisterSettings, use InspectronSettings.RegisterSimple() to expose properties:

public class MyPluginSettings : ISettings
{
    public string CameraName { get; }
    public int PollingInterval { get; set; } = 1000;
    public string ServerAddress { get; set; } = "127.0.0.1";

    public MyPluginSettings(string cameraName) => CameraName = cameraName;

    public void RegisterSettings(InspectronSettings settings)
    {
        settings.RegisterSimple(this, () => PollingInterval, CameraName + "/MyPlugin", "Polling interval (ms)");
        settings.RegisterSimple(this, () => ServerAddress, CameraName + "/MyPlugin", nameof(ServerAddress));
    }
}

IRecognitionControl

Location: VisionBuilder.UI.Common/Processing/IRecognitionControl.cs

The core interface for image processing control. One instance per camera.

public interface IRecognitionControl
{
    bool IsRunning { get; }
    List<RecipeData> GetRecipesData();
    void SetRecipe(RecipeData recipe);
    void Start();
    void Stop();
    void Pause();
    void Resume();
    event Action<ImageProcessedEvent> ImageProcessed;
    event Action<SessionStartedEvent> SessionStarted;
    event Action<SessionEndedEvent> SessionEnded;
    event Action<ErrorsInSequenceEvent> ErrorsInSequenceAlarm;
    void HotReload();
    event Action RecipeFileChanged;
}

Events fired during the processing loop:

• ImageProcessed — After each image is processed (contains result, errors, timing)
• SessionStarted — When Start() is called successfully
• SessionEnded — When Stop() completes
• ErrorsInSequenceAlarm — When consecutive errors exceed the configured threshold

BaseRecognitionControl

Location: VisionBuilder.UI.Common/Processing/BaseRecognitionControl.cs

Abstract base class that implements the recognition loop. Extend this when your plugin needs a custom image processing pipeline.

public abstract class BaseRecognitionControl : IRecognitionControl
{
    // Constructor — requires settings and loading service
    public BaseRecognitionControl(
        BaseRecognitionControlSettings settings,
        ILoadingService loadingService);

    // Must implement:
    public abstract List<RecipeData> GetRecipesData();
    protected abstract void Initialize(RecipeData currentRecipe);
    protected abstract void WarmUp();
    protected abstract (Mat originalImage, Mat analysisImage,
        TimeSpan processingTime, TimeSpan acquisitionTime,
        string[] errorNames)? ProcessImage(CancellationToken token);

    // Optional override:
    protected virtual void Cleanup() { }
    public virtual void HotReload() { }
}

The base class handles:

• Thread-safe start/stop/pause/resume lifecycle
• The main processing loop (calls ProcessImage repeatedly)
• Minimum processing time enforcement (configurable via settings)
• Consecutive error tracking and alarm firing
• Event dispatch (ImageProcessed, SessionStarted, SessionEnded, ErrorsInSequenceAlarm)

RecipeData:

public class RecipeData
{
    public Mat? Image { get; set; }       // Thumbnail image for UI
    public string RecipeName { get; set; } // Recipe identifier
}

BaseRecognitionControlSettings:

public abstract class BaseRecognitionControlSettings(string cameraName) : ISettings
{
    public string CameraName { get; }
    public int MinimumProcessingTime { get; set; }       // ms, enforced between frames
    public string RecipeNameFilter { get; set; } = "*";  // Glob filter for recipe listing
    public int ErrorsInSequenceAlarm { get; set; } = 5;  // Threshold for alarm event
}

IImageSource

Location: VisionBuilder.UI.Common/Processing/IImageSource.cs

Provides images to the recognition control.

public interface IImageSource
{
    Task<Mat> GetImage(CancellationToken token);
}

ILoadingService

Location: VisionBuilder.UI.Common/Processing/ILoadingService.cs

Shows/hides loading indicators in the UI during long operations.

public interface ILoadingService
{
    void StartLoading(string title);
    void StopLoading(string title);
}

IRecipeCreationTool

Location: VisionBuilder.UI.Common/ViewModel/Interfaces/UI/IRecipeCreationTool.cs

UI tool for creating new recipes. Default implementation (NoRecipeCreation) is disabled. Plugins can Rebind this to provide custom recipe creation.

public interface IRecipeCreationTool
{
    bool Enabled { get; set; }
    bool CreateRecipe(out string recipeName);
}

ILearningTool

Location: VisionBuilder.UI.Common/ViewModel/Interfaces/UI/ILearningTool.cs

UI tool for learning/training recipes from images. Default implementation (NoLearning) is disabled.

public interface ILearningTool
{
    bool IsLearningEnabled(string recipeName);
    void Learn(string recipeName, Mat image);
}

Plugin Discovery and Loading

Directory Structure

Plugins are deployed to the ../Data/Plugins/ directory (relative to the application executable). Each plugin resides in its own subfolder:

Data/
  Plugins/
    enabled_plugins.txt          # Lists enabled plugins (one per line)
    my_custom_plugins.txt        # Alternative profile (optional)
    B24SiemensPlugin/
      B24SiemensPlugin.dll       # Plugin assembly (name must match folder)
      (dependency DLLs)
    CandyboxPlugin/
      CandyboxPlugin.dll
      (dependency DLLs)
    MyNewPlugin/
      MyNewPlugin.dll            # Your plugin
      SomeLibrary.dll            # Any additional dependencies

Critical: The folder name and the main DLL name (without extension) must match exactly. The loader looks for <FolderName>/<FolderName>.dll.

enabled_plugins.txt

A simple text file listing enabled plugin folder names, one per line:

B24SiemensPlugin
CandyboxPlugin
MyNewPlugin

Plugins not listed here are skipped during loading. If the file doesn't exist, it's created empty (no plugins enabled).

PluginLoader

Location: VisionBuilder.UI.Common/Plugins/PluginLoader.cs

The PluginLoader is an IVisionBuilderModule with [ModulePriority(1000)] (highest built-in priority). It:

1. Reads the enabled plugins list from ../Data/Plugins/<profile>.txt
2. Iterates over subdirectories in ../Data/Plugins/
3. Skips folders not in the enabled list
4. Loads the main assembly from each enabled folder using PluginLoadContext
5. Scans loaded assembly types for IPlugin implementations
6. Creates instances via Activator.CreateInstance() and stores them in Plugins list

PluginLoadContext (Assembly Isolation)

Location: VisionBuilder.UI.Common/Plugins/PluginLoadContext.cs

Each plugin is loaded in its own AssemblyLoadContext to isolate dependencies:

• Uses AssemblyDependencyResolver to resolve managed assemblies from the plugin folder
• Supports native/unmanaged DLL loading for plugins with native dependencies
• Prevents version conflicts between plugins and the host application

Important project reference configuration: Plugin projects must reference VisionBuilder.UI.Common (and other host assemblies) with <Private>false</Private> and <ExcludeAssets>runtime</ExcludeAssets> to avoid duplicating host assemblies in the plugin output.

Plugin Profiles

The application supports multiple plugin configurations via profiles. The --profile / -p command-line argument selects which .txt file to use:

# Uses ../Data/Plugins/enabled_plugins.txt (default)
VisionBuilder.exe

# Uses ../Data/Plugins/production.txt
VisionBuilder.exe --profile production

# Uses ../Data/Plugins/testing.txt
VisionBuilder.exe -p testing

Event System

The recognition control fires events as Action<T> delegates. Modules subscribe in InitializeModule():

ImageProcessedEvent

Fired after each image is processed.

public class ImageProcessedEvent : IEvent
{
    public DateTime SessionStart { get; set; }
    public string ImageSource { get; set; }        // Camera name
    public string RecipeName { get; set; }
    public Mat ImageOriginal { get; set; }         // Raw camera image
    public Mat ImageAnalysis { get; set; }         // Annotated result image
    public bool HasError { get; set; }
    public List<string> ErrorNames { get; set; }   // Detected defects/errors
    public TimeSpan AnalysisTime { get; set; }
    public TimeSpan AcquisitionTime { get; set; }
    public TimeSpan SleepTime { get; set; }
}

SessionStartedEvent

Fired when recognition starts.

public class SessionStartedEvent
{
    public string RecipeName { get; set; }
    public DateTime SessionStarted { get; set; }
}

SessionEndedEvent

Fired when recognition stops.

public class SessionEndedEvent
{
    public DateTime SessionEnded { get; set; }
}

ErrorsInSequenceEvent

Fired when consecutive error count reaches the threshold.

public class ErrorsInSequenceEvent
{
    public int Errors { get; set; }
}

Extension Methods Reference

Location: VisionBuilder.UI.Common/Extensions.cs

Method	Description
kernel.RegisterModule<T>()	Binds T as both itself and IVisionBuilderModule in singleton scope
kernel.UsePlugins(profile)	Registers and initializes the PluginLoader with the given profile
kernel.RegisterGlobalPlugins()	Calls RegisterGlobalModules() on all loaded plugins
kernel.RegisterCameraPlugins(cameraName)	Calls RegisterCameraModules() on all loaded plugins (child kernel)
kernel.RegisterSettings()	Finds all ISettings instances and calls RegisterSettings()
kernel.InitializeModules()	Initializes all IVisionBuilderModule instances in priority order

How to Implement a Plugin

Step 1: Create the Project

Create a new .NET 8.0 class library project. The critical settings are:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    <!-- Required: enables dynamic loading as a plugin -->
    <EnableDynamicLoading>true</EnableDynamicLoading>

    <!-- Output directly to the Plugins directory for development -->
    <OutDir>..\Data\Plugins\MyPlugin</OutDir>
  </PropertyGroup>

  <ItemGroup>
    <!-- Reference the common library — Private=false prevents copying host DLLs -->
    <ProjectReference Include="..\VisionBuilder.UI.Common\VisionBuilder.UI.Common.csproj">
      <Private>false</Private>
      <ExcludeAssets>runtime</ExcludeAssets>
    </ProjectReference>

    <!-- Reference settings framework if needed -->
    <ProjectReference Include="..\framework\Inspectron.Settings\Inspectron.Settings.csproj">
      <Private>false</Private>
      <ExcludeAssets>runtime</ExcludeAssets>
    </ProjectReference>

    <!-- Add any other host references with Private=false -->
  </ItemGroup>

  <ItemGroup>
    <!-- Plugin-specific NuGet packages (these WILL be copied to output) -->
    <!-- <PackageReference Include="SomeLibrary" Version="1.0.0" /> -->
  </ItemGroup>
</Project>

Key project settings explained:

• EnableDynamicLoading — Tells the build system this assembly will be loaded dynamically. Ensures all plugin-specific dependencies are copied to the output directory.
• Private=false / ExcludeAssets=runtime on host references — Prevents the plugin from copying VisionBuilder.UI.Common.dll and other host assemblies into the plugin folder. The host application already has these loaded; duplicating them causes type identity conflicts.
• OutDir — Set to the Plugins directory for easy development. Adjust the relative path based on your project location.

Step 2: Implement IPlugin

Create a Plugin.cs (the class name can be anything):

using Ninject;
using VisionBuilder.UI.Common;
using VisionBuilder.UI.Common.Plugins;

namespace MyPlugin;

public class Plugin : IPlugin
{
    public void RegisterGlobalModules(IKernel kernel)
    {
        // Called once at startup with the main kernel.
        // Register app-wide services, type converters, or rebind global tools.

        // Example: Register a custom type converter for settings
        // TypeConverterRegistry.Register<List<MyMapping>>(new MyMappingConverter());

        // Example: Override the recipe creation tool globally
        // kernel.Rebind<IRecipeCreationTool>().To<MyRecipeCreationTool>().InSingletonScope();
    }

    public void RegisterCameraModules(IKernel kernel, string cameraName)
    {
        // Called once per camera with the camera's child kernel.
        // Register camera-specific modules and settings.

        // Register a module (will be initialized later via InitializeModules)
        kernel.RegisterModule<MyCameraModule>();

        // Register settings (bound as constant since they hold state)
        kernel.Bind<MyPluginSettings, ISettings>()
            .ToConstant(new MyPluginSettings(cameraName));
    }
}

Step 3: Create Modules

Modules do the actual work. They receive dependencies via constructor injection and perform initialization in InitializeModule().

using Serilog;
using VisionBuilder.UI.Common;
using VisionBuilder.UI.Common.Processing;

namespace MyPlugin;

public class MyCameraModule : IVisionBuilderModule
{
    private readonly IRecognitionControl _recognitionControl;
    private readonly MyPluginSettings _settings;

    // Dependencies are injected by Ninject from the camera's child kernel
    public MyCameraModule(
        IRecognitionControl recognitionControl,
        MyPluginSettings settings)
    {
        _recognitionControl = recognitionControl;
        _settings = settings;
    }

    public void InitializeModule()
    {
        // Subscribe to events
        _recognitionControl.ImageProcessed += OnImageProcessed;
        _recognitionControl.SessionStarted += OnSessionStarted;
        _recognitionControl.SessionEnded += OnSessionEnded;

        Log.Information("MyPlugin initialized for camera {Camera}", _settings.CameraName);
    }

    private void OnImageProcessed(ImageProcessedEvent e)
    {
        if (e.HasError)
        {
            Log.Warning("Errors detected: {Errors}", string.Join(", ", e.ErrorNames));
            // React to errors — send signal, log, notify, etc.
        }
    }

    private void OnSessionStarted(SessionStartedEvent e)
    {
        Log.Information("Session started: {Recipe}", e.RecipeName);
    }

    private void OnSessionEnded(SessionEndedEvent e)
    {
        Log.Information("Session ended");
    }
}

Available services for constructor injection in camera modules:

Service	Description
IRecognitionControl	Image processing control for this camera
SingleCameraVM	ViewModel for this camera (UI state, commands, DynamicButtons for adding custom buttons)
CameraSettings	Camera configuration (source type, label)
IImageSource	Image provider for this camera
InspectronSettings	Global settings manager
ILoadingService	Loading dialog service
Any ISettings bound in the kernel	Plugin-specific settings
Any IVisionBuilderModule bound in the kernel	Other modules (be careful of initialization order)

Step 4: Add Settings

Create a settings class to expose configurable properties:

using Inspectron.Settings;
using VisionBuilder.UI.Common;

namespace MyPlugin;

public class MyPluginSettings : ISettings
{
    public string CameraName { get; }

    // Configurable properties with defaults
    public string ServerAddress { get; set; } = "127.0.0.1";
    public int Port { get; set; } = 5000;
    public bool Enabled { get; set; } = true;

    public MyPluginSettings(string cameraName)
    {
        CameraName = cameraName;
    }

    public void RegisterSettings(InspectronSettings settings)
    {
        // Register each property under a category path
        // Format: settings.RegisterSimple(owner, propertyExpression, category, displayName)
        settings.RegisterSimple(this, () => ServerAddress, CameraName + "/MyPlugin", "Server address");
        settings.RegisterSimple(this, () => Port, CameraName + "/MyPlugin", "Server port");
        settings.RegisterSimple(this, () => Enabled, CameraName + "/MyPlugin", "Enable plugin");
    }
}

Settings are automatically persisted and restored by the InspectronSettings infrastructure. The category path (CameraName + "/MyPlugin") determines how settings appear in the UI.

Step 5: Deploy and Enable

1. Build the plugin — output goes to Data/Plugins/MyPlugin/
2. Add to enabled_plugins.txt — append MyPlugin to ../Data/Plugins/enabled_plugins.txt
3. Restart the application — the plugin will be discovered and loaded

Data/
  Plugins/
    enabled_plugins.txt    # Add "MyPlugin" line
    MyPlugin/
      MyPlugin.dll         # Folder name must match DLL name

Common Plugin Patterns

Pattern: Add a Background Module

Register a module that runs background work (e.g., listening on a port, polling a device).

// In Plugin.RegisterCameraModules:
kernel.RegisterModule<MyBackgroundModule>();

// Module implementation:
public class MyBackgroundModule : IVisionBuilderModule
{
    private readonly MyPluginSettings _settings;
    private TcpListener _listener;

    public MyBackgroundModule(MyPluginSettings settings)
    {
        _settings = settings;
    }

    public void InitializeModule()
    {
        _listener = new TcpListener(IPAddress.Any, _settings.Port);
        _listener.Start();
        Task.Run(() => AcceptClients());
    }

    private async Task AcceptClients() { /* ... */ }
}

Pattern: Replace the Recognition Control

Provide a completely custom image processing pipeline by replacing IRecognitionControl.

// In Plugin.RegisterCameraModules:

// 1. Remove existing settings binding (if replacing BaseRecognitionControlSettings)
var existingSettings = kernel.GetBindings(typeof(BaseRecognitionControlSettings)).First();
var toRemove = kernel.GetBindings(typeof(ISettings))
    .First(x => x.ProviderCallback.Target == existingSettings.ProviderCallback.Target);
kernel.RemoveBinding(toRemove);

// 2. Bind new settings
kernel.Bind<MyRecognitionSettings, BaseRecognitionControlSettings, ISettings>()
    .ToConstant(new MyRecognitionSettings(cameraName));

// 3. Rebind the recognition control
kernel.Rebind<IRecognitionControl>().To<MyRecognitionControl>().InSingletonScope();

// Settings class:
public class MyRecognitionSettings : BaseRecognitionControlSettings
{
    public MyRecognitionSettings(string cameraName) : base(cameraName) { }

    public string CustomProperty { get; set; } = "default";

    public override void RegisterSettings(InspectronSettings settings)
    {
        base.RegisterSettings(settings); // Register base properties
        settings.RegisterSimple(this, () => CustomProperty, CameraName + "/MyRecognition", nameof(CustomProperty));
    }
}

// Recognition control:
public class MyRecognitionControl : BaseRecognitionControl
{
    private readonly IImageSource _imageSource;

    public MyRecognitionControl(
        MyRecognitionSettings settings,
        ILoadingService loadingService,
        IImageSource imageSource) : base(settings, loadingService)
    {
        _imageSource = imageSource;
    }

    public override List<RecipeData> GetRecipesData()
    {
        // Return available recipes
        return new List<RecipeData>
        {
            new() { RecipeName = "Recipe1", Image = CreateThumbnail() }
        };
    }

    protected override void Initialize(RecipeData currentRecipe)
    {
        // Called when Start() is invoked — load recipe data, initialize models, etc.
    }

    protected override void WarmUp()
    {
        // Called once after Initialize, before the main loop.
        // Use for warm-up runs (e.g., first inference is slow for AI models).
    }

    protected override (Mat originalImage, Mat analysisImage,
        TimeSpan processingTime, TimeSpan acquisitionTime,
        string[] errorNames)? ProcessImage(CancellationToken token)
    {
        var sw = Stopwatch.StartNew();
        var image = _imageSource.GetImage(token).Result;
        var acquisitionTime = sw.Elapsed;

        // Process image...
        var errors = AnalyzeImage(image);
        sw.Stop();

        return (image, image, sw.Elapsed, acquisitionTime, errors);
        // Return null to stop the loop
    }

    protected override void Cleanup()
    {
        // Called after Stop() — release resources
    }
}

Pattern: React to Image Processing Events

Subscribe to events from the recognition control to react to results.

public class ResultReporterModule : IVisionBuilderModule
{
    private readonly IRecognitionControl _recognitionControl;

    public ResultReporterModule(IRecognitionControl recognitionControl)
    {
        _recognitionControl = recognitionControl;
    }

    public void InitializeModule()
    {
        _recognitionControl.ImageProcessed += OnImageProcessed;
        _recognitionControl.ErrorsInSequenceAlarm += OnErrorAlarm;
    }

    private void OnImageProcessed(ImageProcessedEvent e)
    {
        // e.HasError — true if any errors detected
        // e.ErrorNames — list of detected error/defect names
        // e.ImageOriginal — raw image (Mat)
        // e.ImageAnalysis — annotated image (Mat)
        // e.AnalysisTime — processing duration
        // e.RecipeName — active recipe
    }

    private void OnErrorAlarm(ErrorsInSequenceEvent e)
    {
        // e.Errors — number of consecutive errors
        // Trigger alarm, stop line, notify operator, etc.
    }
}

Pattern: External Hardware Integration (PLC/Serial)

Integrate with PLC controllers or serial devices.

public class PLCIntegrationModule : IVisionBuilderModule
{
    private readonly IRecognitionControl _recognitionControl;
    private readonly SingleCameraVM _cameraVm;
    private readonly MyPLCSettings _settings;

    public PLCIntegrationModule(
        IRecognitionControl recognitionControl,
        SingleCameraVM cameraVm,
        MyPLCSettings settings)
    {
        _recognitionControl = recognitionControl;
        _cameraVm = cameraVm;
        _settings = settings;
    }

    public void InitializeModule()
    {
        // Start PLC communication
        var server = new TcpListener(IPAddress.Any, _settings.Port);
        server.Start();

        // React to processing results
        _recognitionControl.ImageProcessed += e =>
        {
            SendResultToPLC(e.HasError ? "NOK" : "OK");
        };
    }

    // Use SingleCameraVM to control the camera programmatically
    private void SelectRecipe(string recipeName)
    {
        var vm = _cameraVm.GetRecipeSelectionVm();
        vm.SelectedRecipe = vm.Recipes.FirstOrDefault(r => r.RecipeName == recipeName);
        if (vm.SelectedRecipe != null)
        {
            // Must post to UI thread for SingleCameraVM operations
            _cameraVm.SynchronizationContext!.Post(_ =>
            {
                _cameraVm.ProcessRecipeSelectionVm(vm);
            }, null);
        }
    }
}

Note: When interacting with SingleCameraVM from a background thread, always use SynchronizationContext.Post() to marshal calls to the UI thread.

Pattern: Register Custom Type Converters

If your settings contain complex types that need custom serialization:

// In Plugin.RegisterGlobalModules:
public void RegisterGlobalModules(IKernel kernel)
{
    TypeConverterRegistry.Register<List<MyMapping>>(new MyMappingConverter());
}

// Converter implementation:
public class MyMappingConverter : ITypeConverter
{
    public object ConvertFrom(object value)
    {
        // Deserialize from string (e.g., JSON)
        return JsonSerializer.Deserialize<List<MyMapping>>((string)value);
    }

    public object ConvertTo(object value, Type destinationType)
    {
        // Serialize to string
        return JsonSerializer.Serialize((List<MyMapping>)value);
    }
}

Pattern: Override UI Tools

Replace default (disabled) UI tools with functional implementations:

public void RegisterGlobalModules(IKernel kernel)
{
    // Enable recipe creation
    kernel.Rebind<IRecipeCreationTool>().To<MyRecipeCreationTool>().InSingletonScope();
}

public void RegisterCameraModules(IKernel kernel, string cameraName)
{
    // Enable learning per camera
    kernel.Rebind<ILearningTool>().To<MyLearningTool>().InSingletonScope();
}

Pattern: Add Dynamic UI Buttons

Add custom buttons to the camera control panel (right side) or the main window bottom bar at runtime.

Camera-level buttons (right-side panel per camera):

public class MyCameraModule : IVisionBuilderModule
{
    private readonly SingleCameraVM _cameraVm;

    public MyCameraModule(SingleCameraVM cameraVm)
    {
        _cameraVm = cameraVm;
    }

    public void InitializeModule()
    {
        // Add a click button
        _cameraVm.AddButton(new ButtonDefinition(
            "myPlugin.resetCounters", "Reset Counters", () =>
            {
                // Handle click
            }));

        // Add a toggle button
        _cameraVm.AddButton(new ButtonDefinition(
            "myPlugin.autoMode", "Auto Mode", (isToggled) =>
            {
                // Handle toggle state change
            }, initialState: false));

        // Control visibility later
        _cameraVm.SetButtonVisibility("myPlugin.resetCounters", false);

        // Remove a button
        _cameraVm.RemoveButton("myPlugin.autoMode");
    }
}

App-level buttons (bottom bar):

public void RegisterGlobalModules(IKernel kernel)
{
    var mainVm = kernel.Get<MainWindowVM>();
    mainVm.AddButton(new ButtonDefinition(
        "myPlugin.dashboard", "Dashboard", () =>
        {
            // Open dashboard window
        }));
}

ButtonDefinition properties:

Property	Type	Description
Key	string	Unique identifier for the button (use plugin prefix, e.g., "myPlugin.action")
Title	string	Display text on the button
Type	ButtonType	Click or Toggle
IsVisible	bool	Controls button visibility (default: true). Observable — UI updates automatically
IsToggled	bool	Current toggle state (only for Toggle buttons). Observable

Thread safety: When calling AddButton, RemoveButton, or SetButtonVisibility from a background thread, marshal to the UI thread via SynchronizationContext.Post:

_cameraVm.SynchronizationContext!.Post(_ =>
{
    _cameraVm.AddButton(new ButtonDefinition("key", "Title", () => { }));
}, null);

Existing Plugin Examples

Plugin	Purpose	Key Patterns Used
TestPlugin	Minimal reference implementation	Module registration, custom recognition control
B24SiemensPlugin	Siemens PLC integration for recipe selection	Type converter, camera module, settings, PLC communication
CandyboxPlugin	Full-featured candy box inspection	Replace recognition control, barcode reader, recipe creation tool, learning tool
PralinenPLC	Send pass/fail results to PLC via TCP	Event subscription, network communication
PackstrasseBarcodeReader	Serial barcode reader for recipe selection	Settings, serial port module, barcode-to-recipe mapping

Key Source File Locations

Component	Path
IPlugin interface	VisionBuilder.UI.Common/Plugins/IPlugin.cs
IVisionBuilderModule interface	VisionBuilder.UI.Common/IVisionBuilderModule.cs
ISettings interface	VisionBuilder.UI.Common/ISettings.cs
PluginLoader	VisionBuilder.UI.Common/Plugins/PluginLoader.cs
PluginLoadContext	VisionBuilder.UI.Common/Plugins/PluginLoadContext.cs
Extension methods (RegisterModule, etc.)	VisionBuilder.UI.Common/Extensions.cs
ModulePriorityAttribute	VisionBuilder.UI.Common/Attributes/ModulePriorityAttribute.cs
IRecognitionControl	VisionBuilder.UI.Common/Processing/IRecognitionControl.cs
BaseRecognitionControl	VisionBuilder.UI.Common/Processing/BaseRecognitionControl.cs
BaseRecognitionControlSettings	VisionBuilder.UI.Common/Processing/BaseRecognitionControlSettings.cs
IImageSource	VisionBuilder.UI.Common/Processing/IImageSource.cs
ILoadingService	VisionBuilder.UI.Common/Processing/ILoadingService.cs
IRecipeCreationTool	VisionBuilder.UI.Common/ViewModel/Interfaces/UI/IRecipeCreationTool.cs
ILearningTool	VisionBuilder.UI.Common/ViewModel/Interfaces/UI/ILearningTool.cs
ImageProcessedEvent	VisionBuilder.UI.Common/Commands/ImageProcessedEvent.cs
RecipeData	VisionBuilder.UI.Common/ViewModel/Classes/RecipeData.cs
TypeConverterRegistry	framework/Inspectron.Settings/TypeConverterRegistry.cs
SingleCameraVM	VisionBuilder.UI.Common/ViewModel/SingleCameraVM.cs
ButtonDefinition	VisionBuilder.UI.Common/ViewModel/Classes/ButtonDefinition.cs
Existing plugins	Plugins/ directory