HawkeyeVision/Hawkeye.VisionBuilder.Workflow/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Hawkeye.VisionBuilder.Workflow is a computer vision workflow system built on .NET 8.0 that processes images through configurable operation pipelines. The system supports both CPU and GPU configurations with ONNX runtime for AI models.

Build Configuration

The project uses conditional package references based on configuration:

• CPU Configuration: Uses YoloV8 (4.1.5) and Microsoft.ML.OnnxRuntime (1.18.0)
• GPU Configuration: Uses YoloV8.Gpu (4.1.7) and Microsoft.ML.OnnxRuntime.Gpu (1.18.0)

Build the project with:

dotnet build -c Debug      # For GPU configuration
dotnet build -c CPU        # For CPU-only configuration

Core Architecture

Operation System

• BaseOperation: Abstract base class for all image processing operations. All operations inherit from this and implement InterpretInternal(Context context)
• Context: Holds the current processing state including ActiveImage, LastCameraImage, GraphicsElements, and image Memory
• OperationDiscoveryService: Dynamically discovers and instantiates operations using reflection and Ninject DI

Operation Categories

Operations are organized in folders by category:

• AI/: AI model operations (YOLO, ONNX models, color classification)
• Simple/: Basic operations (blob detection, edge finding)
• Filters/: Image filtering operations
• Morphology/: Morphological operations (dilation, erosion, etc.)
• Image/: Image manipulation (convert, channels, etc.)
• Basic/: Fundamental operations (threshold, blur, etc.)

Key Components

• HawkeyeImage: Wrapper around OpenCV Mat with filename metadata
• WorkflowConfiguration: Manages camera types, Python paths, outputs, and recipe selection
• IImageSource: Interface for image input sources with event-driven architecture
• Graphics Elements: Drawable elements (rectangles, lines, polygons) for ROI definition

Python Integration

• Uses IronPython for scripting capabilities
• Python models executed via PythonModelProxy and PythonModelProxyRPC
• Default Python path: ~/miniconda3/envs/ai3_cpu/python.exe

Operation Attributes

• [Category("name")]: Groups operations in UI
• [NotForTool]: Excludes properties from parameter serialization
• [IgnoreOperation]: Excludes operations from discovery

Development Patterns

Creating New Operations

1. Inherit from BaseOperation
2. Add [Category("CategoryName")] attribute
3. Implement InterpretInternal(Context context) method
4. Use helper methods: CheckImageExists(), CheckColorful(), CheckGrayscale()
5. Set Result and Status properties for operation feedback

Image Processing Flow

1. Operations receive Context with current ActiveImage
2. Process image using OpenCV operations
3. Update context with results (new images, graphics elements)
4. Set operation status and result state

Memory Management

Operations can store/retrieve images using context.Memory dictionary for intermediate results.

Recipe Serialization System

The workflow system supports two serialization formats for saving and loading recipes:

Binary Serialization (.hrcp files)

The original format using BinaryWriter and BinaryReader:

• WorkflowList.Save(BinaryWriter): Saves configuration, operations, and recipe image
• WorkflowList.Load(BinaryReader, OperationDiscoveryService): Loads complete workflow
• BaseOperation.Save(BinaryWriter): Saves operation ID and label
• BaseOperation.Load(BinaryReader): Loads operation ID and label

XML Serialization (.xhrcp files)

The new human-readable format using XmlWriter and XmlReader:

• WorkflowList.SaveXML(XmlWriter): Saves complete workflow as structured XML
• WorkflowList.LoadXML(XmlReader, OperationDiscoveryService): Loads workflow from XML
• BaseOperation.SaveXML(XmlWriter): Saves operation properties and metadata
• BaseOperation.LoadXML(XmlReader): Loads operation properties with type conversion

XML Structure

<Workflow>
  <Configuration RuntimeCameraType="..." DevelopmentCameraType="..." ... />
  <Operations Count="N">
    <Operation Type="AssemblyQualifiedName" Id="..." Label="...">
      <Property Name="PropertyName" Type="System.Int32">Value</Property>
      ...
    </Operation>
  </Operations>
  <RecipeImage Length="N">Base64EncodedImageData</RecipeImage>
</Workflow>

Implementing Custom Operation Serialization

For operations that need custom XML serialization beyond the default property serialization:

Override SaveXML for Custom Save Logic:

public override void SaveXML(XmlWriter writer)
{
    base.SaveXML(writer); // Save standard properties
    
    // Custom serialization logic
    writer.WriteStartElement("CustomData");
    writer.WriteAttributeString("SpecialProperty", SpecialValue.ToString());
    writer.WriteEndElement();
}

Override LoadXML for Custom Load Logic:

public override void LoadXML(XmlReader reader)
{
    base.LoadXML(reader); // Load standard properties
    
    // Custom deserialization logic
    while (reader.Read())
    {
        if (reader.NodeType == XmlNodeType.Element && reader.Name == "CustomData")
        {
            SpecialValue = reader.GetAttribute("SpecialProperty");
        }
        else if (reader.NodeType == XmlNodeType.EndElement && reader.Name == GetOperationElementName())
        {
            break;
        }
    }
}

Override GetOperationElementName for Custom Element Names:

protected override string GetOperationElementName()
{
    return "MyCustomOperation"; // Used for XML end element detection
}

Supported Property Types in XML Serialization

The default XML serialization handles these types automatically:

• Primitive types (int, double, float, bool)
• String
• Guid
• Enums
• Complex types (converted to string representation)

For complex types requiring special serialization, override the SaveXML/LoadXML methods.