Savor: Vision

Role: Solo Developer

Description: I created this specialised computer vision module for the Savor ecosystem to automate pantry inventory tracking by deploying optimised YOLOv11 instance segmentation models onto edge hardware. The system bridges the gap between high-level Python training frameworks and low-latency C++ production environments on the Raspberry Pi 5.

• Edge AI Optimization: Developed a pipeline to train custom YOLOv11 segmentation models and compile them using the Hailo Dataflow Compiler (DFC). This process converts standard PyTorch models into highly optimised HEF binaries capable of running on the Hailo-8 NPU (26 TOPS).
• C++ Inference Engine: Engineered a lightweight C++ runtime application using HailoRT. This replaces Python bindings to minimise overhead, ensuring maximum throughput and reducing CPU load on the host device.
• Custom Post-Processing: Implemented complex post-processing logic in C++ to handle the Hailo NPU's raw tensor output. This includes custom decoding for Segmentation Masks, matrix multiplication for mask assembly, and efficient Non-Maximum Suppression (NMS).
• Hardware Integration: Designed the system to utilise the PCIe bandwidth of the Raspberry Pi 5 effectively, managing data flow between the host memory (DDR) and the NPU's internal SRAM to bypass hardware limitations in specific neural network layers (such as the PSA attention block).

This is a project currently in active development so make sure to check the devlogs to stay updated!