Architecture for Real-Time, Low-SWaP Embedded Vision Using FPGAs

In this thesis we designed, prototyped, and constructed a printed circuit board for real-time, low size, weight, and power (SWaP) HDMI video processing and developed a general purpose library of image processing functions for FPGAs.

The printed circuit board is a baseboard for a Xilinx Zynq based system-on-module (SoM). The board provides power, HDMI input, and HDMI output to the SoM and enables low-SWaP, high-resolution, real-time video processing.

The image processing library for FPGAs is designed for high performance and high reusability. These objectives are achieved by utilizing the Chisel hardware construction language to create parameterized modules that construct low-level FPGA implementations of image processing functions. Each module in the library can be used independently or combined with other modules to develop more complex processing functions. In addition, these modules can be used together with other existing firmware resources.

The circuit board and image processing library were then used together to develop, synthesize, and implement a real-time low-light video enhancement pipeline. It was also used to benchmark and estimate the performance of accelerating convolution for use in convolutional neural networks. In these applications our system was able to perform up to 20 times faster than the CPU at the same power consumption.

The final circuit board measures only 1.5x3x0.6 inches, weighs 1.00 ounce, and consumes less than 5 Watts when fully operational. It has been tested with HDMI video stream at 60 frames per second with input resolutions up to 1440x900 pixels and output resolutions up to 1920x1080 pixels. In addition, all of the modules in the library are optimized to be able to operate at no less than 60 frames per second on full high-definition (1920x1080 pixel) video.