Exploration of Event-Based Camera Data with Spiking Neural Networks

Neuromorphic computing is a novel, non-von Neumann computing architecture that employs power efficient spiking neural networks on specialized hardware. Taking inspiration from the human brain, spiking neural networks are temporal computation units that propagate information throughout the network via binary spikes. Compared to conventional artificial neural networks, these networks can be more sparse, smaller in size, and more efficient power-wise when realized on neuromorphic hardware. Event-based cameras are novel vision sensors that capture visual information through a temporal stream of events instead of as a conventional RGB frame. These cameras are low-power collections of pixels that asynchronously emit events over time with microsecond level precision. Because of this, these cameras are often dubbed ``neuromorphic cameras''. The events generated by a camera are analogous to the spikes that occur within a spiking neural network. This results in the temporal event stream being a very natural input modality for spiking neural networks. This body of work explores pairing spiking neural networks more tightly with event-based cameras and explores not only developing spiking neural networks as classification or control agents, but also explores leveraging spiking neural networks as denoisers and spatial feature extractors. This work aims to further cement pairing neuromorphic systems with neuromorphic cameras in order to foster an extremely low-power computer vision system powered by brain-inspired neural networks.