Optimization of Learning Algorithms in Neuromorphic Computing Systems.

Spiking Neural Networks (SNNs) are a type of artificial neural network that aim to more closely mimic the data processing processes observed in biological neural systems. However, one major challenge in training these networks has been their non-differentiable nature, which makes it difficult to apply traditional gradient-based learning techniques. Different approaches have been proposed to address this challenge, ranging from supervised learning - largely inspired by error backpropagation in Deep Neural Networks - to unsupervised learning, which closely emulates biological learning approaches such as spike-timing dependent plasticity (STDP). Neuromorphic hardware platforms such as Intel's Loihi offer programmable plasticity that allows a user to specify how synaptic plasticity functions. This work investigates the synaptic plasticity implementation in the TENNLab group neuromorphic hardware - RAVENS. Initially, we explored four Spike-Timing Dependent Plasticity (STDP) approaches in the RAVENS neuroprocessor including exponential, linear, flat, and no STDP. We found that the performance of each approach is heavily dependent on the application being evaluated. Subsequently, we looked deeper into the impact of an extended STDP table for a memristive neuroprocessor. The study concludes with optimizing the STDP lookup table entries using Bayesian Optimization to derive custom learning rules for specific applications. Our findings indicate that these custom learning rules, distinct from standard STDP values, outperform the default STDP rules, with performance increases ranging from 2% to 12% for certain applications.

Additionally, some of these rules demonstrate effective generalization across multiple applications. This work establishes an effective approach for deriving optimal STDP learning rules tailored to unique neuromorphic applications. We highlight the importance of programmable plasticity for enhancing algorithm and application performance in neuromorphic computing systems.