Hamiltonian Monte Carlo Methods for Statistical Learning and Applications

The first component of this dissertation consists of the presentation of a scalable Bayesian framework for the analysis of confocal fluorescence spectroscopy data, ad- dressing key limitations in traditional fluorescence correlation spectroscopy methods. Our framework captures molecular motion, microscope optics, and photon detection with high fidelity, enabling statistical inference of molecule trajectories from raw photon count data, introducing a superresolution parameter which further enhances trajectory estimation beyond the native time resolution of data acquisition. We develop a family of Hamiltonian Monte Carlo (HMC) algorithms that leverages the unique characteristics inherent to spectroscopy data analysis. To circumvent the instability of the explicit St¨ormer-Verlet method in this context, we introduce a semi- implicit (IMEX) method which treats the stiff and non-stiff parts differently, while leveraging the sparse structure of the discrete Laplacian for computational efficiency. Detailed numerical experiments demonstrate that this method improves upon fully explicit approaches, allowing larger HMC step sizes and maintaining second-order accuracy in position and energy. A second component of this dissertation is the description and analysis of a learning framework for parameter estimation in initial value problems that are assessed only indirectly via aggregate data such as sample means and/or variances. Our comprehensive framework follows Bayesian principles and consists of specialized Markov chain Monte Carlo computational schemes that rely on modified Hamiltonian Monte Carlo to align with constraints induced by summary statistics and a novel elliptical slice sampler adapted to the parameters of biological models. We benchmark our methods with synthetic data on microbial growth in batch culture meant to mimic growth curve data from laboratory replication experiments on Prochlorococcus microbes. The results indicate that our learning framework can utilize experimental or historical data and lead to robust parameter estimation and data assimilation in ODE models of biological dynamics that outperform least-squares fitting.