Dynamics of Diffusion

This thesis presents a novel approach to understanding Denoising Diffusion Probabilistic Models (DDPMs) by viewing the nonlinear diffusion process as a dynamical system through the lens of Koopman Operator Theory. By representing the nonlinear nature of the reverse diffusion process in DDPMs as a linear operator using Koopman Operator Theory, this work provides unique insights into how diffusion transforms probability distributions between a simple input distribution and a complex target distribution. This perspective allows for the identification of a Koopman-invariant subspace where the diffusion system is linear, enabling a Koopman operator to be learned using Koopman Autoencoders, a task previously accomplished using U-Net or transformer models with large capacity.

This work contributes to the field by offering a different perspective on the image diffusion process, connecting it to a broader range of dynamical systems and providing a new avenue for understanding and improving the performance of diffusion models. Additionally, this work presents a comprehensive analysis of the properties of the Koopman-invariant subspace and its implications for the stability and robustness of the diffusion process. By thoroughly investigating the characteristics of this subspace, this thesis provides insights into the fundamental mechanisms that govern the behavior of diffusion models and offers guidelines for designing more effective and reliable systems.