Repository logo
Log In(current)
  1. Home
  2. Colleges & Schools
  3. Graduate School
  4. Doctoral Dissertations
  5. Selective Predictive Training and Structured Predictability in Neural Training Dynamics
Details

Selective Predictive Training and Structured Predictability in Neural Training Dynamics

Date Issued
August 2026
Author(s)
Wang, Fanqi
The University of Tennessee
Advisor(s)
Hairong Qi
Additional Advisor(s)
Dongsheng Ding
Dan Wilson
Zachary G. Nicolaou
Permanent URI
https://trace.tennessee.edu/handle/20.500.14382/61721
Abstract
The training of modern deep neural networks is still governed by long sequences of iterative gradient-based updates, and most acceleration techniques reduce the cost per step rather than the step-by-step nature of optimization. This dissertation studies predictive training as an alternative paradigm: when recent training dynamics contain usable short-horizon structure, selected parts of the optimization trajectory may be advanced by prediction rather than traversed step by step. The first part develops Predictive Differential Training (PDT), which combines Koopman/DMD-based weight forecasting with standard gradient-based optimization. Rather than applying predictions globally, PDT uses a coordinate-wise reliability mask based on scale control and dynamic consistency: coordinates whose predicted motion is sufficiently large, bounded, and directionally consistent with the local optimizer trajectory are advanced by prediction, while the rest fall back to the baseline optimizer. This hybrid update keeps gradient-based correction as a stabilizing reference while exploiting predictability where available. Across CIFAR-10 and ImageNet-1K classification, several architectures and optimizers, and a representative self-supervised setting, PDT reduces time-to-baseline training targets while maintaining comparable final performance, and ablations show that neither criterion alone is sufficient. The second part develops a measurement framework for structured predictability in training dynamics. Rather than equating predictability with the performance of a single forecaster, it uses complementary probes of directional persistence, recent-update subspace alignment, and predictor-based selectivity to measure where short-horizon temporal redundancy appears across parameter roles, training stages, architectures, and optimizers. Predictability is found to be localized, stage-dependent, architecture-shaped, and optimizer-conditioned: predictable regions often form a minority of the parameter space, and their location and persistence vary substantially across models and training conditions. Together, these contributions support a conditional view of predictive acceleration: training trajectories can contain exploitable temporal redundancy, but this redundancy is neither global nor uniform, so prediction is justified only where short-horizon regularity can be detected and used with appropriate safeguards. By pairing an online selective training method with an offline diagnostic framework for temporal predictability, the dissertation lays groundwork for measurement-guided predictive training, mechanism-aware online masking, optimizer-state-aware prediction control, and overhead-aware acceleration of large-scale training.
Subjects

neural training dynam...

Computer Science

Engineering, Electron...

Artificial Intelligen...

Machine Learning

Degree
Doctor of Philosophy
Major
Computer Engineering
File(s)
Thumbnail Image
Name

WANG-PRIMARY-2026.pdf

Size

18.01 MB

Format

Adobe PDF

Checksum (MD5)

71633b3ca5c0841354789d2f23736db3


University Libraries

1015 Volunteer Boulevard
Knoxville, TN 37996
865-974-4351

Map & Directions
Donate to the Libraries
  • About
  • John C. Hodges Society
  • Speaking Volumes magazine
  • Outreach
  • Directory
  • Employment
  • Policies
  • Library Intranet
University of Tennessee power T logo

The University of Tennessee, Knoxville
Knoxville, Tennessee 37996
865-974-1000

Events
A-Z
Apply
Privacy
Map
Directory
Give to UT
Accessibility

Built with DSpace-CRIS software - Extension maintained and optimized by 4Science