Advancing Self-Supervised Visual Representation Learning: From Multi-Scale Consistency to Disentangled Generative Models
Date Issued
August 2026
Author(s)
Tang, Maofeng
The University of Tennessee
Advisor(s)
Hairong Qi
Additional Advisor(s)
Xiongye Xiao
Ling Liang
Hector Santos Villalobos
Abstract
Self-supervised visual representation learning has achieved remarkable success, yet fundamental questions remain about designing representations that are simultaneously high-performing, interpretable, and controllable. This dissertation proposal advances representation learning through a systematic progression demonstrating that domain-specific challenges inspire architectural innovations that, when properly abstracted, yield broadly applicable principles.
We establish foundations in remote sensing, where extreme scale variations (50× differences in ground sampling distance) stress-test representation learning methods. Global-local balanced contrastive learning explicitly enforces both image-level semantics and spatial-level consistency for improved dense prediction. Cross-Scale MAE extends masked image modeling through multi-scale patch extraction and cross-scale masking strategies, demonstrating that explicit multi-scale mechanisms substantially improve scale robustness. These contributions validate that domain-specific characteristics inspire principled approaches advancing representation learning.
Building on these insights, we demonstrate generalization to natural images through unified masked image modeling that systematically integrates masking strategies, teacher networks, multi-level distillation, and decoder architectures, achieving state-of-the-art results on ImageNet and COCO. This validates that domain-specific insights transfer when properly abstracted, while exposing a critical limitation: high benchmark performance does not guarantee interpretable representations where semantic factors are explicitly organized.
Addressing this gap, we introduce factor tokens—dedicated learnable tokens in vision transformers—constrained by a Minimum Volume Constraint (MVC) that encourages statistical independence through geometric principles. Factor tokens achieve strong disentanglement while maintaining competitive task performance, with discovered factors corresponding to semantically meaningful dimensions enabling precise, independent attribute manipulation. This demonstrates that interpretability and performance are compatible when disentanglement is architecturally integrated.
We propose two future research directions synthesizing accumulated insights. First, integrating factor tokens into Diffusion Transformers hypothesizes that explicitly disentangled latent spaces enable fine-grained controllable generation superior to existing text-based or spatial conditioning methods. This tests whether the same principles enabling interpretable recognition also enable controllable synthesis. Second, next-generation remote sensing foundation models combining multi-scale processing, unified MIM principles, optional factor tokens, and Mixture of Experts for efficient scaling validate bidirectional knowledge transfer—from domain-specific challenges to general principles and back to advanced applications. A detailed 6-month timeline allocates equal effort to these complementary directions, with controllable DiT testing disentanglement in generation and foundation models demonstrating systematic integration benefits.
The research progression exemplifies a methodology where extreme domain-specific challenges inspire innovations, abstraction yields general principles, imposed structure enables interpretability, and synthesis creates advanced applications—demonstrating that the path from specific to general to specific is progressive, with each iteration building on accumulated insights.
Subjects
Degree
Doctor of Philosophy
Major
Computer Engineering
File(s)![Thumbnail Image]()
Name
TANG-PRIMARY-2026.pdf
Size
22.77 MB
Format
Adobe PDF
Checksum (MD5)
b28619d258d43a34440cf81d8bbf961b