Preference-Aware Treatment Planning via Reinforcement Learning for Chronic Multi-morbidity
Date Issued
August 2026
Author(s)
Payani, Nafiseh
The University of Tennessee
Advisor(s)
Anahita Khojandi
Additional Advisor(s)
Bing Yao
Soham Das
Wilson, G. Anthony
Abstract
Multimorbidity affects a growing proportion of the global population and poses fundamental challenges for clinical decision-making, where treatment must balance competing therapeutic targets, adverse drug interactions, and patient-specific variability that static guideline-based protocols cannot adequately address. Reinforcement learning (RL) offers a principled framework for sequential treatment planning under such complexity, enabling policies to optimize long-horizon patient outcomes directly from observational data. This thesis develops a preference-aware offline RL framework for chronic multimorbidity treatment planning, evaluated on the co-management of hypertension (HTN) and type 2 diabetes mellitus (T2DM), one of the most prevalent and clinically burdensome comorbid pairs, affecting nearly 75\% of diabetic adults worldwide.
The first chapter introduces the first large-scale offline deep RL agent for the joint pharmacological management of comorbid HTN and T2DM. Trained on 603,966 longitudinal transitions from 48,015 real-world outpatients, the agent optimizes age-stratified quality-adjusted life years (QALYs) and is paired with a robust EHR data processing pipeline and interpretablity analysis decision. The chapter also introduces an intuitive visual policy evaluation tool, establishing a strong empirical baseline for comorbidity-aware treatment recommendation.
The second chapter extends this foundation to a hierarchical reinforcement learning (HRL) architecture that decomposes the treatment planning problem across three simultaneous intervention axes: antidiabetic therapy, antihypertensive therapy, and BMI reduction. This architecture incorporates a cooperation-aware action masking mechanism in both data preprocessing and training that encodes patient preference and behavioral adherence directly into the policy, enabling clinically coherent, preference-sensitive joint recommendations.
The third chapter establishes structural properties of the full hierarchical factored action framework. Formal proofs are provided for the soundness of cooperation masking, the error bounds of the additive Q-value decomposition used across the action and reward space. Together, these three chapters contribute to advance principled, preference-aware treatment planning for chronic multimorbidity and lay the groundwork for generalizing the framework to broad multimorbidity settings.
Degree
Doctor of Philosophy
Major
Industrial Engineering
Embargo Date
August 1, 2027