This presentation was given at the Prevention Research Center at Penn State University on March 25, 2026 (https://ssri.psu.edu/events/digital-adaptive-interventions-common-misconceptions-and-opportunities-prc-quantdev-joint) and at the Mobile Health Training Institute (mHTi) on April 13, 2026 (https://mhti.md2k.org/program/2026-program/).
Abstract: We present our ongoing work on the development of an online reinforcement learning (RL) algorithm for dyadic digital intervention settings in which the task for the RL algorithm is to assist the target person with a difficult illness be adherent to behavioral activities. To achieve this goal the RL algorithm will not only deliver digital interventions to the target person but also deliver interventions to assist the care partner to manage caregiving burden and help the two individuals improve their relationship. That is, different RL components target different elements of the dyad. The RL algorithm is a multi-agent RL algorithm in which the 3 agents make decisions on the 3 elements of the dyad. We incorporate domain knowledge in the form of approximal causal directed acyclic graphs to speed up online learning in this sparse data setting. This work is…
Murphy, S.A., & Nahum-Shani, I. (2026, January) Just In Time Adaptive Interventions: Opportunities, Misconceptions, and the Promise of AI-based Algorithms. NIH Digital Health Scientific Interest Group (SIG)’s Speaker Series. NIH. Bethesda, MD. You can watch this presentation here: https://videocast.nih.gov/watch/0b1c35f9-0852-11f1-9f14-124f0a52e769
Authors: Kelly W Zhang, Nowell Closser, Anna L Trella, Susan A Murphy Abstract Adaptive treatment assignment algorithms, such as bandit algorithms, are increasingly used in digital health intervention clinical trials. Frequently the data collected from these trials is used to conduct causal inference and related data analyses to decide how to refine the intervention, and whether to roll-out the intervention more broadly. This work studies inference for estimands that depend on the adaptive algorithm itself; a simple example is the mean reward under the adaptive algorithm. Specifically, we investigate the replicability of statistical analyses concerning such estimands when using data from trials deploying adaptive treatment assignment algorithms. We demonstrate that many standard statistical estimators can be inconsistent and fail to be replicable across repetitions of the clinical trial, even as the sample size grows large.…
Abstract: In this talk, Susan will discuss first solutions to some of the challenges we face in developing online RL algorithms for use in digital health interventions targeting patients struggling with health problems such as substance misuse, hypertension, and bone marrow transplantation. Digital health raises a number of challenges to the RL community including different sets of actions, each set intended to impact patients over a different time scale; the need to learn both within an implementation and between implementations of the RL algorithm; noisy environments; and a lack of mechanistic models. In all of these settings, the online line algorithm must be stable and autonomous. Despite these challenges, RL, with careful initialization, with careful management of bias/variance tradeoff, and by close collaboration with health scientists, can be successful. We can make an impact! Dr. Susan Murphy has…
Authors: Inbal Nahum-Shani, Susan A Murphy Abstract The past decade has seen a surge in developing just-in-time adaptive interventions (JITAIs)-an intervention approach that leverages advancements in digital technologies to address the rapidly changing needs of individuals in daily life. This article provides an overview of the state of science on JITAI development and highlights important directions for future research. We explain what a JITAI is (and what it is not) and review the scientific and practical rationales underlying this approach. We also call attention to three key challenges relating to the development of JITAIs. The first challenge is that individuals may not be able to engage with (i.e., invest energy in) an intervention when they need it most in daily life. The second concerns the generally suboptimal engagement of individuals in interventions that leverage digital…
Authors: Ziping Xu, Hinal Jajal, Sung Won Choi, Inbal Nahum-Shani, Guy Shani, Alexandra M. Psihogios, Pei-Yao Hung, Susan A. Murphy Abstract Medication adherence is critical for the recovery of adolescents and young adults (AYAs) who have undergone hematopoietic cell transplantation. However, maintaining adherence is challenging for AYAs after hospital discharge, who experience both individual (e.g. physical and emotional symptoms) and interpersonal barriers (e.g., relational difficulties with their care partner, who is often involved in medication management). To optimize the effectiveness of a three-component digital intervention targeting both members of the dyad as well as their relationship, we propose a novel Multi-Agent Reinforcement Learning (MARL) approach to personalize the delivery of interventions. By incorporating the domain knowledge, the MARL framework, where each agent is responsible for the delivery of one intervention component, allows for faster learning…