Public Health

Forthcoming
Killian JA, Xu L, Biswas A, Tambe M. Restless and Uncertain: Robust Policies for Restless Bandits via Deep Multi-Agent Reinforcement Learning, in Uncertainty in Artificial Intelligence (UAI 2022). ; Forthcoming. arXiv
2023
Biswas A, Killian JA, Diaz PR, Ghosh S, Tambe M. Fairness for Workers Who Pull the Arms: An Index Based Policy for Allocation of Restless Bandit Tasks, in 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023). ; 2023.
Killian JA, Biswas A, Xu L, Verma S, Nair V, Taneja A, Madhiwala N, Hedge A, Diaz PR, Johnson-Yu S, et al. Robust Planning over Restless Groups: Engagement Interventions for a Large-Scale Maternal Telehealth Program, in Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI 2023). ; 2023.
2022
Biswas A, Patro GK, Ganguly N, Gummadi KP, Chakraborty A. Towards Fair Recommendation in Two-Sided Platforms. ACM Transactions on the Web (TWEB). 2022;16 (2) :1-34. Publisher's Version
Narang S, Biswas A, Yadati N. On Achieving Leximin Fairness and Stability in Many-to-One Matchings, in Proceedings of the International Conference on Autonomous Agents and Multiagent Systems as an extended abstract (AAMAS 2022). ; 2022. arXiv
Mate A, Biswas A, Siebenbrunner C, Ghosh S, Tambe M. Efficient Algorithms for Finite Horizon and Streaming Restless Multi-Armed Bandit Problems, in Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2022). ; 2022. arXiv
2021
Killian JA, Biswas A, Shah S, Tambe M. Q-Learning Lagrange Policies for Multi-Action Restless Bandits, in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2021). ; 2021. arXiv
Biswas A, Aggarwal G, Varakantham P, Tambe M. Learn to Intervene: An Adaptive Learning Policy for Restless Bandits in Application to Preventive Healthcare, in Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI 2021). ; 2021 :4036-4049. Publisher's Version
Biswas A, Aggarwal G, Varakantham P, Tambe M. Learning Index Policies for Restless Bandits with Application to Maternal Healthcare, in Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021). ; 2021. Publisher's Version
Saksono H. Asset-Based Insights on Designing Fitness Promotion Techs in Boston’s Low-SES Neighborhoods, in ; 2021. Publisher's VersionAbstract

The disproportionate burden of obesity among low-socioeconomic status (SES), Black, and Latinx households underscores the ever-increasing health disparities in the United States. This epidemic can be prevented by prioritizing physical activity interventions for these populations. However, many technology-based physical activity interventions were designed for the individuals rather than for the individuals amidst their social environment.

In this position paper, I report my eight-year research and technology designs processes. Although this research was not specifically guided by asset-based design principles, assets consistently emerged during the in-depth fieldwork. They include family relationships and caregiving communities. These assets appeared to influence the motivation to engage with health technologies and also enhance family physical activity self-efficacy. However, since my studies were not guided by asset-based design principles, some key assets may not be sufficiently identified. By participating in this workshop, I seek to collaboratively explore how to support communities to identify their assets and also how to translate assets into technology designs towards enhancing health equity.

Saksono H, Castaneda-Sceppa C, Hoffman JA, el-Nasr MS, Parker AG. StoryMap: Using Social Modeling and Self-Modeling to Support Physical Activity Among Families of Low-SES Backgrounds, in CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Yokahama, Japan: 2021 Proceedings ; 2021 :1-14. Publisher's VersionAbstract
Physical activity (PA) is crucial for reducing the risk of obesity, an epidemic that disproportionately burdens families of low-socioeconomic status (SES). While fitness tracking tools can increase PA awareness, more work is needed to examine (1) how such tools can help people benefit from their social environment, and (2) how reflections can help enhance PA attitudes. We investigated how fitness tracking tools for families can support social modeling and self-modeling (through reflection), two critical processes in Social Cognitive Theory. We developed StoryMap, a novel fitness tracking app for families aimed at supporting both modes of modeling. Then, we conducted a five-week qualitative study evaluating StoryMap with 16 low-SES families. Our findings contribute an understanding of how social and self-modeling can be implemented in fitness tracking tools and how both modes of modeling can enhance key PA attitudes: self-efficacy and outcome expectations. Finally, we propose design recommendations for social personal informatics tools.
storymap_using_social_modeling_and_self-modeling_to_support_physical_activity_among_families_of_low-ses_backgrounds.pdf
Zhang H, Dullerud N, Seyyed-Kalantari L, Morris Q, Joshi S, Ghassemi M. An empirical framework for domain generalization in clinical settings, in ACM Conference on Health, Inference, and Learning. Virtual Event, UA ; 2021.Abstract
Clinical machine learning models experience significantly degraded performance in datasets not seen during training, e.g., new hospitals or populations. Recent developments in domain generalization offer a promising solution to this problem by creating models that learn invariances across environments. In this work, we benchmark the performance of eight domain generalization methods on multi-site clinical time series and medical imaging data. We introduce a framework to induce synthetic but realistic domain shifts and sampling bias to stress-test these methods over existing nonhealthcare benchmarks. We find that current domain generalization methods do not achieve significant gains in out-of-distribution performance over empirical risk minimization on real-world medical imaging data, in line with prior work on general imaging datasets. However, a subset of realistic induced-shift scenarios in clinical time series data exhibit limited performance gains. We characterize these scenarios in detail, and recommend best practices for domain generalization in the clinical setting.
zhang_et_al._-_2021_-_an_empirical_framework_for_domain_generalization_i.pdf
Ou H-C, Chen H, Jabbari S, Tambe M. Active Screening for Recurrent Diseases: A Reinforcement Learning Approach. 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS). 2021.Abstract
Active screening is a common approach in controlling the spread of recurring infectious diseases such as tuberculosis and influenza. In this approach, health workers periodically select a subset of population for screening. However, given the limited number of health workers, only a small subset of the population can be visited in any given time period. Given the recurrent nature of the disease and rapid spreading, the goal is to minimize the number of infections over a long time horizon. Active screening can be formalized as a sequential combinatorial optimization over the network of people and their connections. The main computational challenges in this formalization arise from i) the combinatorial nature of the problem, ii) the need of sequential planning and iii) the uncertainties in the infectiousness states of the population.
ou_et_al._-_2021_-_active_screening_for_recurrent_diseases_a_reinfor.pdf
Killian JA, Perrault A, Tambe M. Beyond “To Act or Not to Act”: Fast Lagrangian Approaches to General Multi-Action Restless Bandits. IJCAI 2021 Workshop on AI for Social Good. 2021.Abstract
We present a new algorithm and theoretical results for solving Multi-action Multi-armed Restless Bandits, an important but insufficiently studied generalization of traditional Multi-armed Restless Bandits (MARBs). Multi-action MARBs are capable of handling critical problem complexities often present in AI4SG domains like anti-poaching and healthcare, but that traditional MARBs fail to capture. Limited previous work on Multi-action MARBs has only been specialized to sub-problems. Here we derive BLam, an algorithm for general Multi-action MARBs using Lagrangian relaxation techniques and convexity to quickly converge to good policies via bound optimization. We also provide experimental results comparing BLam to baselines on a simulated distributions motivated by a real-world community health intervention task, achieving up to five-times speedups over more general methods without sacrificing performance.
Beyond “To Act or Not to Act”: Fast Lagrangian Approaches to General Multi-Action Restless Bandits
Biswas A, Aggarwal G, Varakantham P, Tambe M. Learning Restless Bandits in Application to Call-based Preventive Care Programs for Maternal Healthcare, in IJCAI 2021 Workshop on AI for Social Good. ; 2021.Abstract
This paper focuses on learning index-based policies in rest- less multi-armed bandits (RMAB) with applications to public health concerns such as maternal health. Maternal health is a very important public health concern. It refers to the health of women during their pregnancy, childbirth, and the post- natal period. Although maternal health has received significant attention [World Health Organization, 2015], the number of maternal deaths remains unacceptably high, mainly because of the delay in obtaining adequate care [Thaddeus and Maine, 1994]. Most maternal deaths can be prevented by providing timely preventive care information. However, such information is not easily accessible by underprivileged and low-income communities. For ensuring timely information, a non-profit organization, called ARMMAN [2015], carries out a free call-based program called mMitra for spreading preventive care information among pregnant women. Enrollment in this program happens through hospitals and non-government organizations. Each enrolled woman receives around 140 automated voice calls, throughout their pregnancy period and up to 12 months after childbirth. Each call equips women with critical life-saving healthcare information. This program pro- vides support for around 80 weeks. To achieve the vision of improving the well-being of the enrolled women, it is important to ensure that they listen to most of the information sent to them via the automated calls. However, the organization observed that, for many women, their engagement (i.e., the overall time they spend listening to the automated calls) gradually decreases. One way to improve their engagement is by providing an intervention (that would involve a personal visit by health-care worker). These interventions require the dedicated time of the health workers, which is often limited. Thus, only a small fraction of the overall enrolled women can be provided with interventions during a time period. More- over, the extent to which the engagement improves upon intervention varies among individuals. Hence, it is important to carefully choose the beneficiaries who should be provided interventions at a particular time period. This is a challenging problem owing to multiple key reasons: (i) Engagement of the individual beneficiaries is un- certain and changes organically over time; (ii) Improvement in the engagement of a beneficiary post-intervention is un- certain; (iii) Decision making with respect to interventions (which beneficiaries should have intervention) is sequential, i.e., decisions at a step have an impact on the state of beneficiaries and decisions to be taken at the next step; (iv) Number of interventions are budgeted and are significantly smaller than the total number of beneficiaries. Due to the uncertainty, sequential nature of decision making, and weak dependency amongst patients through a budget, existing research [Lee et al., 2019; Mate et al., 2020; Bhattacharya, 2018] in health interventions has justifiably employed RMABs. However, existing research focuses on the planning problem assuming a priori knowledge of the underlying uncertainty model, which can be quite challenging to obtain. Thus, we focus on learning intervention decisions in absence of the knowledge of underlying uncertainty.
2020
Xu L, Bondi E, Fang F, Perrault A, Wang K, Tambe M. Dual-Mandate Patrols: Multi-Armed Bandits for Green Security. arXiv:2009.06560 [cs, stat]. 2020. Publisher's VersionAbstract
Conservation efforts in green security domains to protect wildlife and forests are constrained by the limited availability of defenders (i.e., patrollers), who must patrol vast areas to protect from attackers (e.g., poachers or illegal loggers). Defenders must choose how much time to spend in each region of the protected area, balancing exploration of infrequently visited regions and exploitation of known hotspots. We formulate the problem as a stochastic multi-armed bandit, where each action represents a patrol strategy, enabling us to guarantee the rate of convergence of the patrolling policy. However, a naive bandit approach would compromise short-term performance for long-term optimality, resulting in animals poached and forests destroyed. To speed up performance, we leverage smoothness in the reward function and decomposability of actions. We show a synergy between Lipschitzcontinuity and decomposition as each aids the convergence of the other. In doing so, we bridge the gap between combinatorial and Lipschitz bandits, presenting a no-regret approach that tightens existing guarantees while optimizing for short-term performance. We demonstrate that our algorithm, LIZARD, improves performance on real-world poaching data from Cambodia.
aaai21_dual_mandate_patrols.pdf
Prins A, Mate A, Killian JA, Abebe R, Tambe M. Incorporating Healthcare Motivated Constraints in Restless Bandit Based Resource Allocation. NeurIPS 2020 Workshops: Challenges of Real World Reinforcement Learning, Machine Learning in Public Health (Best Lightning Paper), Machine Learning for Health (Best on Theme), Machine Learning for the Developing World. 2020. human_in_the_loop_rmab_short.pdf
Mate A*, Killian J*, Xu H, Perrault A, Tambe M. Collapsing Bandits and their Application to Public Health Interventions. Advances in Neural and Information Processing Systems (NeurIPS) . 2020. Publisher's VersionAbstract
We propose and study Collapsing Bandits, a new restless multi-armed bandit (RMAB) setting in which each arm follows a binary-state Markovian process with a special structure: when an arm is played, the state is fully observed, thus “collapsing” any uncertainty, but when an arm is passive, no observation is made, thus allowing uncertainty to evolve. The goal is to keep as many arms in the “good” state as possible by planning a limited budget of actions per round. Such Collapsing Bandits are natural models for many healthcare domains in which health workers must simultaneously monitor patients and deliver interventions in a way that maximizes the health of their patient cohort. Our main contributions are as follows: (i) Building on the Whittle index technique for RMABs, we derive conditions under which the Collapsing Bandits problem is indexable. Our derivation hinges on novel conditions that characterize when the optimal policies may take the form of either “forward” or “reverse” threshold policies. (ii) We exploit the optimality of threshold policies to build fast algorithms for computing the Whittle index, including a closed form. (iii) We evaluate our algorithm on several data distributions including data from a real-world healthcare task in which a worker must monitor and deliver interventions to maximize their patients’ adherence to tuberculosis medication. Our algorithm achieves a 3-order-of-magnitude speedup compared to state-of-the-art RMAB techniques, while achieving similar performance.
collapsing_bandits_full_paper_camready.pdf
Wang K, Wilder B, Perrault A, Tambe M. Automatically Learning Compact Quality-aware Surrogates for Optimization Problems, in NeurIPS (Spotlight). Vancouver, Canada ; 2020.Abstract
Solving optimization problems with unknown parameters often requires learning a predictive model to predict the values of the unknown parameters and then solvingthe problem using these values. Recent work has shown that including the optimization problem as a layer in the model training pipeline results in predictions of the unobserved parameters that lead to higher decision quality. Unfortunately, this process comes at a large computational cost because the optimization problem must be solved and differentiated through in each training iteration; furthermore, it may also sometimes fail to improve solution quality due to non-smoothness issues that arise when training through a complex optimization layer. To address these shortcomings, we learn a low-dimensional surrogate model of a large optimization problem by representing the feasible space in terms of meta-variables, each of which is a linear combination of the original variables. By training a low-dimensional surrogate model end-to-end, and jointly with the predictive model, we achieve: i) a large reduction in training and inference time; and ii) improved per-formance by focusing attention on the more important variables in the optimization and learning in a smoother space. Empirically, we demonstrate these improvements on a non-convex adversary modeling task, a submodular recommendation task and a convex portfolio optimization task.
automatically-learning-compact-quality-aware-surrogates-for-optimization-problems-paper.pdf
Perrault A, Fang F, Sinha A, Tambe M. AI for Social Impact: Learning and Planning in the Data-to-Deployment Pipeline. AI Magazine. 2020.Abstract
With the maturing of AI and multiagent systems research, we have a tremendous
opportunity to direct these advances towards addressing complex societal problems. In pursuit of this goal of AI for Social Impact, we as AI researchers must go beyond improvements in computational methodology; it is important to step out in the field to demonstrate social impact. To this end, we focus on the problems of public safety and security, wildlife conservation, and public health in low-resource communities, and present research advances in multiagent systems to address one key cross-cutting challenge: how to effectively deploy our limited intervention resources in these problem domains. We present case studies from our deployments around the world as well as lessons learned that we hope are of use to researchers who are interested in AI for Social Impact. In pushing this research agenda, we believe AI can indeed play an important
role in fighting social injustice and improving society.
2001.00088.pdf

Pages