Generalized fitted Q-iteration with clustered data

Hu, Liyuan, Wang, Jitao, Wu, Zhenke and Shi, Chengchun ORCID: 0000-0001-7773-2099 (2025) Generalized fitted Q-iteration with clustered data. Stat, 14 (4). ISSN 2049-1573

Text (Stat - 2025 - Hu - Generalized Fitted Q‐Iteration With Clustered Data) - Published Version Available under License Creative Commons Attribution. Download (3MB)

• Scopus publication

Abstract

This paper focuses on reinforcement learning (RL) with clustered data, which is commonly encountered in healthcare applications. We propose a generalized fitted Q‐iteration (FQI) algorithm that incorporates generalized estimating equations into policy learning to handle the intra‐cluster correlations. Theoretically, we demonstrate (i) the optimalities of our Q‐function and policy estimators when the correlation structure is correctly specified and (ii) their consistencies when the structure is mis‐specified. Empirically, through simulations and analyses of a mobile health dataset, we find the proposed generalized FQI achieves, on average, a half reduction in regret compared to the standard FQI.

Item Type:	Article
Additional Information:	© 2025 The Author(s)
Divisions:	Statistics
Subjects:	Q Science > QA Mathematics
Date Deposited:	08 Oct 2025 09:48
Last Modified:	25 Nov 2025 11:45
URI:	http://eprints.lse.ac.uk/id/eprint/129713

Actions (login required)

View Item