Conformal off-policy prediction

Zhang, Yingying, Shi, Chengchun ORCID: 0000-0001-7773-2099 and Luo, Shikai (2023) Conformal off-policy prediction. Proceedings of Machine Learning Research, 206. pp. 2751-2768. ISSN 2640-3498

Text (Zhang et al. 2023) - Accepted Version Download (505kB)

• Author
• Publisher
• Scopus publication

Abstract

Off-policy evaluation is critical in a number of applications where new policies need to be evaluated offline before online deployment. Most existing methods focus on the expected return, define the target parameter through averaging and provide a point estimator only. In this paper, we develop a novel procedure to produce reliable interval estimators for a target policy’s return starting from any initial state. Our proposal accounts for the variability of the return around its expectation, focuses on the individual effect and offers valid uncertainty quantification. Our main idea lies in designing a pseudo policy that generates subsamples as if they were sampled from the target policy so that existing conformal prediction algorithms are applicable to prediction interval construction. Our methods are justified by theories, synthetic data and real data from short-video platforms.

Item Type:	Article
Official URL:	https://proceedings.mlr.press/v206/zhang23c.html
Additional Information:	© 2023 The Author.
Divisions:	Statistics
Subjects:	H Social Sciences > HA Statistics
Date Deposited:	22 Feb 2023 14:45
Last Modified:	18 Nov 2024 17:06
URI:	http://eprints.lse.ac.uk/id/eprint/118250

Actions (login required)

View Item