Log-sum-exponential estimator for off-policy evaluation and learning

Behnamnia, Armin, Aminian, Gholamali, Aghaei, Alireza, Shi, Chengchun ORCID: 0000-0001-7773-2099, Tan, Vincent Y. F. and R. Rabiee, Hamid (2025) Log-sum-exponential estimator for off-policy evaluation and learning. In: Proceedings of the 42nd International Conference on Machine Learning. ACM Press. (In Press)

Text (_ICML_2025___LSE_estimator) - Accepted Version Download (1MB)

Abstract

Off-policy learning and evaluation scenarios leverage logged bandit feedback datasets, which contain context, action, propensity score, and feedback for each data point. These scenarios face significant challenges due to high variance and poor performance with low-quality propensity scores and heavy-tailed reward distributions. We address these issues by introducing a novel estimator based on the log-sum-exponential (LSE) operator, which outperforms traditional inverse propensity score estimators. our LSE estimator demonstrates variance reduction and robustness under heavytailed conditions. For off-policy evaluation, we derive upper bounds on the estimator’s bias and variance. In the off-policy learning scenario, we establish bounds on the regret—the performance gap between our LSE estimator and the optimal policy—assuming bounded (1 + ϵ)-th moment of weighted reward. Notably, we achieve a convergence rate of O(n−ϵ/(1+ϵ)), where n is the number of training samples for the regret bounds and ϵ ∈ [0, 1]. Theoretical analysis is complemented by comprehensive empirical evaluations in both off-policy learning and evaluation scenarios, confirming the practical advantages of our approach. The code for our estimator is available at the following Link.

Item Type:	Book Section
Additional Information:	© 2025 The Author(s)
Divisions:	Statistics
Date Deposited:	27 Jun 2025 14:18
Last Modified:	27 Jun 2025 15:45
URI:	http://eprints.lse.ac.uk/id/eprint/128271

Actions (login required)

View Item