An adaptive approach for infinitely many-armed bandits under generalized rotting constraints

Kim, Jung-Hun, Vojnovic, Milan ORCID: 0000-0003-1382-022X and Yun, Se-Young (2024) An adaptive approach for infinitely many-armed bandits under generalized rotting constraints. In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. and Zhang, C., (eds.) Advances in Neural Information Processing Systems 37 (NeurIPS 2024. Neural Information Processing Systems Foundation. ISBN 9798331314385

Text (NeurIPS-2024-an-adaptive-approach-for-infinitely-many-armed-bandits-under-generalized-rotting-constraints-Paper-Conference) - Published Version Download (701kB)

Abstract

In this study, we consider the infinitely many-armed bandit problems in a rested rotting setting, where the mean reward of an arm may decrease with each pull, while otherwise, it remains unchanged. We explore two scenarios regarding the rotting of rewards: one in which the cumulative amount of rotting is bounded by VT, referred to as the slow-rotting case, and the other in which the cumulative number of rotting instances is bounded by ST, referred to as the abrupt-rotting case. To address the challenge posed by rotting rewards, we introduce an algorithm that utilizes UCB with an adaptive sliding window, designed to manage the bias and variance trade-off arising due to rotting rewards. Our proposed algorithm achieves tight regret bounds for both slow and abrupt rotting scenarios. Lastly, we demonstrate the performance of our algorithm using numerical experiments.

Item Type:	Book Section
Official URL:	https://papers.nips.cc/paper_files/paper/2024/hash...
Additional Information:	© 2025 The Author(s)
Divisions:	Statistics
Subjects:	Q Science > QA Mathematics
Date Deposited:	26 Jun 2025 12:00
Last Modified:	26 Jun 2025 15:15
URI:	http://eprints.lse.ac.uk/id/eprint/128572

Actions (login required)

View Item