Cookies?
Library Header Image
LSE Research Online LSE Library Services

Does the Markov decision process fit the data: testing for the Markov property in sequential decision making

Shi, Chengchun, Wan, Runzhe, Song, Rui, Lu, Wenbin and Leng, Ling (2020) Does the Markov decision process fit the data: testing for the Markov property in sequential decision making. In: International Conference on Machine Learning, 2020-07-12 - 2020-07-18, Online. (In Press)

[img] Text (Shi_Does-the-Markov-Decision-Process-Fit-the-Data_2020) - Accepted Version
Download (1MB)

Abstract

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an important role for identifying the optimal policy in high-order Markov decision processes and partially observable MDPs. We apply our test to both synthetic datasets and a real data example from mobile health studies to illustrate its usefulness.

Item Type: Conference or Workshop Item (Paper)
Official URL: https://icml.cc/virtual/2020
Additional Information: © 2020
Divisions: Statistics
Subjects: Q Science > QA Mathematics
H Social Sciences > HA Statistics
Date Deposited: 03 Aug 2020 15:03
Last Modified: 15 Sep 2023 08:37
URI: http://eprints.lse.ac.uk/id/eprint/105852

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics