Behaviourally cloning river raid agents

We investigate the feasibility and difficulties of using behavioural cloning to obtain player models using the 1982 video game River Raid. We attempt to clone both virtual game-playing agents (a fixed (non-improving) reinforcement learning agent and a random agent sampling actions uniformly) as well as an actual human agent. The behavioural clones' performance is evaluated on the micro-level through comparison of the state-conditioned and unconditional action distributions, and on the macro-level by comparing the (cloned) agents' survival time and score per episode. Using our methodology, cloning virtual agents seems feasible to varying extents, even with somewhat limited amounts of data. However, our method fails to create reliable behavioural clones of human players. We conclude with a discussion of some of the more important reasons that might cause this: a lack of training data, the problem of covariate shift, and improving and inconsistent play-style over time.

Publication year:2021

Accessibility:Open