Multi-Task Approach to Reinforcement Learning for Factored-State Markov Decision Problems

• SIMM Jaak

  Tallinn University of Technology

• SUGIYAMA Masashi

  Tokyo Institute of Technology PRESTO, Japan Science and Technology Agency

• HACHIYA Hirotaka

  Tokyo Institute of Technology

Abstract

Reinforcement learning (RL) is a flexible framework for learning a decision rule in an unknown environment. However, a large number of samples are often required for finding a useful decision rule. To mitigate this problem, the concept of transfer learning has been employed to utilize knowledge obtained from similar RL tasks. However, most approaches developed so far are useful only in low-dimensional settings. In this paper, we propose a novel transfer learning idea that targets problems with high-dimensional states. Our idea is to transfer knowledge between state factors (e.g., interacting objects) within a single RL task. This allows the agent to learn the system dynamics of the target RL task with fewer data samples. The effectiveness of the proposed method is demonstrated through experiments.

Journal

• IEICE Transactions on Information and Systems

  IEICE Transactions on Information and Systems E95.D (10), 2426-2437, 2012

  The Institute of Electronics, Information and Communication Engineers

Keywords

• reinforcement learning
• multi-task learning
• transfer learning
• factored state models
• Markov decision process

Details

• CRID: 1390282679356468224
• NII Article ID: 10031142861
• NII Book ID: AA10826272
• DOI: 10.1587/transinf.e95.d.2426
• ISSN: 17451361
• ISSN: 09168532
• Web Site: https://www.jstage.jst.go.jp/article/transinf/E95.D/10/E95.D_2426/_pdf
• Text Lang: en
• Data Source:
  • JaLC
  • Crossref
  • CiNii Articles
• Abstract License Flag: Disallowed