Profit Sharing Introducing the Judgement of Incomplete Perception

DOI Web Site Web Site 3 Citations 12 References
  • Saito Ken
    Graduate School of Engineering, Tokyo Metropolitan Institute of Techonology
  • Masuda Shiro
    Graduate School of Engineering, Tokyo Metropolitan Institute of Techonology

Bibliographic Information

Other Title
  • 不完全知覚判定法を導入したＰｒｏｆｉｔ　Ｓｈａｒｉｎｇ
  • フカンゼン チカク ハンテイホウ オ ドウニュウ シタ Profit Sharing

Search this article

Abstract

To apply reinforcement learning to difficult classes such as real-environment learning, we need to use a method robust to perceptual aliasing problem. The exploitation-oriented methods such as Profit Sharing can deal with the perceptual aliasing problem to a certain extent. However, when the agent needs to select different actions at the same sensory input, the learning efficiency worsens. To overcome the problem, several state partition methods using history information of state-action pairs are proposed. These methods try to convert a POMDP environment into an MDP environment, and thus they are sometimes very useful. However, their computation cost is very high especially in large state spaces. In contrast, memory-less approaches try to escape from the aliased states by outputting actions stochastically. However, these methods output actions stochastically even in unaliased states, and thus the learning efficiency is bad. If we desire to guarantee the rationality in POMDPs, it is efficient to output actions stochastically only in the aliased states and to output one action deterministically in the other unaliased states. Hence, to discriminate between aliased states and unaliased states, the utilization of χ² -goodness-of-fit test is proposed by Miyazaki et al. They point out that, in aliased states, the distributions of the state transitions by random search and a particular policy are different. This difference doesn't occur owing to non-deterministic actions. Hence, if the agent can collect enough samples to implement the test, the agent can distinguish between aliased states and unaliased states well. However, such a test needs a large amount of data, and it's a problem how the agent collects samples without worsening learning efficiency. If the agent uses random search in the course of learning, the learning efficiency worsens especially in unaliased states. Therefore, in this research, we propose a new method called Extended On-line Profit Sharing with Judgement (EOPSwJ) to detect important incomplete perception, which doesn't need large computation cost and numerous samples. We use two criterions for detecting important incomplete perceptions to attain a task. One is the rate of transitions to each state, and the other is the deterministic rate of actions. We confirm the availability of EOPSwJ using two simulations.

Journal

Citations (3)*help

See more

References(12)*help

See more

Related Projects

See more

Keywords

Details 詳細情報について

Report a problem

Back to top