A Theoretical Analysis of On-Line Learning Using Correlated Examples
-
- SEKI Chihiro
- Advanced course of Electrical and Electronic Engineering, Kobe City College of Technology Matsushita Excel Technology Co., Ltd.
-
- SAKURAI Shingo
- Advanced course of Electrical and Electronic Engineering, Kobe City College of Technology Sysmex Corporation
-
- MATSUNO Masafumi
- Fujitsu FSAS Inc.
-
- MIYOSHI Seiji
- Department of Electronic Engineering, Kobe City College of Technology Department of Electrical and Electronic Engineering, Faculty of Engineering Science, Kansai University The Institute of Electronics, Information and Communication Engineers
Search this article
Abstract
In this paper we analytically investigate the generalization performance of learning using correlated inputs in the framework of online learning with a statistical mechanical method. We consider a model composed of linear perceptrons with Gaussian noise. First, we analyze the case of the gradient method. We analytically clarify that the larger the correlation among inputs is or the larger the number of inputs is, the stricter the condition the learning rate should satisfy is, and the slower the learning speed is. Second, we treat the block orthogonal projection learning as an alternative learning rule and derive the theory. In a noiseless case, the learning speed does not depend on the correlation and is proportional to the number of inputs used in an update. The learning speed is identical to that of the gradient method with uncorrelated inputs. On the other hand, when there is noise, the larger the correlation among inputs is, the slower the learning speed is and the larger the residual generalization error is.
Journal
-
- IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
-
IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E91-A (9), 2663-2670, 2008
The Institute of Electronics, Information and Communication Engineers
- Tweet
Keywords
Details 詳細情報について
-
- CRID
- 1390282681287647104
-
- NII Article ID
- 10026852143
-
- NII Book ID
- AA10826239
-
- ISSN
- 17451337
- 09168508
-
- Text Lang
- en
-
- Data Source
-
- JaLC
- Crossref
- CiNii Articles
-
- Abstract License Flag
- Disallowed