A Size Reduction Method of A Hierarchical Neural Network Using A Principal Component Analysis

DOI Web Site Web Site 1 Citations 9 References

Bibliographic Information

Other Title
  • 主成分分析を用いた階層型ニューラルネットの縮約法
  • シュセイブン ブンセキ オ モチイタ カイソウガタ ニューラル ネット ノ シ

Search this article

Abstract

The size of a hierarchical neural network is desirable to be compact according to each application problem, because the network size is closely related to not only the computer memory but also the generalization ability. If the network is too large, it will be over-fitting the training data and bad generalization will occur. In an opposite situation, training for its own sake will fail due to under-fitting.<br>This paper deals with a method to reduce the network size and to obtain a compact network equivalent to the original one. Principal Component Analysis (PCA) performs a central role in a size reduction. The reduction can be implemented in the follwing way: (a) At first, train a largish network in order to avoid under-fitting. (b) after training, the output matrix of hidden units is changed into the component score matrix by PCA. (c) determine n components corresponding to the n largest eigenvalues according to PCA so that the cumulative proportion may be almost 100%. Then the number of hidden units can be reduced up to n. (d) determine weights and thresholds by solving simple linear equations so that the reduced network may be equivalent to the original one.<br>The proposed method is applied to a simple pattern recognition problem and the result shows the effectiveness of the proposed method.

Journal

Citations (1)*help

See more

References(9)*help

See more

Keywords

Details 詳細情報について

Report a problem

Back to top