Algebraic Analysis for Nonidentifiable Learning Machines

• Sumio Watanabe

  P&I Laboratory, Tokyo Institute of Technology, Yokohama, 226-8503 Japan

抄録

This article clarifies the relation between the learning curve and the algebraic geometrical structure of a nonidentifiable learning machine such as a multilayer neural network whose true parameter set is an analytic set with singular points. By using a concept in algebraic analysis, we rigorously prove that the Bayesian stochastic complexity or the free energy is asymptotically equal to λ 1 logn − (m 1 − 1) loglogn + constant, where n is the number of training samples and λ 1 and m 1 are the rational number and the natural number, which are determined as the birational invariant values of the singularities in the parameter space. Also we show an algorithm to calculate λ 1 and m 1 based on the resolution of singularities in algebraic geometry. In regular statistical models, 2λ 1 is equal to the number of parameters and m 1 = 1, whereas in nonregular models, such as multilayer networks, 2λ 1 is not larger than the number of parameters and m 1 ≥ 1. Since the increase of the stochastic complexity is equal to the learning curve or the generalization error, the nonidentifiable learning machines are better models than the regular ones if Bayesian ensemble learning is applied.

収録刊行物

• Neural Computation

  Neural Computation 13 (4), 899-933, 2001-04-01

  MIT Press - Journals

キーワード

• Cognitive Neuroscience
• Arts and Humanities (miscellaneous)

詳細情報

• CRID: 1362825893712240896
• NII論文ID: 80012334991
• DOI: 10.1162/089976601300014402
• ISSN: 1530888X
• ISSN: 08997667
• Web Site: https://www.mitpressjournals.org/doi/pdf/10.1162/089976601300014402
• データソース種別:
  • Crossref
  • CiNii Articles