First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method

• Roberto Battiti

  Dipartimento di Matematica, Università di Trento, 38050 Povo (Trento), Italy

抄録

On-line first-order backpropagation is sufficiently fast and effective for many large-scale classification problems but for very high precision mappings, batch processing may be the method of choice. This paper reviews first- and second-order optimization methods for learning in feedforward neural networks. The viewpoint is that of optimization: many methods can be cast in the language of optimization techniques, allowing the transfer to neural nets of detailed results about computational complexity and safety procedures to ensure convergence and to avoid numerical problems. The review is not intended to deliver detailed prescriptions for the most appropriate methods in specific applications, but to illustrate the main characteristics of the different methods and their mutual relations.

収録刊行物

• Neural Computation

  Neural Computation 4 (2), 141-166, 1992-03

  MIT Press - Journals

キーワード

• Cognitive Neuroscience
• Arts and Humanities (miscellaneous)

詳細情報

• CRID: 1362262945269092736
• NII論文ID: 30035678407
• NII書誌ID: AA10724454
• DOI: 10.1162/neco.1992.4.2.141
• ISSN: 1530888X
• ISSN: 08997667
• Web Site: https://www.mitpressjournals.org/doi/pdf/10.1162/neco.1992.4.2.141
• データソース種別:
  • Crossref
  • CiNii Articles