A UNIFYING MATHEMATICAL THEORY FOR TRAINING LEARNING NETS.
Abstract
This report analyzes the deterministic approaches which have been applied to the description of neural net configurations and their training algorithms which employ a single layer of trainable gain elements and partition the input space by hyperplanes. The nets are described by n-dimensional geometric vector methods. A general algorithm is developed based on gradient or steepest-descent methods for optimizing a system given a quadratic index of performance. Reductions of this algorithm to the two basic classes of (a) error correcting and (b) forced learning algorithms as special cases are considered. Effects are discussed of component imperfections such as saturation, nonlinear adaption rates, hysteresis, and component failure. Examination of advantages and disadvantages of the various algorithms indicate that the error correcting algorithm and its modified forms have the following areas of superiority: (1) capability in separating separable classes, (2) ability to form least-mean-square error for non-separable classes, (3) minimum magnitude gain vector, and (4) relative insensitivity to component imperfections. The forced learning algorithms respond to the relative frequency of the input classes. Where this sensitivity is important, the forced learning algorithm may be superior. (Author)
Document Details
- Document Type
- Technical Report
- Publication Date
- Oct 29, 1965
- Accession Number
- AD0627908
Entities
People
- F. D. Powell
- J. G. Goerner
- L. A. Gerhardt
Organizations
- Bell Aircraft Corporation