CONVERGENCE PROPERTIES OF AN ADAPTIVE ALGORITHM FOR LINEAR THRESHOLD ELEMENTS.
Abstract
Linear threshold element is the generic term for a device which forms the sum a sub 0 + a sub 1 x sub 1 + a sub 2 x sub 2 + ... + a sub d x sub d from an input vector (x sub 1, x sub 2, ... , x sub d) and yields one of two outputs depending on whether or not the sum is positive. A pattern classification machine may utilize a linear threshold element along with a controller which receives the one of the two values corresponding to correct classification of the input vector. The purpose of the controller is to modify the gain vector (a sub 0, a sub 1, ... , a sub d) so that the next input vector has a greater likelihood of being correctly classified by the threshold element. This likelihood depends on the value of the gain vector and an adaptive algorithm of the steepest descent variety can be used to attempt to adjust the gain vector to its optimal value as the machine is exposed to a stationary sequence of statistically independent input vectors. The components of these vectors are commonly two valued, and it has been shown that convergence of the expected value of the gain vector is dependent on the value of the adjustment parameter, the values of the components, and the distribution of the input vectors. It is shown that a bound on the adjustment parameter, simply related to the values of the input components, is sufficient to insure this convergence. The variance of the gain vector is derived under the assumptions of a uniform input sequence and oppositely signed components of equal magnitude and it is shown that a similar bound on the adjustment parameter implies convergence of the variance. The variance is graphed under representative conditions. (Author)
Document Details
- Document Type
- Technical Report
- Publication Date
- May 01, 1966
- Accession Number
- AD0489087
Entities
People
- Robert R. Pearson
Organizations
- Naval Postgraduate School