MULTI-LAYER LEARNING NETWORKS,
Abstract
In multi-layer learning networks composed of adaptable linear threshold elements interconnected so as to produce a single output, inputs are given in sequence accompanied by an error signal when the network's output is wrong. The main problem is to devise a means for the network to choose which of its elements to alter so as to correct an error and yet do the least possible amount of damage to previously learned behavior. Five general principles are given to aid in this: 1)to minimize the number of elements altered; 2) to prefer elements with small sums over large ones; 3) to allow the network to participate in determining the effectiveness of tentative changes to see if they should be made permanent; 4) to allow each element to make, as nearly as possible, an autonomous decision as to whether to alter its weights; 5) and to maintain an excess capacity of elements and weights above the minimum needed for fixed generation of the desired logical functions. Two specific models are given which are in accord with these principles. One restricts its interconnecting weights to positive values and uses a single common bias term in each element's sum to perturb the elements' states. The other model removes the restriction on the weights but must use a more complicated perturbation involving some random action by elements. (Author)
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 01, 1964
- Accession Number
- AD0448896
Entities
People
- Roger A. Stafford