Madaline Rule II: A New Method for Training Networks of Adalines
Abstract
Madaline Rule II, MRII, is a supervised learning algorithm for training layered feed-forward networks of Adalines. An Adaline is a basic neuron-like processing element which forms a binary output determined by a weighted sum of its inputs. The algorithm is based on a minimal disturbance principle. This principle states that changes made to the network's weights in order to correct erroneous responses for a particular input pattern should disturb the responses to other input patterns as little as possible. MRII uses a sequence of trial adaptions to correct output errors during training. The trials that require the smallest weight changes are performed first. A method to insure that all neural elements share responsibility for forming the global solution is introduced and called usage. The algorithm exhibits interesting generalization properties. Generalization is the network's ability to make correct responses to inputs not included in the training set. Networks that contain more Adalines than necessary to solve a given training problem exhibit good generalization when trained by MRII on a sufficiently large training set. The algorithm does not always converge to known solutions. A training failure sometimes occurs when a fraction of the output units achieves an early solution. Convergence is blocked because the changes of the hidden pattern set that are needed for a global solution, are incompatible with the partial solution already formed. (KR)
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 01, 1989
- Accession Number
- ADA217377
Entities
People
- Rodney G. Winter
Organizations
- Air Force Institute of Technology