A Simple 'Linearized' Learning Algorithm Which Outperforms Back-Propagation

Abstract

A class of algorithms is presented for training multilayer perceptrons using purely linear techniques. The methods are based upon linearizations of the network using error surface analysis, followed by a contemporary least squares estimation procedure. Specific algorithms are presented to estimate weights node-wise, layer-wise, and for estimating the entire set of network weights simultaneously. In several experimental studies, the node-wise method is superior to back-propagation and an alternative linearization method due to Azimi-Sadjadi et at. in terms of number of convergences and convergence rate. The layer and network-wise updating offer further improvement.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 1992
Accession Number
ADA249697

Entities

People

  • J. R. Deller Jr.
  • S. D. Hunt

Organizations

  • Michigan State University

Tags

DTIC Thesaurus Topics

  • Algorithms
  • Convergence
  • Decomposition
  • Electrical Engineering
  • Engineering
  • Equations
  • Iterations
  • Learning
  • Military Research
  • Neural Networks
  • Nonlinear Systems
  • Puerto Rico
  • Signal Processing
  • Simulations
  • Surface Analysis
  • Surfaces
  • Training

Readers

  • Calculus or Mathematical Analysis
  • Neural Network Machine Learning.