Synaptic Annealing: Anisotropic Simulated Annealing and its Application to Neural Network Synaptic Weight Selection
Abstract
Neural networks are one of the most successful classes of machine learning algorithm, and have been applied to solve problems previously considered to be the exclusive domain of human intellect. Several methods for constructing neural networks exists. This research explores the effectiveness of a feed-forward neural network weight selection procedure called synaptic annealing. Synaptic annealing is the application of the simulated annealing algorithm to the problem of selecting synaptic weights. A novel extension of the simulated annealing algorithm, called anisotropicity, is defined and developed. The cross-validated performance of each synaptic annealing algorithm is evaluated, and compared to back-propagation when trained on several typical machine learning problems. Synaptic annealing is found to be more effective than back-propagation training on classification and regression data sets. These improvements in feed-forward neural network training performance indicate that synaptic annealing may be a viable alternative to back-propagation in many applications of neural networks.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jun 16, 2016
- Accession Number
- AD1054216
Entities
People
- Justin R. Fletcher
Organizations
- Air Force Institute of Technology