Hyper-Parameter Optimization of a Convolutional Neural Network
Abstract
In the world of machine learning, neural networks have become a powerful pattern recognition technique that gives a user the ability to interpret high-dimensional data where as conventional methods, such as logistic regression, would fail. There exists different types of neural networks, each containing its own set of hyper-parameters, that are dependent on the type of analysis required, but the focus of this paper will be on the hyper-parameters of convolutional neural networks. Convolutional neural networks are commonly used for classifications of visual imagery. For example, if you were to build a network for the purpose of predicting a specific animal, it would hopefully output, with high fidelity, the correct classification of a new animal introduced to the model. Traditionally, hyper-parameters were never optimized because it required a lot of computational power and time. If hyper-parameters were adjusted, analysts would manually change a few hyper-parameters, re-run the model, and hopefully get a better classification accuracy. But because of the advancements in technology, hyper-parameter tuning can now be done through complex and powerful optimization algorithms to improve the model. This paper implements and compares three different optimization techniques: random search, Bayesian Optimization with Gaussian Process, and tree of parzenestimator approach. The best performing technique is then improved through the Kiefer-Wolfowitz approximation
Document Details
- Document Type
- Technical Report
- Publication Date
- Mar 01, 2019
- Accession Number
- AD1077380
Entities
People
- Steven H. Chon
Organizations
- Air Force Institute of Technology