Piecewise-Affine Classifiers In Support Vector Machines
Abstract
The Support Vector Machine (SVM) model has been a topic of study for over twenty years, and novel approaches to the classification problem using SVM continue to be established. In this work, we develop anew, nonlinear version of SVM based on a piecewise-affine classifier. This class of classifiers constitutes a tractable class beyond the affine functions that enables approximation of nonparametric SVM in high dimensions. We solve the resulting Piecewise-Affine SVM (PA-SVM) model using the Difference-of-Convex Algorithm (DCA) and a stochastic gradient descent (SGD) algorithm. The PA-SVM model is nonconvex, and the algorithms generally only provide locally optimal solutions. Still, they provide for a robust, capable classifier. Results show that by using DCA, the PA-SVM model can significantly reduce training misclassifications relative to the common Affine SVM (A-SVM) model by as much as 92%. Additionally, we show that test set errors can be reduced by as much as 67% compared to A-SVM. We find that solutions are more affected by the number of pieces employed rather than by regularization penalties. These results come from applying the PA-SVM model to three real-world data sets whose total features range from 16 to 41 and whose total observations range from 194 to 1,553.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jun 01, 2019
- Accession Number
- AD1080344
Entities
People
- Matthew T. Miller
Organizations
- Naval Postgraduate School