Sparse Coding and Dictionary Learning Based on the MDL Principle
Abstract
The power of sparse signal coding with learned dictionaries has been demonstrated in a variety of applications and fields, from signal processing to statistical inference and machine learning. However, the statistical properties of these models, such as underfitting or overfitting given sets of data, are still not well characterized in the literature. This work aims at filling this gap by means of the Minimum Description Length (MDL) principle -- a well established information-theoretic approach to statistical inference. The resulting framework derives a family of efficient sparse coding and modeling (dictionary learning) algorithms, which by virtue of the MDL principle are completely parameter free. Furthermore, such framework allows to incorporate additional prior information in the model, such as Markovian dependencies, in a natural way. We demonstrate the performance of the proposed framework with results for image denoising and classification tasks.
Document Details
- Document Type
- Technical Report
- Publication Date
- Oct 01, 2010
- Accession Number
- ADA540659
Entities
People
- Guillermo Sapiro
- Ignacio Ramirez
Organizations
- University of Minnesota