Priors, Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines.
Abstract
We had previously shown that regularization principles lead to approximation schemes, as Radial Basis Functions, which are equivalent to networks with one layer of hidden units, called Regularization Networks. In this paper we show that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models, Breiman's hinge functions and some forms of Projection Pursuit Regression. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In summary, different multilayer networks with one hidden layer, which we collectively call Generalized Regularization Networks, correspond to different classes of priors and associated smoothness functionals in a classical regularization principle. Three broad classes are Radial Basis Functions that generalize into Hyper Basis Functions, some tensor product splines, and additive splines that generalize into schemes of the type of ridge approximation, hinge functions and one-hidden-layer perceptrons. (AN)
Document Details
- Document Type
- Technical Report
- Publication Date
- Jun 01, 1993
- Accession Number
- ADA290097
Entities
People
- Federico Girosi
- Michael E Jones
- Tomaso Poggio
Organizations
- Massachusetts Institute of Technology