Softening the Prediction Surface of Decision Tree Regressors

Abstract

While decision trees function as base-learners for many machine learning methods and exhibit several useful properties, the compromise between prediction accuracy and generalization limits their utility in non-ensemble applications. We propose a new method to improve the predictive performance of pre-trained decision tree regressors. Using the tree structure and metadata, we derive a set of decision threshold-based weights that modify the leaf prediction values. The weighted values are then aggregated into a final softened prediction which more accurately represents the true target distribution. We demonstrate the approach on a variety of benchmark data sets and observe a mean improvement of 11 percent over the baseline decision tree R2 values. We further explore the parameters of the approach and characterize their effects.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jun 10, 2024
Accession Number
AD1229938

Entities

People

  • Andrew B. Sabater
  • Benjamin A. Michlin
  • Jamal T. Rorie
  • Joshua A. Duclos

Organizations

  • Naval Information Warfare Center Pacific

Tags

Fields of Study

  • Computer science

Readers

  • Forest Ecology
  • Neural Network Machine Learning.

Technology Areas

  • AI & ML
  • AI & ML - Bayesian Inference
  • AI & ML - Neural Networks