Inference in Adaptive Regression via the Kac-Rice Formula

Abstract

We derive an exact p-value for testing a global null hypothesis in a general adaptive regression setting. Our approach uses the Kac-Rice formula (as described in Adler & Taylor 2007) applied to the problem of maximizing a Gaussian process. The resulting test statistic has a known distribution in finite samples, assuming Gaussian errors. We examine this test statistic in the case of the lasso, group lasso, principal components and matrix completion problems. For the lasso problem, our test relates closely to the recently proposed covariance test of Lockhart et al. (2013). Our approach also yields exact selective inference for the mean parameter at the global maximizer of the process.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
May 15, 2014
Accession Number
ADA610023

Entities

People

  • Jonathan E. Taylor
  • Joshua Loftus
  • Ryan J. Tibshirani

Organizations

  • Stanford University

Tags

DTIC Thesaurus Topics

  • Computations
  • Convex Sets
  • Covariance
  • Data Science
  • Data Sets
  • Distribution Functions
  • Gaussian Distributions
  • Gaussian Processes
  • Information Science
  • Normal Distribution
  • Order Statistics
  • Probability
  • Random Variables
  • Simulations
  • Statistical Algorithms
  • Statistical Analysis
  • Statistics

Fields of Study

  • Computer science

Readers

  • Operations Research
  • Regression Analysis.

Technology Areas

  • AI & ML
  • AI & ML - Bayesian Inference
  • AI & ML - Machine Learning Algorithms