Non-Parametric Model Drift Detection

Abstract

The IARPA seedling effort explored an automated framework for model maintenance. The effort calculated in an unsupervised fashion the difference between the dataset that was used to train the model and the new dataset on which the model is to be applied (this is done using a new tool called CorEx that automatically estimates structure in high dimensional data through correlation) . The experimentation took place on datasets made up of text documents. The difference between datasets used to estimate potential error (drop in accuracy) that the model would incur if applied on the new dataset. The tradeoff between time cost of retraining the model and potential error of applying the original model on the new dataset will used in making the decision on whether to retrain or not.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jul 01, 2016
Accession Number
AD1012527

Entities

People

  • Aram Galstyan

Organizations

  • Information Sciences Institute

Tags

Communities of Interest

  • Autonomy
  • Energy and Power Technologies
  • Human Systems
  • Materials and Manufacturing Processes

DTIC Thesaurus Topics

  • Accuracy
  • Air Force
  • Air Force Research Laboratories
  • Artificial Intelligence
  • Detection
  • Errors
  • Governments
  • Information Processing
  • Information Science
  • Information Systems
  • Machine Learning
  • Machine Translation
  • Neural Networks
  • Probability
  • Retraining
  • Test Sets
  • Training

Fields of Study

  • Computer science
  • Environmental science

Readers

  • Neural Network Machine Learning.
  • Team-Based Human-Centered Cognitive Task Decision Making and Information Performance.