TDD-ML: Test-Driven Development for ML Systems
Abstract
What we aim to do in this project is develop a methodology and a tool for analysis of test data sets used in ML model development and acquisition processes. We will then integrate into a MLOps pipeline to support maintenance and expansion of the test sets in a production environment. Specifically, we will develop a methodology and tool were:1. Representative ML model developers, when using the tool in conjunction with software engineering TDD practices, will better specify the ML model, resulting in an 30%increase in model accuracy, in line with results of Afan, et al. 2022.2. Use of the tool during ML model evaluation and acquisition will lead to the selection of an ML model that will perform at least 20% better in production, in line with the findings of Oakden-Rayner, et al. 2020.3. Use of the tool, when integrated into a DoD MLOps pipeline, during ML model evaluation and acquisition will lead to the selection of an ML model that will not experience as large of a performance drop off when in production when compared to baseline.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 06, 2022
- Accession Number
- AD1189826
Entities
People
- Rachel Brower-sinning
Organizations
- Carnegie Mellon University