Who's behind these predictions? Reconciling transparency and privacy in machine learning
Abstract
The PI completed this project investigating the tradeoff between external query and response access to machine learning (ML) models while preserving security and confidentiality. The objective was to use queries and responses from an unknown ML model to discover the model family (neural net, decision tree, support vector machine, generalized linear model, etc.) of the unknown model. This represents a security risk as attackers can leverage known aspects of a ML model to inject adversarial examples during training and/or model deployment. The main idea was to train surrogate models based on the inputs and outputs of the unknown model. The PI then employed a dissimilarity measure (as specified in the attached final report) to determine which surrogate model best matched input-output data from the unknown model. The results show such an approach has potential as they were able to correctly identify model families much better than random chance, but there remains much room for further investigation. More specifics may be found in the attached final report and the technical papers reference therein.
Document Details
- Document Type
- Technical Report
- Publication Date
- Feb 14, 2019
- Accession Number
- AD1077647
Entities
People
- María José Quintana Hernández
Organizations
- Technical University of Valencia