Development of a Framework for Combining Competency Assessments
Abstract
The Mapping Assessments to Competencies (MAC) Process evaluates existing assessments to determine if they can be used to evaluate for broader competencies. An important consideration in gauging whether assessments developed for one purpose may be used for another purpose is to consider the ways the assessments will be used. In developing the MAC Process, three possible uses were considered, each requiring a higher level of rigor, validation, and evidence to justify their use: developmental, diagnostic, and predictive. The current effort expands on the MAC Process by developing a framework to consider when combining the results of assessments to create a new competency measure. This report provides background for such a framework based on a review of scientific literature and policy documents on Army talent management and assessments, competency assessments, and methods to combine assessments. This paper documents how techniques such as item response theory, network analysis, and machine learning algorithms can be used to evaluate items, scores, and evidence of appropriate use (i.e., reliability, validity, and fairness). This paper is intended for a technical audience to use as they explore creating new competency measures for a variety of uses.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 01, 2024
- Accession Number
- AD1219743
Entities
People
- April Cho
- Heather Wolters
- Jacqueline K. Deuling
- Marcus A. Fagan
- Neil Carey
- Patricia Kannapel
- Susan Starcovic
- William S. Weyhrauch
Organizations
- Center for Naval Analyses