Scoring Situational Judgment Tests Using Profile Similarity Metrics

Abstract

This paper describes the application of profile similarity metrics to score Situational Judgment Tests (SJTs) that utilize rating scales to register examinee responses. The paper presents and discusses mathematical analyses that decompose distance-based measures into component indices based on correlation, dispersion and elevation metrics. The mathematical analyses demonstrate that distance measures represent a mixture of variance that can be associated with these separate components. Comparing the validities of distance and component indices using Leader Knowledge Test (LKT) data supports conclusions that the use of component indices (i.e., correlation, dispersion and elevation scores) improves the validity of SJTs that utilize rating scales.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jul 01, 2010
Accession Number
ADA530091

Entities

People

  • Dan Putka
  • Joseph Psotka
  • Peter J. Legree
  • Robert Kilcullen
  • Ryan N. Ginter

Organizations

  • U.S. Army Research Institute for the Behavioral and Social Sciences

Tags

Communities of Interest

  • Human Systems

DTIC Thesaurus Topics

  • Applied Psychology
  • Computations
  • Data Science
  • Databases
  • Dispersions
  • Elevation
  • Information Science
  • Judgment
  • Mathematical Analysis
  • Mathematics
  • Military Research
  • Psychological Phenomena And Processes
  • Psychology
  • Regression Analysis
  • Social Psychology
  • Social Sciences
  • Statistical Analysis

Fields of Study

  • Computer science
  • Psychology

Readers

  • Computational Modeling and Simulation
  • Psychometric Testing or Psychological Assessment.