Psychometric Considerations and a General Scoring Strategy for Assessments of Collaborative Problem Solving

Abstract

Collaborative problem solving (CPS) is an important 21st-century skill that is crucial for both career and academic success. However, developing a large-scale and standardized assessment of CPS that can be administered on a regular basis is very challenging. In this report, we introduce a set of psychometric considerations and a general scoring strategy around assessing CPS, summarized based on the results of the extensive empirical studies we conducted at Educational Testing Service (ETS) over the past 6 years. Using the ETS Collaborative Science Assessment Prototype as an example, we show how these psychometric considerations have been incorporated into the development of the assessment prototype and how the scoring strategy has been implemented.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Dec 01, 2019
Accession Number
AD1218673

Entities

People

  • Alina A. Von Davier
  • Jiangang Hao
  • Lei Liu
  • Michael Flor
  • Patrick Kyllonen

Organizations

  • Educational Testing Service

Tags

DTIC Thesaurus Topics

  • Artificial Intelligence
  • Computational Linguistics
  • Computational Science
  • Data Analysis
  • Data Mining
  • Education
  • Information Science
  • Machine Learning
  • Military Research
  • Models
  • Natural Language Processing
  • New York
  • Social Sciences
  • Statistical Analysis
  • Students
  • Supervised Machine Learning
  • United States

Readers

  • Instructional Design and Training Evaluation.
  • Systems Analysis and Design