Collaborative Problem Solving (CPS) Skill: Estimating an IndividualÕs Contribution to Small Group Performance (Domain 1. Personnel Testing and Performance)
Abstract
The purpose of this proposed 3-year research program is to develop and evaluate approaches to measuring individual contributions in a set of collaborative tasks, which we refer to as problem solving (CPS) skill. CPS skill is defined here broadly, with tasks including (a) problem solving, (b) decision-making/hidden profile, and (c) negotiation. We explore a set of issues related to developing and evaluating individual and team performance on a set of CPS tasks. The main questions are: 1. Can we measure an individualÕs contribution to collective performance? a. based solely on collective performance outcomes. b. based on contribution to group process variables that are related to collective outcomes. 2. Are there individual knowledge, skill, ability, and other factor (KSAO) determinants of an individualÕs contribution? 3. How generalizable are individual contributions to collective outcomes? a. across task variants within the same type. b. across task types. 4. Are there differences in individual contributions to collective performance across 2-vs. 4-person teams? 5. Are there differences in individual contributions to collective performance in online vs. face-to-face contexts? We plan to administer tasks to both in both face-to-face and online contexts. Online studies will primarily be conducted in Years 1 and 2. They will involve 3,000 participants who will be distributed over 44 design conditions in 500 dyads (n = 1000) and 500 tetrads (n = 2000). Face-to-face studies will be conducted in Year 3. They will involve 400 participants in 100 tetrads in an on-campus setting. Face-to-face participants will also participate in an online context to enable comparisons. Participants will be drawn from an ETS-maintained panel of approximately 9,000 members who already have scores from numerous personality (e.g., TAPAS-like scores) and cognitive (e.g., general cognitive ability) tasks, which we can capitalize on for this research. Online versions of tasks will be administered using ETSÕs Platform for Collaborative Assessment and Learning (EPCAL). The platform is available for use by ARI during and following research completion. The third year, face-to-face data collection will involve the use of sociometric badges and group work will be audio and video recorded and transcribed for analysis. We will use both multivariate analyses for scoring collective outcomes and determining individual contributions and natural language processing (NLP) and machine learning approaches for analyzing process data during group interactions.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Dec 04, 2019
- Source ID
- W911NF1910106
Entities
People
- Patrick Kyllonen
Organizations
- Army Contracting Command
- Educational Testing Service
- United States Army