Using Latent Semantic Analysis to Score Short Answer Constructed Responses: Automated Scoring of the Consequences Test
Abstract
Automated scoring based on Latent Semantic Analysis (LSA) has been successfully used to score essays and constrained short answer responses. Scoring tests that capture open-ended, short answer responses poses some challenges for machine learning approaches. We used LSA techniques to score short answer responses to the Consequences Test, a measure of creativity and divergent thinking that encourages a wide range of potential responses. Analyses demonstrated that the LSA scores were highly correlated with conventional Consequence Test scores, reaching a correlation of .94 with human raters and were moderately correlated with performance criteria. This approach to scoring short answer constructed responses solves many practical problems including the time for humans to rate open-ended responses and the difficulty in achieving reliable scoring.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Jul 09, 2019
- Source ID
- 10.1177/0013164419860575
Entities
People
- J. E. S. Parker
- Noelle LaVoie
- Peter J. Legree
- Robert N. Kilcullen
- Sharon Ardison
Organizations
- U.S. Army Research Institute for the Behavioral and Social Sciences