Automated Protocol Analysis: Tools and Methodology
Abstract
The TextLab Research Group, over the past seven years, has developed a collection of tools and techniques for recording users' interactions with computer systems in machine-readable form and for automatically analyzing and displaying those data. This report catalogs those tools and discusses their methodological context and implications. Tools discussed include the following: tracking users behaviors and producing a machine recorded protocol at the level of action--data an order of magnitude larger than keystrokes; replaying users' sessions from protocol data; modeling users' strategies using formal cognitive grammars; analyzing user sessions by parsing them with the grammars; and displaying results in visual form--both static and animated--to facilitate interpretation and understanding by researchers. These tools are placed in a methodological context by reviewing issues associated with concurrent think- aloud, keystroke, and video protocols; and other computer systems are reviewed that support these forms of protocol data. The discussion concludes by noting the increased importance of data management required by automated methods and our thoughts for a comprehensive environment, built around a protocol management facility, that would integrate the tools discussed and support development of new ones.
Document Details
- Document Type
- Technical Report
- Publication Date
- Aug 01, 1991
- Accession Number
- ADA242040
Entities
People
- Dana K. Smith
- Eileen Kupstas
- John B. Smith
Organizations
- University of North Carolina at Chapel Hill