Assessment and Scaffolding for Learners of Complex, Dynamic Domain Knowledge: With Application to Ship Handling
Abstract
Problem solving in many real-life situations is very different from solving typical classroom problems. Real-life problem solving often draws upon a complex network of knowledge to determine multiple steps toward achieving a specific goal. In many cases the problem is not fixed or static; instead the problem solver is confronted with a series of subproblems that factors outside his control change dynamically. The path actually traversed is determined by these external factors together with steps the problem solver takes. T"o reach a problem~s goal, knowledge must be deployed to choose steps that are appropriate in light of prevailing uncontrollable infl"uences.Assessing problem solvers~ command of the knowledge in question requires an appropriately matched theory of testing. Individual items of knowledge are rarely tested in isolation by this type of problem solving; most steps require combinations of interacti"ng knowledge. When an individual item is tested repeatedly, these tests commonly mix it in varying combinations of knowledge. A suit"able testing theory must explain how results from a series of tests of varying combinations of knowledge in extrinsically altered subproblems support conclusions about a problem solver~s degree of mastery of individual knowledge items and how much he labors under particular misconceptions.Our approach treats knowledge as a network of items that can have interdependencies or be independent o"f each other, and treats problems as having the goal of reaching a certain state from a given initial state while remaining within a" class of acceptable states at each intermediate step. State-to-state transitions are determined by observable actions whose effects depend on external influences. Combinations of knowledge items determine how problem solving actions will alter a problem state when particular external influences prevail. Not only actions but also the resulting changes in problem state are observable. Being pri"vy to expert knowledge, testers can determine which knowledge items are in fact tested by a given problem state, given prevailing ex""ternal influences, goal state, and class of acceptable intermediate states. Misconceptions are treated like knowledge except to be e"xpert one must lack all misconceptions and not lack any knowledge.A tester observing a learner attempt to solve one or more proble"ms can construct a table that for each problem solving step shows which combinations of knowledge, and misconceptions, that are test"ed at that step lead to the action the student took or the change of state it produced. Each entry in this table constitutes evidenc"e for or against the hypothesis that the learner has knowledge, or lacks it, and lacks a misconception, or has it, which the corresp"onding step tests. A formal theory is to be developed for utilizing this evidence to arrive at a statistically sound estimate of the learner~s degree of mastery of knowledge items and freedom from misconceptions to the extent these are tested in the observed probl"em solving.Using the resulting estimate, specific knowledge gaps or present misconceptions can be identified as most likely respon""sible for some of the learner~s choices of action, or choice not to act. Knowledge gaps or misconceptions identified in this manner" are the most useful ones to give the learner feedback about regarding his choice at these problem solving steps. In this manner formative feedback can be adapted to individual learners at the same time as they are assessed.The project plans to implement algorit"hms for such assessment and adaptive feedback selection, and to instantiate them with knowledge of ship handling.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 07, 2017
- Source ID
- N000141712666
Entities
People
- Stanley Peters
Organizations
- Office of Naval Research
- Stanford University
- United States Navy