Human-In-The-Loop Machine Learning
Abstract
The quest for a fully personalized learning experience began with the development of intelligent tutoring systems (ITSs). However, t""o date, ITSs are primarily rules-based, meaning that building an ITS requires domain experts to consider every possible learning sce"nario the students can encounter and then manually specify the corresponding learning actions in each case. This approach is not sca"lable, since it is both labor-intensive and domain-specific. A scalable approach is to build personalized learning systems (PLSs) th"at automatically generate analytics and feedbackto students by analyzing the data they generate as they interact with learning reso"urces (e.g., practice questions,textbooks, lecture videos) using machine learning algorithms.This project builds on our promising"" preliminary work on sparse factor analysis for learning and content analytics(SPARFA), focused on analyzing the gradebook data whi"ch consists of students graded responses to assessment questions. We proposed a new statistical model that encodes the probability that a student will answer a question correctly in terms of three different factors: 1) the student s knowledge on each of a set of" latent factors (termed ~concepts~); 2) how the question relates to each concept, and 3) the intrinsic difficulty of each question." SPARFA provided the first ingredients towards building a PLS as we proposed a set of algorithms to estimate these quantities direct"ly from data. Using this information, SPARFA enables a PLS toautomatically provide analytics to each student on their strengths and" weaknesses and to instructors on the content and quality of the learning resources.Another undeveloped yet equally important func"tionality of a PLS is to recommend learning actions for each student to take. These actions, if personalized, can greatly enhance th""e learning efficiency of students by catering to their individual needs, interests, goals, and learning contexts. SPARFA estimates t"his information and enables us to select learning actions for each student to perform given their personal learning history.Moreov"er, learning is not simply about knowledge and skill acquisition; there are also important social aspects of it since interaction wi"th peers and forming rational viewpoints are also important in civic discourse. Traditional ways of personalization might confound this aspect of learning. The term ~filter bubble~ refers to the result of a personalized search in which a website algorithm selectiv"ely guesses what information a user would like to see based on information about the user (such as location, past click behavior and"" search history). As a result, users become separated frominformation that disagrees with their viewpoints, effectively isolating t"hem in their own cultural or ideological bubbles. There is a pressing need to develop automated tools and algorithms for the prevent"ion of the filter bubble in personalized learning and social media, since the subjectivity introduced by human intervention is inevi"table.This project is organized into two main research thrusts: Thrust 1 will develop and experimentally validate algorithms to au"tomatically recommend personalized learning actions (PLAs) for each student to maximize their learning outcome, given their estimate""d knowledge states estimated by SPARFA. Thrust 2 will investigate the social aspect of learning by avoiding the filter bubble, i.e.," ensure diversity among the opinions that students are exposed to as they interact with a PLS or other students.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- May 05, 2017
- Source ID
- N000141712551
Entities
People
- Richard G. Baraniuk
Organizations
- Office of Naval Research
- Rice University
- United States Navy