An end-to-end architecture for efficient choice: From perception to goals
Abstract
For many real-world problems that we would like humans and AI to solve, a fundamental challenge is that there are more possible solu,tions than it is feasible to evaluate. This is why games like chess and go do not have simple algorithmic solutions. And, as compl,ex and open-ended as these games are, they embody just a small fraction of the difficulty encountered in many real world settings., In order to understand how humans make intelligent decisionsand to build machines that can do the samewe need to understand how,they make big problems small, efficiently selecting from a vast set of possibilities just a small set of candidate solutions for, rigorous evaluation.In prior work we illustrated an important part of the solution: value-guided consideration set construction. M,uch prior work shows that humans, like many AI architectures, estimate the value of actions. During deliberative choice they often,do this by computing the expected value of an action from a generative causal model of the situation at handa method that attains o,ptimal precision, but at severe computational cost. A computationally cheap alternative, however, is to estimate the value of a pre,sent action by extrapolating from its past value in similar contexts. In other words, if trading a pawn for a queen was valuable i,n many past matches, maybe its a good trade right nowand therefore deserving of further consideration. Our past work shows that p,eople used cached, model-free estimates of value to efficiently select a small set of candidate actions, and then use model-based de,liberation to choose among them. This work isolates and identifies a key element of how humans make efficient choices. But this pr,eliminary investigation involved several simplifying assumptions. Like most models in the reinforcement learning framework, we assu,med that the agent is presented with a simple representation of the world carved into states, actions and rewards suitable for, a tabular representationi.e., a Markov Decision Process. In the real world, of course, we are not presented with discrete states, and actions, but instead a high-dimensional perceptual input. And, a powerful tradition of past research in the cognitive scien,ces shows that humans do not structure decision-making around undifferentiated rewards, but rather a rich and hierarchical set of,goals and subgoals. In order to apply our insights to real world problems, we must integrate it with low-level perceptual systems, and high-level, semantically rich hierarchical goal representation.Our current proposal takes up this challenge. We propose a pr,ogram of research that places our simple model of efficient choice within the broader context of human cognition. The result will b,e end-to-end architectures for efficient choice in humans that are suitable for application in machine learning and robotics in real,-world settings.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- May 16, 2022
- Source ID
- N000142212205
Entities
People
- Fiery Cushman
Organizations
- Office of Naval Research
- President and Fellows of Harvard College
- United States Navy