Of goals and habits

Abstract

In the television series The Wire , addicts Bubbles and Johnny regularly engage in bizarre and elaborate schemes to obtain drugs, ranging from robbing an ambulance to stealing drugs by lowering a fishing line from a rooftop. That these fictional crimes can be both so meticulously planned and yet focused on such narrow, shortsighted goals highlights a gap in our understanding of how the real brain deploys deliberative vs. automatic mechanisms to make decisions. On a standard account, people can deliberatively evaluate the consequences of candidate actions, which gives us our flexibility to dream up novel plans. Alternatively, the brain can crystallize repeatedly successful behaviors into habits: learned reflexes that free up resources by executing the behaviors automatically (although at the expense of inflexibility and, it is believed, underpinning pathological compulsions). As with most dichotomies, the problem with this view is that the world is not so black and white. Much as the drug-seeking behavior of addicts seems not to fit into either category, for healthy behaviors also, neither of these two sorts of decision making is very practical on its own. In PNAS, Cushman and Morris suggest a hybrid of these mechanisms, and show behavioral evidence that humans use it to plan actions (1). The study of Cushman and Morris (1) draws on recent advances using computational models of learning to make these strategies explicit enough that their hallmarks can be measured in choice behavior. Decisions are often modeled as determined by one’s estimates of the rewards expected from different options. There are many different methods to compute these decision variables. Deliberative planning can be formalized by an algorithmic family, called “model-based reinforcement learning” (2), which evaluates candidate sequences of actions much like a chess computer does, by exhaustively searching the “tree” of their future consequences, generated using … [↵][1]1Email: ndaw{at}princeton.edu. [1]: #xref-corresp-1-1

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 19 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.