Habits and goals in the brain: Competitive or cooperative?

Samuel J. Gershman
Massachusetts Institute of Technology

Modern theories of reinforcement learning posit two systems competing for control of behavior: a 
"model-free" or "habitual" system that learns cached state-action values, and a "model-based" or "goal-
directed" system that learns a world model which is then used to plan actions. Many behavioral and 
neural studies have indicated that these two systems can be dissociated, but recent data cast doubt on a 
strict separation of the two systems. In this talk, I will outline a cooperative architecture in which the 
model-based system transfers knowledge to the model-free system via simulation of surrogate 
experience. This kind of cooperative architecture comes with a number of computational benefits. I will 
then present a series of behavioral studies that are most parsimoniously explained by cooperation rather 
than competition between habits and goals.