Surgam

The Philolexian Foundation

In service to the Philolexian Society of Columbia University · Founded MDCCCII

Article · 2009

Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values

Publication

Title
Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values
Author
Nathaniel Daw (Columbia College, Class of 1993)
Form
Article
First published
2009
Publisher
The Journal of Neuroscience

Complete works of Nathaniel Daw (84)

  1. 2000 Behavioral considerations suggest an average reward TD model of the dopamine system · Neurocomputing
  2. 2001 Operant behavior suggests attentional gating of dopamine system inputs · Neurocomputing
  3. 2002 Long-Term Reward Prediction in TD Models of the Dopamine System · Neural Computation
  4. 2002 Opponent interactions between serotonin and dopamine · Neural Networks
  5. 2004 Matchmaking · Science
  6. 2005 Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control · Nature Neuroscience
  7. 2006 Choice values · Nature Neuroscience
  8. 2006 Planning and model-free versus model-driven behavior · PsycEXTRA Dataset
  9. 2006 The computational neurobiology of learning and reward · Current Opinion in Neurobiology
  10. 2007 Dopamine: at the intersection of reward and action · Nature Neuroscience
  11. 2008 Semi-rational models of conditioning: · The Probabilistic Mind:
  12. 2008 The Cognitive Neuroscience of Motivation and Learning · Social Cognition
  13. 2009 Computational cognitive neuroscience · Brain Research
  14. 2009 Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values · The Journal of Neuroscience this record
  15. 2009 Multiple Forms of Value Learning and the Function of Dopamine · Neuroeconomics
  16. 2009 Reinforcement learning and higher level cognition: Introduction to special issue · Cognition
  17. 2010 A closer look at choice · Nature Neuroscience
  18. 2010 Serotonin and Dopamine: Unifying Affective, Activational, and Decision Functions · Neuropsychopharmacology
  19. 2010 States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning · Neuron
  20. 2011 Grid Cells, Place Cells, and Geodesic Generalization for Spatial Reinforcement Learning · PLoS Computational Biology
  21. 2011 Multiplicity of control in the basal ganglia: computational roles of striatal subregions · Current Opinion in Neurobiology
  22. 2011 Neural Correlates of Forward Planning in a Spatial Decision Task in Humans · The Journal of Neuroscience
  23. 2011 Signals in Human Striatum Are Appropriate for Policy Update Rather than Value Prediction · The Journal of Neuroscience
  24. 2011 Trial-by-trial data analysis using computational models · Decision Making, Affect, and Learning
  25. 2012 Dissociating hippocampal and striatal contributions to sequential prediction learning · European Journal of Neuroscience
  26. 2012 Dual-System Learning Models and Drugs of Abuse · Computational Neuroscience of Drug Addiction
  27. 2012 Dynamic Estimation of Task-Relevant Variance in Movement under Risk · The Journal of Neuroscience
  28. 2012 Generalization of value in reinforcement learning by humans · European Journal of Neuroscience
  29. 2012 Perception, Action, and Utility: The Tangled Skein · Principles of Brain Dynamics
  30. 2012 The ubiquity of model-based reinforcement learning · Current Opinion in Neurobiology
  31. 2013 Action selection in multi-effector decision making · NeuroImage
  32. 2013 Cortical and Hippocampal Correlates of Deliberation during Model-Based Decisions for Rewards in Humans · PLoS Computational Biology
  33. 2013 Extraversion differentiates between model-based and model-free strategies in a reinforcement learning task · Frontiers in Human Neuroscience
  34. 2013 Testing Whether Humans Have an Accurate Model of Their Own Motor Uncertainty in a Speeded Reaching Task · PLoS Computational Biology
  35. 2013 The Irrationality of Categorical Perception · The Journal of Neuroscience
  36. 2014 Advanced Reinforcement Learning · Neuroeconomics
  37. 2014 Habits and Reinforcement Learning · The Cognitive Neurosciences
  38. 2014 Multiple Systems for Value Learning · Neuroeconomics
  39. 2014 The algorithmic anatomy of model-based evaluation · Philosophical Transactions of the Royal Society B: Biological Sciences
  40. 2014 Value Learning through Reinforcement · Neuroeconomics
  41. 2015 Criterion Learning in an Orientation-discrimination Task · Journal of Vision
  42. 2015 Deciding How To Decide: Self-Control and Meta-Decision Making · Trends in Cognitive Sciences
  43. 2015 Depression: A Decision-Theoretic Analysis · Annual Review of Neuroscience
  44. 2015 Human representation of visuo-motor uncertainty as mixtures of orthogonal basis distributions · Nature Neuroscience
  45. 2015 Integrating memories to guide decisions · Current Opinion in Behavioral Sciences
  46. 2015 Learning Processes in Parkinson’s Disease and Healthy Aging (P6.063) · Neurology
  47. 2015 Learning the opportunity cost of time in a patch-foraging task · Cognitive, Affective, & Behavioral Neuroscience
  48. 2015 Multiple memory systems as substrates for multiple decision systems · Neurobiology of Learning and Memory
  49. 2015 Of goals and habits · Proceedings of the National Academy of Sciences
  50. 2016 Taking Psychiatry Research Online · Neuron
  51. 2016 The expanding role of dopamine · eLife
  52. 2017 A retrieved context model of the emotional modulation of memory
  53. 2017 Prioritized memory access explains planning and hippocampal replay
  54. 2017 Reinforcement Learning and Episodic Memory in Humans and Animals: An Integrative Framework · Annual Review of Psychology
  55. 2017 Self-evaluation of decision-making: A general Bayesian framework for metacognitive computation. · Psychological Review
  56. 2017 The opportunity cost of time modulates cognitive effort
  57. 2018 Are we of two minds? · Nature Neuroscience
  58. 2018 In for a penny, in for a pound: Examining motivated memory through the lens of retrieved context models
  59. 2018 Memory mechanisms predict sampling biases in sequential decision tasks · 2018 Conference on Cognitive Computational Neuroscience
  60. 2018 Surviving threats: neural circuit and computational implications of a new taxonomy of defensive behaviour · Nature Reviews Neuroscience
  61. 2018 T216. Deficient Belief Updating Explains Abnormal Information Seeking Associated With Delusions in Schizophrenia · Biological Psychiatry
  62. 2019 A particle filtering account of selective attention during learning · 2019 Conference on Cognitive Computational Neuroscience
  63. 2019 Linear reinforcement learning: Flexible reuse of computation in planning, grid fields, and cognitive control
  64. 2019 Rational Arbitration of Hippocampal Replay · 2019 Conference on Cognitive Computational Neuroscience
  65. 2020 A model for learning based on the joint estimation of stochasticity and volatility
  66. 2020 A simple model for learning in volatile environments · PLOS Computational Biology
  67. 2020 Anxiety, Avoidance, and Sequential Evaluation · Computational Psychiatry
  68. 2020 Beyond the Average View of Dopamine · Trends in Cognitive Sciences
  69. 2020 Biased belief updating and suboptimal choice in foraging decisions · Nature Communications
  70. 2020 Model-Based and Model-Free Learning in Anorexia Nervosa and Other Disorders · Biological Psychiatry
  71. 2021 Anxiety is Associated With Reduced Value of Control in Sequential Decision Making · Biological Psychiatry
  72. 2021 Context-sensitive valuation and learning · Current Opinion in Behavioral Sciences
  73. 2021 Linear reinforcement learning in planning, grid fields, and cognitive control · Nature Communications
  74. 2022 Author response: Uncertainty alters the balance between incremental learning and episodic memory
  75. 2022 Measuring Behavioral Arbitration of the Successor Representation · 2022 Conference on Cognitive Computational Neuroscience
  76. 2022 Uncertainty alters the balance between incremental learning and episodic memory
  77. 2023 Computational processes of simultaneous learning of stochasticity and volatility in humans
  78. 2023 Model based control can give rise to devaluation insensitive choice · Addiction Neuroscience
  79. 2024 Trial-by-trial learning of successor representations in human behavior
  80. 2025 Humans rationally balance detailed and temporally abstract world models · Communications Psychology
  81. 2025 Proactive and reactive construction of memory-based preferences · Nature Communications
  82. 2025 Publisher Correction: Humans rationally balance detailed and temporally abstract world models · Communications Psychology
  83. 2025 Reconciling flexibility and efficiency: medial entorhinal cortex represents a compositional cognitive map · Nature Communications
  84. 2026 Planning in the Brain: It's Not What You Think It Is · Annual Review of Neuroscience

Cite this record

Daw, Nathaniel. Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values. The Journal of Neuroscience, 2009.

Foundation record: https://philolexianfoundation.org/members/daw-nathaniel-cc-1993/human-reinforcement-learning-subdivides-structured-action-spaces-by-learning-eff-2009.html