Citation Relationships



Izhikevich EM (2007) Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling. Cereb Cortex 17(10):2443-2452[PubMed]

   Linking STDP and Dopamine action to solve the distal reward problem (Izhikevich 2007)

References and models cited by this paper

References and models that cite this paper

Ahissar E, Vaadia E, Ahissar M, Bergman H, Arieli A, Abeles M (1992) Dependence of cortical plasticity on correlated activity of single neurons and on behavioral context. Science 257:1412-5 [PubMed]

Au-Young SM, Shen H, Yang CR (1999) Medial prefrontal cortical output neurons to the ventral tegmental area (VTA) and their responses to burst-patterned stimulation of the VTA: neuroanatomical and in vivo electrophysiological analyses. Synapse 34:245-55 [PubMed]

Barad M, Bourtchouladze R, Winder DG, Golan H, Kandel E (1998) Rolipram, a type IV-specific phosphodiesterase inhibitor, facilitates the establishment of long-lasting long-term potentiation and improves memory. Proc Natl Acad Sci U S A 95:15020-5 [PubMed]

Barto AG, Sutton RS, Anderson CW (1983) Neuronlike elements that can solve difficult learning control problems IEEE Trans Systems Man Cybern 13:835-846

Bi GQ, Poo MM (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18:10464-72 [PubMed]

Calabresi P, Gubellini P, Centonze D, Picconi B, Bernardi G, Chergui K, Svenningsson P, Fienb (2000) Dopamine and cAMP-regulated phosphoprotein 32 kDa controls both striatal long-term depression and long-term potentiation, opposing forms of synaptic plasticity. J Neurosci 20:8443-51 [PubMed]

Cass WA, Gerhardt GA (1995) In vivo assessment of dopamine uptake in rat medial prefrontal cortex: comparison with dorsal striatum and nucleus accumbens. J Neurochem 65:201-7 [PubMed]

Centonze D, Gubellini P, Picconi B, Calabresi P, Giacomini P, Bernardi G (1999) Unilateral dopamine denervation blocks corticostriatal LTP. J Neurophysiol 82:3575-9 [Journal] [PubMed]

Choi S, Lovinger DM (1997) Decreased probability of neurotransmitter release underlies striatal long-term depression and postnatal development of corticostriatal synapses. Proc Natl Acad Sci U S A 94:2665-70 [PubMed]

Connors BW, Gutnick MJ (1990) Intrinsic firing patterns of diverse neocortical neurons. Trends Neurosci 13:99-104 [PubMed]

Dayan P, Abbott L (2001) Neural encoding: Firing rates and spike statistics Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems

Drew PJ, Abbott LF (2006) Extending the effects of spike-timing-dependent plasticity to behavioral timescales. Proc Natl Acad Sci U S A 103:8876-81

Frey U, Morris RG (1997) Synaptic tagging and long-term potentiation. Nature 385:533-6 [PubMed]

Frey U, Schroeder H, Matthies H (1990) Dopaminergic antagonists prevent long-term maintenance of posttetanic LTP in the CA1 region of rat hippocampal slices. Brain Res 522:69-75 [PubMed]

Fusi S, Drew PJ, Abbott LF (2005) Cascade models of synaptically stored memories. Neuron 45:599-611 [PubMed]

Garris PA, Ciolkowski EL, Pastore P, Wightman RM (1994) Efflux of dopamine from the synaptic cleft in the nucleus accumbens of the rat brain. J Neurosci 14:6084-93 [PubMed]

Gerstner W, Kempter R, van Hemmen JL, Wagner H (1996) A neuronal learning rule for sub-millisecond temporal coding. Nature 383:76-81 [PubMed]

Gurden H, Takita M, Jay TM (2000) Essential role of D1 but not D2 receptors in the NMDA receptor-dependent long-term potentiation at hippocampal-prefrontal cortex synapses in vivo. J Neurosci 20:RC106-93 [PubMed]

Houk JC, Adams JL, Barto AGA (1995) A model of how the basal ganglia generate and use neural signals that predict reinforcement. Models Of Information Processing In The Basal Ganglia, Houk JC:Davis JL:Beiser DG, ed. pp.249

Houk JC, Davis JL, Beiser DG (1995) Models of Information Processing in the Basal Ganglia

Hull CL (1943) Principles of behavior

Impey S, Mark M, Villacres EC, Poser S, Chavkin C, Storm DR (1996) Induction of CRE-mediated gene expression by stimuli that generate long-lasting LTP in area CA1 of the hippocampus. Neuron 16:973-82 [PubMed]

Izhikevich EM (2005) Polychronization: Computation with Spikes Neural Comput 18:245-282 [Journal] [PubMed]

   Polychronization: Computation With Spikes (Izhikevich 2005) [Model]

Izhikevich EM, Gally JA, Edelman GM (2004) Spike-timing dynamics of neuronal groups. Cereb Cortex 14:933-44 [PubMed]

Jay TM, Burette F, Laroche S (1996) Plasticity of the hippocampal-prefrontal cortex synapses. J Physiol Paris 90:361-6 [PubMed]

Kempter R, Gerstner W, Van_hemmen J (1999) Spike-based compared to rate-based Hebbian learning Advances In Neural Information Processing Systems, Kearns MS:Solla SA:Cohn DA, ed. pp.125

Kempter R, Gerstner W, van_Hemmen JL (1999) Hebbian learning and spiking neurons Physical Review E 59:4498-4514 [Journal]

Klopf AH (1982) The hedonistic neuron

Koene RA, Hasselmo ME (2005) An integrate-and-fire model of prefrontal cortex neuronal activity during performance of goal-directed decision making. Cereb Cortex 15:1964-81 [PubMed]

Lauwereyns J, Watanabe K, Coe B, Hikosaka O (2002) A neural correlate of response bias in monkey caudate nucleus. Nature 418:413-7 [PubMed]

Levy WB, Steward O (1983) Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus. Neuroscience 8:791-7 [PubMed]

Lisman J (1989) A mechanism for the Hebb and the anti-Hebb processes underlying learning and memory. Proc Natl Acad Sci U S A 86:9574-8 [PubMed]

Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145-63 [Journal] [PubMed]

Markram H, Lubke J, Frotscher M, Sakmann B (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275:213-5 [PubMed]

Minsky ML (1963) Steps toward artificial intelligence Computers and thought, Feigenbaum EA:Feldman J, ed. pp.406

Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936-47 [PubMed]

Montague PR, McClure SM, Baldwin PR, Phillips PE, Budygin EA, Stuber GD, Kilpatrick MR, Wight (2004) Dynamic gain control of dopamine delivery in freely moving animals. J Neurosci 24:1754-9 [PubMed]

Otani S, Daniel H, Roisin MP, Crepel F (2003) Dopaminergic modulation of long-term synaptic plasticity in rat prefrontal neurons. Cereb Cortex 13:1251-6 [PubMed]

Otmakhova NA, Lisman JE (1996) D1-D5 dopamine receptor activation increases the magnitude of early long-term potentiation at CA1 hippocampal synapses. J Neurosci 16:7478-86 [PubMed]

Otmakhova NA, Lisman JE (1998) D1/D5 dopamine receptors inhibit depotentiation at CA1 synapses via cAMP-dependent mechanism. J Neurosci 18:1270-9

Pan WX, Schmidt R, Wickens JR, Hyland BI (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25:6235-42 [PubMed]

Pasupathy A, Miller EK (2005) Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433:873-6 [PubMed]

Pavlov IP (1927) Conditioned reflexes.

Rao RP, Sejnowski TJ (2001) Spike-timing-dependent Hebbian plasticity as temporal difference learning. Neural Comput 13:2221-37 [PubMed]

Schirmer A, Tang SL, Penney TB, Gunter TC, Chen HC (2005) Brain responses to segmentally and tonally induced semantic violations in Cantonese. J Cogn Neurosci 17:1-12 [PubMed]

Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1-27 [Journal] [PubMed]

Schultz W (2002) Getting formal with dopamine and reward. Neuron 36:241-63 [PubMed]

Schultz W (2007) Reward Signals Scholarpedia (Forthcoming)

Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-9 [PubMed]

Seamans JK, Yang CR (2004) The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Prog Neurobiol 74:1-58 [PubMed]

Seung HS (2003) Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40:1063-73 [PubMed]

Song S, Miller KD, Abbott LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 3:919-26 [PubMed]

Suri RE, Schultz W (2001) Temporal difference model reproduces anticipatory neural activity. Neural Comput 13:841-62 [PubMed]

Sutton RS (1988) Learning to predict by the method of temporal diferences Machine Learning 3:9-44

Sutton RS, Barto AG (1998) Reinforcement learning: an introduction [Journal]

   A reinforcement learning example (Sutton and Barto 1998) [Model]

Swadlow HA (1990) Efferent neurons and suspected interneurons in S-1 forelimb representation of the awake rabbit: receptive fields and axonal properties. J Neurophysiol 63:1477-98 [Journal] [PubMed]

Swadlow HA (1994) Efferent neurons and suspected interneurons in motor cortex of the awake rabbit: axonal properties, sensory receptive fields, and subthreshold synaptic inputs. J Neurophysiol 71:437-53 [Journal]

Watanabe M (1996) Reward expectancy in primate prefrontal neurons. Nature 382:629-32 [PubMed]

Wightman RM, Zimmerman JB (2007) Control of dopamine extracellular concentration in rat striatum by impulse flow and uptake. Brain Res Brain Res Rev 15:135-44

Worgotter F, Porr B (2005) Temporal sequence learning, prediction, and control: a review of different models and their relation to biological mechanisms. Neural Comput 17:245-319 [PubMed]

Bittner KC, Milstein AD, Grienberger C, Romani S, Magee JC (2017) Behavioral time scale synaptic plasticity underlies CA1 place fields. Science 357:1033-1036 [Journal] [PubMed]

   Behavioral time scale synaptic plasticity underlies CA1 place fields (Bittner et al. 2017) [Model]

Brzosko Z, Zannone S, Schultz W, Clopath C, Paulsen O (2017) Sequential neuromodulation of Hebbian plasticity offers mechanism for effective reward-based navigation. Elife [Journal] [PubMed]

   Sequential neuromodulation of Hebbian plasticity in reward-based navigation (Brzosko et al 2017) [Model]

Chadderdon GL, Neymotin SA, Kerr CC, Lytton WW (2012) Reinforcement learning of targeted movement in a spiking neuronal model of motor cortex PLoS ONE 2012 7(10):e47251 [Journal]

   Reinforcement learning of targeted movement (Chadderdon et al. 2012) [Model]

Clopath C, Ziegler L, Vasilaki E, Busing L, Gerstner W (2008) Tag-trigger-consolidation: a model of early and late long-term-potentiation and depression. PLoS Comput Biol 4:e1000248 [Journal] [PubMed]

   Tag Trigger Consolidation (Clopath and Ziegler et al. 2008) [Model]

Dura-Bernal S, Neymotin SA, Kerr CC, Sivagnanam S, Majumdar A, Francis JT, Lytton WW (2017) Evolutionary algorithm optimization of biological learning parameters in a biomimetic neuroprosthesis. IBM Journal of Research and Development (Computational Neuroscience special issue) 61(2/3):6:1-6:14 [Journal]

   Motor system model with reinforcement learning drives virtual arm (Dura-Bernal et al 2017) [Model]

Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [Journal] [PubMed]

   Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015) [Model]

Esposito U, Giugliano M, Vasilaki E (2015) Adaptation of short-term plasticity parameters via error-driven learning may explain the correlation between activity-dependent synaptic properties, connectivity motifs and target specificity. Front Comput Neurosci 8:175 [Journal] [PubMed]

   Adaptation of Short-Term Plasticity parameters (Esposito et al. 2015) [Model]

Gurney KN, Humphries MD, Redgrave P (2015) A new framework for cortico-striatal plasticity: behavioural theory meets in vitro data at the reinforcement-action interface. PLoS Biol 13:e1002034 [Journal] [PubMed]

   Cortico-striatal plasticity in medium spiny neurons (Gurney et al 2015) [Model]

Legenstein R, Pecevski D, Maass W (2008) A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput Biol 4:e1000180 [Journal] [PubMed]

   Reward modulated STDP (Legenstein et al. 2008) [Model]

Nakano T, Otsuka M, Yoshimoto J, Doya K (2015) A spiking neural network model of model-free reinforcement learning with high-dimensional sensory input and perceptual ambiguity. PLoS One 10:e0115620 [Journal] [PubMed]

   A spiking neural network model of model-free reinforcement learning (Nakano et al 2015) [Model]

Neymotin SA, Chadderdon GL, Kerr CC, Francis JT, Lytton WW (2013) Reinforcement learning of 2-joint virtual arm reaching in a computer model of sensorimotor cortex Neural Computation 25(12):3263-93 [Journal] [PubMed]

   Sensorimotor cortex reinforcement learning of 2-joint virtual arm reaching (Neymotin et al. 2013) [Model]

Richmond P, Buesing L, Giugliano M, Vasilaki E (2011) Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations. PLoS One 6:e18539-58 [Journal] [PubMed]

   Democratic population decisions result in robust policy-gradient learning (Richmond et al. 2011) [Model]

Rivest F, Kalaska JF, Bengio Y (2010) Alternative time representation in dopamine models. Journal of Computational Neuroscience 28:107-30 [Journal] [PubMed]

   Alternative time representation in dopamine models (Rivest et al. 2009) [Model]

(73 refs)