Citation Relationships



Worgotter F, Porr B (2005) Temporal sequence learning, prediction, and control: a review of different models and their relation to biological mechanisms. Neural Comput 17:245-319 [PubMed]

References and models cited by this paper

References and models that cite this paper

Abarbanel HD, Gibb L, Huerta R, Rabinovich MI (2003) Biophysical model of synaptic plasticity dynamics. Biol Cybern 89:214-26 [PubMed]

Abarbanel HD, Huerta R, Rabinovich MI (2002) Dynamical model of long-term synaptic plasticity. Proc Natl Acad Sci U S A 99:10132-7 [PubMed]

Abbott LF, Blum KI (2000) Functional significance of long-term potentiation for sequence learning and prediction. Cereb Cortex 6:406-16 [PubMed]

Abbott LF, Song S (1999) Temporally asymmetric Hebbian learning, postspike timing and neuronal response variability. Adv Neural Informat Processing Syst 11:69-75

Abeles M (1991) Corticonics: Neural Circuits of the Cerebral Cortex.

Akopian G, Musleh W, Smith R, Walsh JP (2000) Functional state of corticostriatal synapses determines their expression of short- and long-term plasticity. Synapse 38:271-80 [PubMed]

Altar CA, Boyar WC, Kim HS (1990) Discriminatory roles for D1 and D2 dopamine receptor subtypes in the in vivo control of neostriatal cyclic GMP. Eur J Pharmacol 181:17-21 [PubMed]

Anderson C, Crawford-Hines S (1994) Multigrid q-learning Tech. Rep. No. CS-94-121

Artola A, Singer W (1993) Long-term depression of excitatory synaptic transmission and its relationship to long-term potentiation. Trends Neurosci 16:480-7 [PubMed]

Ashby WR (1952) Design for a brain

Balkenius C, Moren J (1998) Computational models of classical conditioning: A comparative study LUCS 62 ISSN 1101-8453

Barto AG (1995) Adaptive critics and the basal ganglia Models of Information Processing in the Basal Ganglia, Houk JC:Davis JL:Beiser DG, ed. pp.1

Barto AG, Sutton RS, Anderson CW (1983) Neuronlike elements that can solve difficult learning control problems IEEE Trans Systems Man Cybern 13:835-846

Bell CC, Han VZ, Sugawara Y, Grant K (1997) Synaptic plasticity in a cerebellum-like structure depends on temporal order. Nature 387:278-81 [PubMed]

Bellman R (1957) Dynamic Programming

Bennett MR (2000) The concept of long term potentiation of transmission at synapses. Prog Neurobiol 60:109-37 [PubMed]

Berger B, Trottier S, Verney C, Gaspar P, Alvarez C (1988) Regional and laminar distribution of the dopamine and serotonin innervation in the macaque cerebral cortex: a radioautographic study. J Comp Neurol 273:99-119 [PubMed]

Berns GS, McClure SM, Pagnoni G, Montague PR (2001) Predictability modulates human brain response to reward. J Neurosci 21:2793-8 [PubMed]

Berns GS, Sejnowski TJ (1998) A computational model of how the basal ganglia produce sequences. J Cogn Neurosci 10:108-21 [PubMed]

Berridge KC, Robinson TE (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Brain Res Rev 28:309-69 [PubMed]

Berridge MJ (1998) Neuronal calcium signaling. Neuron 21:13-26 [PubMed]

Bi G, Poo M (2001) Synaptic modification by correlated activity: Hebb's postulate revisited. Annu Rev Neurosci 24:139-66 [PubMed]

Bi GQ (2002) Spatiotemporal specificity of synaptic plasticity: cellular rules and mechanisms. Biol Cybern 87:319-32 [PubMed]

Bi GQ, Poo MM (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18:10464-72 [PubMed]

Bibb JA, Snyder GL, Nishi A, Yan Z, Meijer L, Fienberg AA, Tsai LH, Kwon YT, Girault JA, Czer (1999) Phosphorylation of DARPP-32 by Cdk5 modulates dopamine signalling in neurons. Nature 402:669-71 [PubMed]

Bienenstock EL, Cooper LN, Munro PW (1982) Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J Neurosci 2:32-48 [PubMed]

Bliss TV, Gardner-Medwin AR (1973) Long-lasting potentiation of synaptic transmission in the dentate area of the unanaestetized rabbit following stimulation of the perforant path. J Physiol 232:357-74 [PubMed]

Bliss TV, Lomo T (1970) Plasticity in a monosynaptic cortical pathway. J Physiol 207:61P

Bliss TV, Lomo T (1973) Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J Physiol 232:331-56 [PubMed]

Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P (2001) Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30:619-39 [PubMed]

Brown J, Bullock D, Grossberg S (1999) How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J Neurosci 19:10502-11 [PubMed]

Brown JR, Arbuthnott GW (1983) The electrophysiology of dopamine (D2) receptors: a study of the actions of dopamine on corticostriatal transmission. Neuroscience 10:349-55

Buchs NJ, Senn W (2002) Spike-based synaptic plasticity and the emergence of direction selective simple cells: simulation results. J Comput Neurosci 13:167-86 [Journal]

Bunney BS, Chiodo LA, Grace AA (1991) Midbrain dopamine system electrophysiological functioning: a review and new hypothesis. Synapse 9:79-94 [PubMed]

Buzsaki G, Penttonen M, Nadasdy Z, Bragin A (1996) Pattern and inhibition-dependent invasion of pyramidal cell dendrites by fast spikes in the hippocampus in vivo. Proc Natl Acad Sci U S A 93:9921-5 [PubMed]

Calabresi P, Centonze D, Gubellini P, Marfia GA, Bernardi G (1999) Glutamate-triggered events inducing corticostriatal long-term depression. J Neurosci 19:6102-10 [PubMed]

Calabresi P, Fedele E, Pisani A, Fontana G, Mercuri NB, Bernardi G, Raiteri M (1995) Transmitter release associated with long-term synaptic depression in rat corticostriatal slices. Eur J Neurosci 7:1889-94 [PubMed]

Calabresi P, Gubellini P, Centonze D, Picconi B, Bernardi G, Chergui K, Svenningsson P, Fienb (2000) Dopamine and cAMP-regulated phosphoprotein 32 kDa controls both striatal long-term depression and long-term potentiation, opposing forms of synaptic plasticity. J Neurosci 20:8443-51 [PubMed]

Calabresi P, Maj R, Mercuri NB, Bernardi G (1992) Coactivation of D1 and D2 dopamine receptors is required for long-term synaptic depression in the striatum. Neurosci Lett 142:95-9 [PubMed]

Calabresi P, Maj R, Pisani A, Mercuri NB, Bernardi G (1992) Long-term synaptic depression in the striatum: physiological and pharmacological characterization. J Neurosci 12:4224-33 [PubMed]

Calabresi P, Mercuri N, Stanzione P, Stefani A, Bernardi G (1987) Intracellular studies on the dopamine-induced firing inhibition of neostriatal neurons in vitro: evidence for D1 receptor involvement. Neuroscience 20:757-71 [PubMed]

Calabresi P, Pisani A, Mercuri NB, Bernardi G (1992) Long-term Potentiation in the Striatum is Unmasked by Removing the Voltage-dependent Magnesium Block of NMDA Receptor Channels. Eur J Neurosci 4:929-935 [PubMed]

Calabresi P, Pisani A, Mercuri NB, Bernardi G (1996) The corticostriatal projection: from synaptic plasticity to dysfunctions of the basal ganglia. Trends Neurosci 19:19-24 [PubMed]

Cannon CM, Palmiter RD (2003) Reward without dopamine. J Neurosci 23:10827-31 [PubMed]

Carpenter G, Grossberg S (1987) Art 2: Self-organization of stable category recognition codes for analog input patterns. Appl Opt 26:4919-4930

Castellani GC, Quinlan EM, Cooper LN, Shouval HZ (2001) A biophysical model of bidirectional synaptic plasticity: dependence on AMPA and NMDA receptors. Proc Natl Acad Sci U S A 98:12772-7 [PubMed]

Cateau H, Kitano K, Fukai T (2002) An accurate and widely applicable method to determine the distribution of synaptic strengths formed by the spike-timing-dependent learning Neurocomputing 44:343-351

Centonze D, Grande C, Usiello A, Gubellini P, Erbs E, Martin AB, Pisani A, Tognazzi N, Bernar (2003) Receptor subtypes involved in the presynaptic and postsynaptic actions of dopamine on striatal interneurons. J Neurosci 23:6245-54 [PubMed]

Centonze D, Gubellini P, Picconi B, Calabresi P, Giacomini P, Bernardi G (1999) Unilateral dopamine denervation blocks corticostriatal LTP. J Neurophysiol 82:3575-9 [Journal] [PubMed]

Charpier S, Deniau JM (1997) In vivo activity-dependent plasticity at cortico-striatal connections: evidence for physiological long-term potentiation. Proc Natl Acad Sci U S A 94:7036-40 [PubMed]

Charpier S, Mahon S, Deniau JM (1999) In vivo induction of striatal long-term potentiation by low-frequency stimulation of the cerebral cortex. Neuroscience 91:1209-22 [PubMed]

Choi S, Lovinger DM (1997) Decreased probability of neurotransmitter release underlies striatal long-term depression and postnatal development of corticostriatal synapses. Proc Natl Acad Sci U S A 94:2665-70 [PubMed]

Contreras-Vidal JL, Schultz W (2004) A predictive reinforcement model of dopamine neurons for learning approach behavior. J Comput Neurosci 6:191-214 [Journal] [PubMed]

Daw ND (2003) Reinforcement learning models of the dopamine system and their behavioral implications Unpublished doctoral dissertation

Dayan P (1992) The convergence of TD(Lambda) Mach Learn 8:341-362

Dayan P (2002) Matters temporal. Trends Cogn Sci 6:105-106

Dayan P, Balleine BW (2002) Reward, motivation, and reinforcement learning. Neuron 36:285-98 [PubMed]

Dayan P, Kakade S, Montague PR (2000) Learning and selective attention. Nat Neurosci 3 Suppl:1218-23 [PubMed]

Dayan P, Sejnowski T (1994) Td (Lambda) converges with probability 1 Mach Learn 14:295-301

De Koninck P, Schulman H (1998) Sensitivity of CaM kinase II to the frequency of Ca2+ oscillations. Science 279:227-30 [PubMed]

Debanne D, Gahwiler BH, Thompson SM (1994) Asynchronous pre- and postsynaptic activity induces associative long-term depression in area CA1 of the rat hippocampus in vitro. Proc Natl Acad Sci U S A 91:1148-52 [PubMed]

Debanne D, Gahwiler BH, Thompson SM (1998) Long-term synaptic plasticity between pairs of individual CA3 pyramidal cells in rat hippocampal slice cultures. J Physiol 507 ( Pt 1):237-47 [PubMed]

Deisseroth K, Heist EK, Tsien RW (1998) Translocation of calmodulin to the nucleus supports CREB phosphorylation in hippocampal neurons. Nature 392:198-202 [PubMed]

Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84:3072-7 [Journal] [PubMed]

Dennett DC (1984) Cognitive wheels: The frame problem of AI Minds, machines and evolution, Hookway C, ed. pp.129

Dolmetsch RE, Pajvani U, Fife K, Spotts JM, Greenberg ME (2001) Signaling to the nucleus by an L-type calcium channel-calmodulin complex through the MAP kinase pathway. Science 294:333-9 [PubMed]

Doya K (2000) Reinforcement learning in continuous time and space. Neural Comput 12:219-45

Egger V, Feldmeyer D, Sakmann B (1999) Coincidence detection and changes of synaptic efficacy in spiny stellate neurons in rat barrel cortex. Nat Neurosci 2:1098-105 [PubMed]

Elliott R, Friston KJ, Dolan RJ (2000) Dissociable neural responses in human reward systems. J Neurosci 20:6159-65 [PubMed]

Eurich C, Pawelzik K, Ernst U, Cowan J, Milton J (1999) Dynamics of self-organazed delay adaptation Phys Rev Lett 82:1594-1597

Feldman DE (2000) Timing-based LTP and LTD at vertical inputs to layer II/III pyramidal cells in rat barrel cortex. Neuron 27:45-56 [PubMed]

Froemke RC, Dan Y (2002) Spike-timing-dependent synaptic modification induced by natural spike trains. Nature 416:433-8 [PubMed]

Fusi S (2002) Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol Cybern 87:459-70 [PubMed]

Futatsugi A, Kato K, Ogura H, Li ST, Nagata E, Kuwajima G, Tanaka K, Itohara S, Mikoshiba K (1999) Facilitation of NMDAR-independent LTP and spatial learning in mutant mice lacking ryanodine receptor type 3. Neuron 24:701-13 [PubMed]

Gerfen CR (1985) The neostriatal mosaic. I. Compartmental organization of projections from the striatum to the substantia nigra in the rat. J Comp Neurol 236:454-76 [PubMed]

Gerfen CR (1992) The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia. Annu Rev Neurosci 15:285-320 [PubMed]

Gerfen CR (2004) The neostriatal mosaic: compartmentalization of corticostriatal input and striatonigral output systems. Nature 311:461-4 [PubMed]

Gerfen CR, Herkenham M, Thibault J (1987) The neostriatal mosaic: II. Patch- and matrix-directed mesostriatal dopaminergic and non-dopaminergic systems. J Neurosci 7:3915-34 [PubMed]

Gerstner W, Kempter R, van Hemmen JL, Wagner H (1996) A neuronal learning rule for sub-millisecond temporal coding. Nature 383:76-81 [PubMed]

Gerstner W, Kistler WM (2002) Mathematical formulations of Hebbian learning. Biol Cybern 87:404-15 [PubMed]

Goldberg J, Holthoff K, Yuste R (2002) A problem with Hebb and local spikes. Trends Neurosci 25:433-5 [PubMed]

Golding NL, Kath WL, Spruston N (2001) Dichotomy of action-potential backpropagation in CA1 pyramidal neuron dendrites. J Neurophysiol 86:2998-3010 [Journal] [PubMed]

   Dichotomy of action-potential backpropagation in CA1 pyramidal neuron dendrites (Golding et al 2001) [Model]

Golding NL, Staff NP, Spruston N (2002) Dendritic spikes as a mechanism for cooperative long-term potentiation. Nature 418:326-31 [PubMed]

Gormezano I, Kehoe EJ, Marshall BS (1983) Twenty years ofclassical conditioning research with the rabbit Prog Psychobiol Phys Psychol 10:197-275

Graef IA, Mermelstein PG, Stankunas K, Neilson JR, Deisseroth K, Tsien RW, Crabtree GR (1999) L-type calcium channels and GSK-3 regulate the activity of NF-ATc4 in hippocampal neurons. Nature 401:703-8 [PubMed]

Greengard P, Allen PB, Nairn AC (1999) Beyond the dopamine receptor: the DARPP-32-protein phosphatase-1 cascade. Neuron 23:435-47 [PubMed]

Groenewegen HJ, Berendse HW, Wolters JG, Lohman AH (1990) The anatomical relationship of the prefrontal cortex with the striatopallidal system, the thalamus and the amygdala: evidence for a parallel organization. Prog Brain Res 85:95-116; discussion 116-8 [PubMed]

Grossberg S (1995) A spectral network model of pitch perception J Acoust Soc Am 98:862-879

Grossberg S, Merrill JWL (1996) The hippocampus and cerebellum in adaptively timed learning, recognition and movement J Cogn Neurosci 8:257-277

Grossberg S, Schmajuk NA (1989) Neural dynamics of adaptive timing and temporal discrimination during associative learning Neural Netw Networks2:79-102

Groves PM, Garcia-munoz M, Linder JC, Manley MS, Martone ME, Young SJ (1995) Elements of the intrinsic organization and information processing in the neostriatum Models of information processing in the basal ganglia, Houk JC:Davis JL:Beiser DG, ed. pp.51

Gruber AJ, Solla SA, Surmeier DJ, Houk JC (2003) Modulation of striatal single units by expected reward: a spiny neuron model displaying dopamine-induced bistability. J Neurophysiol 90:1095-114 [Journal] [PubMed]

   Spiny neuron model with dopamine-induced bistability (Gruber et al 2003) [Model]

Gustafsson B, Wigstrom H, Abraham WC, Huang YY (1987) Long-term potentiation in the hippocampus using depolarizing current pulses as the conditioning stimulus to single volley synaptic potentials. J Neurosci 7:774-80 [PubMed]

Gutig R, Aharonov R, Rotter S, Sompolinsky H (2003) Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity. J Neurosci 23:3697-714 [PubMed]

Haber SN, Fudge JL, McFarland NR (2000) Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20:2369-82 [PubMed]

Han VZ, Grant K, Bell CC (2000) Reversible associative depression and nonassociative potentiation at a parallel fiber synapse. Neuron 27:611-22 [PubMed]

Hassani OK, Cromwell HC, Schultz W (2001) Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol 85:2477-89 [Journal] [PubMed]

Hauber W, Bohn I, Giertler C (2000) NMDA, but not dopamine D(2), receptors in the rat nucleus accumbens areinvolved in guidance of instrumental behavior by stimuli predicting reward magnitude. J Neurosci 20:6282-8

Hausser M, Spruston N, Stuart GJ (2000) Diversity and dynamics of dendritic signaling. Science 290:739-44 [PubMed]

Hebb DO (1949) The Organization Of Behavior

Hemmings HC, Greengard P, Tung HY, Cohen P (2005) DARPP-32, a dopamine-regulated neuronal phosphoprotein, is a potent inhibitor of protein phosphatase-1. Nature 310:503-5 [PubMed]

Hernandez-Lopez S, Bargas J, Surmeier DJ, Reyes A, Galarraga E (1997) D1 receptor activation enhances evoked discharge in neostriatal medium spiny neurons by modulating an L-type Ca2+ conductance. J Neurosci 17:3334-42 [PubMed]

Hoffman DA, Magee JC, Colbert CM, Johnston D (1997) K+ channel regulation of signal propagation in dendrites of hippocampal pyramidal neurons. Nature 387:869-75 [PubMed]

Hollerman JR, Schultz W (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304-9 [PubMed]

Hollerman JR, Tremblay L, Schultz W (1998) Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80:947-63 [Journal] [PubMed]

Houk JC, Adams JL, Barto AGA (1995) A model of how the basal ganglia generate and use neural signals that predict reinforcement. Models Of Information Processing In The Basal Ganglia, Houk JC:Davis JL:Beiser DG, ed. pp.249

Hull CL (1939) The problem of stimulus equivalence in behavior theory Psychol Rev 46:9-30

Hull CL (1943) Principles of behavior

Izhikevich EM, Desai NS (2003) Relating STDP to BCM. Neural Comput 15:1511-23 [PubMed]

Jaakkola T, Jordan MI, Singh SP (1995) On the convergence of stochastic iterative dynamic programming algorithms Neural Comput 6:1185-1201

Joel D, Niv Y, Ruppin E (2005) Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw 15:535-47 [PubMed]

Joel D, Weiner I (2000) The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 96:451-74 [PubMed]

Johnston D, Magee JC, Colbert CM, Cristie BR (1996) Active properties of neuronal dendrites. Annu Rev Neurosci 19:165-86 [PubMed]

Joyce JN, Marshall JF (1987) Quantitative autoradiography of dopamine D2 sites in rat caudate-putamen: localization to intrinsic neurons and not to neocortical afferents. Neuroscience 20:773-95

Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: A survey J Art Intell Res 4:237-285

Karmarkar UR, Buonomano DV (2002) A model of spike-timing dependent plasticity: one or two coincidence detectors? J Neurophysiol 88:507-13 [Journal] [PubMed]

Karmarkar UR, Najarian MT, Buonomano DV (2002) Mechanisms and significance of spike-timing dependent plasticity. Biol Cybern 87:373-82 [PubMed]

Kawagoe R, Takikawa Y, Hikosaka O (1998) Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1:411-6 [PubMed]

Kempter R, Gerstner W, van Hemmen JL (2001) Intrinsic stabilization of output rates by spike-based Hebbian learning. Neural Comput 13:2709-41 [PubMed]

Kempter R, Gerstner W, van_Hemmen JL (1999) Hebbian learning and spiking neurons Physical Review E 59:4498-4514 [Journal]

Kempter R, Leibold C, Wagner H, van Hemmen JL (2001) Formation of temporal-feature maps by axonal propagation of synaptic learning. Proc Natl Acad Sci U S A 98:4166-71 [PubMed]

King MM, Huang CY, Chock PB, Nairn AC, Hemmings HC, Chan KF, Greengard P (1984) Mammalian brain phosphoproteins as substrates for calcineurin. J Biol Chem 259:8080-3 [PubMed]

Kistler WM (2002) Spike-timing dependent synaptic plasticity: a phenomenological framework. Biol Cybern 87:416-27 [PubMed]

Kistler WM, van Hemmen JL (2000) Modeling synaptic plasticity in conjuction with the timing of pre- and postsynaptic action potentials. Neural Comput 12:385-405 [PubMed]

Kitano K, Okamoto H, Fukai T (2003) Time representing cortical activities: two models inspired by prefrontal persistent activity. Biol Cybern 88:387-94

Klopf AH (1972) Brain function and adaptive systems-a heterostatic theory Tech. Rep. Air Force Cambridge Research Laboratories Special Report No. 133

Klopf AH (1982) The hedonistic neuron

Klopf AH (1986) A drive-reinforcement model of single neuron function Neural networks for computing: AIP Conference Proceedings, Denker JS, ed.

Klopf AH (1988) A neuronal model of classical conditioning Psychobiology 16:85-125

Knutson B, Fong GW, Adams CM, Varner JL, Hommer D (2001) Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport 12:3683-7 [PubMed]

Koester HJ, Sakmann B (1998) Calcium dynamics in single spines during coincident pre- and postsynaptic activity depend on relative timing of back-propagating action potentials and subthreshold excitatory postsynaptic potentials. Proc Natl Acad Sci U S A 95:9596-601 [PubMed]

Kosco B (1986) Differential Hebbian learning Neural networks for computing: AIP Conference Proceedings, Denker JS, ed. pp.277

Lasser-Ross N, Ross WN (1992) Imaging voltage and synaptically activated sodium transients in cerebellar Purkinje cells. Proc R Soc Lond B Biol Sci 247:35-9 [PubMed]

Linden DJ (1999) The return of the spike: postsynaptic action potentials and the induction of LTP and LTD. Neuron 22:661-6 [PubMed]

Lisman J (1989) A mechanism for the Hebb and the anti-Hebb processes underlying learning and memory. Proc Natl Acad Sci U S A 86:9574-8 [PubMed]

Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145-63 [Journal] [PubMed]

Lynd-Balta E, Haber SN (1994) Primate striatonigral projections: a comparison of the sensorimotor-related striatum and the ventral striatum. J Comp Neurol 345:562-78 [PubMed]

Magee JC, Johnston D (1997) A synaptically controlled, associative signal for Hebbian plasticity in hippocampal neurons. Science 275:209-13 [PubMed]

Malenka RC, Kauer JA, Perkel DJ, Mauk MD, Kelly PT, Nicoll RA, Waxham MN (1989) An essential role for postsynaptic calmodulin and protein kinase activity in long-term potentiation. Nature 340:554-7 [PubMed]

Malenka RC, Nicoll RA (1999) Long-term potentiation--a decade of progress? Science 285:1870-4 [PubMed]

Malinow R, Schulman H, Tsien RW (1989) Inhibition of postsynaptic PKC or CaMKII blocks induction but not expression of LTP. Science 245:862-6 [PubMed]

Markram H, Lubke J, Frotscher M, Sakmann B (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275:213-5 [PubMed]

Mayer ML, Westbrook GL, Guthrie PB (2000) Voltage-dependent block by Mg2+ of NMDA responses in spinal cord neurones. Nature 309:261-3 [PubMed]

Mehta MR (2004) Cooperative LTP can map memory sequences on dendritic branches. Trends Neurosci 27:69-72 [PubMed]

Mehta MR, Lee AK, Wilson MA (2002) Role of experience and oscillations in transforming a rate code into a temporal code. Nature 417:741-6 [PubMed]

Migliore M, Hoffman DA, Magee JC, Johnston D (1999) Role of an A-type K+ conductance in the back-propagation of action potentials in the dendrites of hippocampal pyramidal neurons. J Comput Neurosci 7:5-15 [Journal] [PubMed]

   CA1 pyramidal neuron (Migliore et al 1999) [Model]

Miller JD, Sanghera MK, German DC (1981) Mesencephalic dopaminergic unit activity in the behaviorally conditioned rat. Life Sci 29:1255-63 [PubMed]

Montague PR, Dayan P, Person C, Sejnowski TJ (1995) Bee foraging in uncertain environments using predictive hebbian learning. Nature 377:725-8 [PubMed]

Mulkey RM, Endo S, Shenolikar S, Malenka RC (1994) Involvement of a calcineurin/inhibitor-1 phosphatase cascade in hippocampal long-term depression. Nature 369:486-8 [PubMed]

Nicola SM, Surmeier J, Malenka RC (2000) Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu Rev Neurosci 23:185-215 [PubMed]

Nishiyama M, Hong K, Mikoshiba K, Poo MM, Kato K (2000) Calcium stores regulate the polarity and input specificity of synaptic modification. Nature 408:584-8 [PubMed]

No Authors Listed (2002) Neuronal coding '01. Papers presented at the International Workshop on Neuronal Coding '01. Plymouth, United Kingdom. September 9-14, 2001. Biosystems 67:1-300 [PubMed]

Nowak L, Bregestovski P, Ascher P, Herbet A, Prochiantz A (2000) Magnesium gates glutamate-activated channels in mouse central neurones. Nature 307:462-5 [PubMed]

O'Doherty J, Rolls ET, Francis S, Bowtell R, McGlone F (2001) Representation of pleasant and aversive taste in the human brain. J Neurophysiol 85:1315-21 [Journal] [PubMed]

O'Doherty JP, Deichmann R, Critchley HD, Dolan RJ (2002) Neural responses during anticipation of a primary taste reward. Neuron 33:815-26 [PubMed]

Overton PG, Clark D (1997) Burst firing in midbrain dopaminergic neurons. Brain Res Brain Res Rev 25:312-34 [PubMed]

Pacheco-Cano MT, Bargas J, Hernandez-Lopez S, Tapia D, Galarraga E (1996) Inhibitory action of dopamine involves a subthreshold Cs(+)-sensitive conductance in neostriatal neurons. Exp Brain Res 110:205-11 [PubMed]

Pagnoni G, Zink CF, Montague PR, Berns GS (2002) Activity in human ventral striatum locked to errors of reward prediction. Nat Neurosci 5:97-8 [PubMed]

Parent A (1990) Extrinsic connections of the basal ganglia. Trends Neurosci 13:254-8 [PubMed]

Partridge JG, Tang KC, Lovinger DM (2000) Regional and postnatal heterogeneity of activity-dependent long-term changes in synaptic efficacy in the dorsal striatum. J Neurophysiol 84:1422-9 [Journal] [PubMed]

Pike FG, Meredith RM, Olding AW, Paulsen O (1999) Rapid report: postsynaptic bursting is essential for 'Hebbian' induction of associative long-term potentiation at excitatory synapses in rat hippocampus. J Physiol 518 ( Pt 2):571-6 [PubMed]

Porr B, von Ferber C, Worgotter F (2003) ISO learning approximates a solution to the inverse-controller problem in an unsupervised behavioral paradigm. Neural Comput 15:865-84 [PubMed]

Porr B, Worgotter F (2003) Isotropic-sequence-order learning in a closed-loop behavioural system. Philos Transact A Math Phys Eng Sci 361:2225-44

Porr B, Worgotter F (2003) Isotropic sequence order learning. Neural Comput 15:831-64 [PubMed]

Prokasy WF, Hall JF, Fawcett JT (1962) Adaptation, sensitization, forward and backward conditioning, and pseudo-conditioning of the GSR Psychol Rep 10:103-106

Pucak ML, Grace AA (1994) Regulation of substantia nigra dopamine neurons. Crit Rev Neurobiol 9:67-89 [PubMed]

Rao RP, Sejnowski TJ (2001) Spike-timing-dependent Hebbian plasticity as temporal difference learning. Neural Comput 13:2221-37 [PubMed]

Redgrave P, Prescott TJ, Gurney K (1999) Is the short-latency dopamine response too short to signal reward error? Trends Neurosci 22:146-51 [PubMed]

Regehr WG, Konnerth A, Armstrong CM (1992) Sodium action potentials in the dendrites of cerebellar Purkinje cells. Proc Natl Acad Sci U S A 89:5492-6 [PubMed]

Rescorla R, Wagner A (1972) A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement Classical Conditioning II: Current Research and Theory, Black A:Prokasy W, ed. pp.64

Reynolds JN, Wickens JR (2005) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 15:507-21 [PubMed]

Reynolds SI (2002) Reinforcement learning with exploration Unpublished doctoral dissertation

Roberts PD (2000) Computational consequences of temporally asymmetric learning rules: I. Differential hebbian learning. J Comput Neurosci 7:235-46 [Journal] [PubMed]

Roberts PD, Bell CC (2002) Spike timing dependent synaptic plasticity in biological systems. Biol Cybern 87:392-403 [PubMed]

Roche KW, O'Brien RJ, Mammen AL, Bernhardt J, Huganir RL (1996) Characterization of multiple phosphorylation sites on the AMPA receptor GluR1 subunit. Neuron 16:1179-88 [PubMed]

Rose CR, Konnerth A (2001) Stores not just for storage. intracellular calcium release and synaptic plasticity. Neuron 31:519-22 [PubMed]

Rubin J, Lee DD, Sompolinsky H (2001) Equilibrium properties of temporally asymmetric Hebbian plasticity. Phys Rev Lett 86:364-7 [PubMed]

Rummery GA (1995) Problem solving with reinforcement learning Unpublished doctoral dissertation

Russell VA, Allin R, Lamm MC, Taljaard JJ (1992) Regional distribution of monoamines and dopamine D1- and D2-receptors in the striatum of the rat. Neurochem Res 17:387-95

Santos JM, Touzet C (1999) Dynamic update of the reinforcement function during learning Connection Science [Special issue]

Santos JM, Touzet C (1999) Exploration tuned reinforcement function Neurocomputing [Special issue] 28:93-105

Sato N, Yamaguchi Y (2003) Memory encoding by theta phase precession in the hippocampal network. Neural Comput 15:2379-97 [PubMed]

Saudargiene A, Porr B, Worgotter F (2004) How the shape of pre- and postsynaptic signals can influence STDP: a biophysical model. Neural Comput 16:595-625 [PubMed]

Schiller J, Schiller Y, Stuart G, Sakmann B (1997) Calcium action potentials restricted to distal apical dendrites of rat neocortical pyramidal neurons. J Physiol 505 ( Pt 3):605-16 [PubMed]

Schoenbaum G, Chiba AA, Gallagher M (1998) Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci 1:155-9 [PubMed]

Schultz W (1986) Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol 56:1439-61 [Journal] [PubMed]

Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1-27 [Journal] [PubMed]

Schultz W (2002) Getting formal with dopamine and reward. Neuron 36:241-63 [PubMed]

Schultz W, Apicella P, Scarnati E, Ljungberg T (1992) Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12:4595-610 [PubMed]

Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-9 [PubMed]

Schultz W, Dickinson A (2000) Neuronal coding of prediction errors. Annu Rev Neurosci 23:473-500 [PubMed]

Senn W, Buchs NJ (2004) Spike-based synaptic plasticity and the emergence of direction selective simple cells: mathematical analysis. J Comput Neurosci 14:119-38 [Journal]

Senn W, Markram H, Tsodyks M (2001) An algorithm for modifying neurotransmitter release probability based on pre- and postsynaptic spike timing. Neural Comput 13:35-67 [PubMed]

Seung HS (1998) Learning continuous attractors in recurrent networks Adv Neural Info Proc Sys 10:654-660

Shouval HZ, Bear MF, Cooper LN (2002) A unified model of NMDA receptor-dependent bidirectional synaptic plasticity. Proc Natl Acad Sci U S A 99:10831-6 [PubMed]

Sibley DR, Monsma FJ (1992) Molecular biology of dopamine receptors. Trends Pharmacol Sci 13:61-9

Singh SP, Sutton RS (1996) Reinforcement learning with replacing eligibility traces Mach Learn 22:123-158

Sjostrom PJ, Turrigiano GG, Nelson SB (2001) Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 32:1149-64 [PubMed]

Song I, Huganir RL (2002) Regulation of AMPA receptors during synaptic plasticity. Trends Neurosci 25:578-88

Song S, Miller KD, Abbott LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 3:919-26 [PubMed]

Spencer JP, Murphy KP (2000) Bi-directional changes in synaptic plasticity induced at corticostriatal synapses in vitro. Exp Brain Res 135:497-503 [PubMed]

Sterratt DC, van_Ooyen A (2002) Does morphology influence temporal plasticity? ICANN 2002, Dorronsoro JR, ed. pp.186

Stoof JC, Kebabian JW (1981) Opposing roles for D-1 and D-2 dopamine receptors in efflux of cyclic AMP from rat neostriatum. Nature 294:366-8 [PubMed]

Stuart G, Spruston N, Sakmann B, Hausser M (1997) Action potential initiation and backpropagation in neurons of the mammalian CNS. Trends Neurosci 20:125-31 [PubMed]

Stuart GJ, Sakmann B (1994) Active propagation of somatic action potentials into neocortical pyramidal cell dendrites. Nature 367:69-72 [PubMed]

Suri RE, Bargas J, Arbib MA (2001) Modeling functions of striatal dopamine modulation in learning and planning. Neuroscience 103:65-85 [PubMed]

Suri RE, Schultz W (1998) Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Exp Brain Res 121:350-4 [PubMed]

Suri RE, Schultz W (1999) A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91:871-90 [PubMed]

Suri RE, Schultz W (2001) Temporal difference model reproduces anticipatory neural activity. Neural Comput 13:841-62 [PubMed]

Surmeier DJ, Bargas J, Hemmings HC, Nairn AC, Greengard P (1995) Modulation of calcium currents by a D1 dopaminergic protein kinase-phosphatase cascade in rat neostriatal neurons. Neuron 14:385-97 [PubMed]

Surmeier DJ, Kitai ST (1993) D1 and D2 dopamine receptor modulation of sodium and potassium currents in rat neostriatal neurons. Prog Brain Res 99:309-24 [PubMed]

Sutton RS (1984) Temporal credit assignment in reinforcement learning Unpublished doctoral dissertation

Sutton RS (1988) Learning to predict by the method of temporal diferences Machine Learning 3:9-44

Sutton RS (1996) Generalization in reinforcement learning: Successful examples using sparse coarse coding Advances in neural information processing systems, Touretzky DS:Mozer MC:Hasselmo ME, ed. pp.1038

Sutton RS (1999) Open theoretical questions in reinforcement learning EUROColt :11-17

Sutton RS, Barto AG (1981) Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev 88:135-70 [PubMed]

Sutton RS, Barto AG (1990) Time-derivative models of Pavlovian reinforcement Learning and computational neuroscience: Foundations of adaptive networks, Gabriel M:Moore J, ed. pp.497

Sutton RS, Barto AG (1998) Reinforcement learning: an introduction [Journal]

   A reinforcement learning example (Sutton and Barto 1998) [Model]

Svenningsson P, Lindskog M, Rognoni F, Fredholm BB, Greengard P, Fisone G (1998) Activation of adenosine A2A and dopamine D1 receptors stimulates cyclic AMP-dependent phosphorylation of DARPP-32 in distinct populations of striatal projection neurons. Neuroscience 84:223-8

Svoboda K, Mainen ZF (1999) Synaptic [Ca2+]: intracellular stores spill their guts. Neuron 22:427-30 [PubMed]

Tang K, Low MJ, Grandy DK, Lovinger DM (2001) Dopamine-dependent synaptic plasticity in striatum during in vivo development. Proc Natl Acad Sci U S A 98:1255-60 [PubMed]

Tesauro G (1992) Practical issues in temporal difference learning Mach Learn 8:257-277

Tingley WG, Ehlers MD, Kameyama K, Doherty C, Ptak JB, Riley CT, Huganir RL (1997) Characterization of protein kinase A and protein kinase C phosphorylation of the N-methyl-D-aspartate receptor NR1 subunit using phosphorylation site-specific antibodies. J Biol Chem 272:5157-66 [PubMed]

Toan DL, Schultz W (1985) Responses of rat pallidum cells to cortex stimulation and effects of altered dopaminergic activity. Neuroscience 15:683-94

Touzet C (1999) Neural networks and Q-learning for robotics Available online at: http:--www.sciences-cognitives.org-scico-annuaire-Touzet_Claude-Touzet_IJCNN_Tut.pdf

Tremblay L, Hollerman JR, Schultz W (1998) Modifications of reward expectation-related neuronal activity during learning in primate striatum. J Neurophysiol 80:964-77 [Journal] [PubMed]

Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398:704-8 [PubMed]

Umemiya M, Raymond LA (1997) Dopaminergic modulation of excitatory postsynaptic currents in rat neostriatal neurons. J Neurophysiol 78:1248-55 [Journal] [PubMed]

van Rossum MC, Bi GQ, Turrigiano GG (2000) Stable Hebbian learning from spike timing-dependent plasticity. J Neurosci 20:8812-21 [PubMed]

van_Hemmen JL (2001) Theory of synaptic plasticity Handbook Of Biological Physics, neuro-informatics, neural modelling, Moss F:Gielen S, ed. pp.771

Walsh JP (1993) Depression of excitatory synaptic input in rat striatal neurons. Brain Res 608:123-8 [PubMed]

Walsh JP, Dunia R (1993) Synaptic activation of N-methyl-D-aspartate receptors induces short-term potentiation at excitatory synapses in the striatum of the rat. Neuroscience 57:241-8

Watkins CJCH (1989) Learning from delayed rewards Unpublished doctoral dissertation

Watkins CJCH, Dayan P (1992) Unpublished doctoral dissertation Mach Learn 8:279-292

West AE, Chen WG, Dalva MB, Dolmetsch RE, Kornhauser JM, Shaywitz AJ, Takasu MA, Tao X, Green (2001) Calcium regulation of neuronal gene expression. Proc Natl Acad Sci U S A 98:11024-31 [PubMed]

Whitford KL, Ghosh A (2001) Plexin signaling via off-track and rho family GTPases. Neuron 32:1-3 [PubMed]

Wickens JR, Begg AJ, Arbuthnott GW (1996) Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex in vitro. Neuroscience 70:1-5 [PubMed]

Widrow B, Hoff M (1960) Adaptive switching circuits Western Electronic Show And Convention Reco 4:96-104

Wiering M, Schmidhuber J (1998) Fast online Q(Lambda) Mach Learn 33:105-115

Williams SM, Goldman-Rakic PS (2000) Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. Cereb Cortex 3:199-222 [PubMed]

Witten IH (1977) An adaptive optimal controller for discrete-time Markov environments Information And Control 34:86-295

Xie X, Seung S (2000) Spike-based learning rules and stabilization of persistent neural activity. Advances in neural information processing systems, Solla SA, Leen T, Mu ller K-R, ed. pp.199

Yamamoto Y, Nakanishi H, Takai N, Shimazoe T, Watanabe S, Kita H (1999) Expression of N-methyl-D-aspartate receptor-dependent long-term potentiation in the neostriatal neurons in an in vitro slice after ethanol withdrawal of the rat. Neuroscience 91:59-68

Yang SN, Tang YG, Zucker RS (1999) Selective induction of LTP and LTD by postsynaptic [Ca2+]i elevation. J Neurophysiol 81:781-7 [Journal] [PubMed]

Yao H, Dan Y (2001) Stimulus timing-dependent plasticity in cortical processing of orientation. Neuron 32:315-23 [PubMed]

Yim CY, Mogenson GJ (1982) Response of nucleus accumbens neurons to amygdala stimulation and its modification by dopamine. Brain Res 239:401-15 [PubMed]

Zhang LI, Tao HW, Holt CE, Harris WA, Poo M (1998) A critical window for cooperation and competition among developing retinotectal synapses. Nature 395:37-44 [PubMed]

Zink CF, Pagnoni G, Martin ME, Dhamala M, Berns GS (2003) Human striatal response to salient nonrewarding stimuli. J Neurosci 23:8092-7

Zucker RS (1999) Calcium- and activity-dependent synaptic plasticity. Curr Opin Neurobiol 9:305-13 [PubMed]

Baras D, Meir R (2007) Reinforcement learning, spike-time-dependent plasticity, and the BCM rule. Neural Comput 19:2245-79 [PubMed]

Gruning A (2007) Elman backpropagation as reinforcement for simple recurrent networks. Neural Comput 19:3108-31 [PubMed]

Gurney KN, Humphries MD, Redgrave P (2015) A new framework for cortico-striatal plasticity: behavioural theory meets in vitro data at the reinforcement-action interface. PLoS Biol 13:e1002034 [Journal] [PubMed]

   Cortico-striatal plasticity in medium spiny neurons (Gurney et al 2015) [Model]

Izhikevich EM (2007) Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling. Cereb Cortex 17(10):2443-2452 [Journal] [PubMed]

   Linking STDP and Dopamine action to solve the distal reward problem (Izhikevich 2007) [Model]

Manninen T, Hituri K, Kotaleski JH, Blackwell KT, Linne ML (2010) Postsynaptic signal transduction models for long-term potentiation and depression. Front Comput Neurosci 4:152 [Journal] [PubMed]

Porr B, Worgotter F (2006) Strongly Improved Stability and Faster Convergence of Temporal Sequence Learning by Using Input Correlations Only Neural Comput 18:1380-1412

Porr B, Worgotter F (2007) Learning with "relevance": using a third factor to stabilize Hebbian learning. Neural Comput 19:2694-719 [Journal] [PubMed]

(256 refs)