References and models cited by this paper

References and models that cite this paper

Abbott LF, Gerstner W (2005) Homeostasis and learning through spike-timing dependent plasticity Methods and models in neurophysics: Proceedings of the Les Houches SummerSchool 2003, Gutkin B:Hansel D:Meunier C:Dalibard J:Chow C, ed.

Abbott LF, Nelson SB (2000) Synaptic plasticity: taming the beast. Nat Neurosci 3 Suppl:1178-83 [Journal] [PubMed]

Aizenman CD, Linden DJ (2000) Rapid, synaptically driven increases in the intrinsic excitability of cerebellar deep nuclear neurons. Nat Neurosci 3:109-11 [Journal] [PubMed]

Alstrom P, Stassinopoulos D (1995) Versatility and adaptive performance. Phys Rev E Stat Phys Plasmas Fluids Relat Interdiscip Topics 51:5027-5032 [PubMed]

Bartlett P, Baxter J (1999) Hebbian synaptic modifications in spiking neurons that learn Tech Rep

Bartlett P, Baxter J (2000) Stochastic optimization of controlled partially observable Markov decision processes Proc 39th IEEE Conf Decision and Control

Bartlett PL, Baxter J (1999) Direct gradient-based reinforcement learning: I. Gradient estimation algorithms Tech Rep Australian National University, Research School of Information Sciences and Engineering

Bartlett PL, Baxter J (2000) A biologically plausible and locally optimal learning algorithm for spiking neurons Available online at http:--arp.anu.edu.au-ftp-papers-jon-brains.pdf.gz

Bartlett PL, Baxter J (2000) Estimation and approximation bounds for gradient based reinforcement learning Proc 13th Ann Conf Comput Learn Theory :133-141

Barto AG (1985) Learning by statistical cooperation of self-interested neuron-like computing elements. Hum Neurobiol 4:229-56 [PubMed]

Barto AG, Anandan P (1985) Pattern-recognizing stochastic learning automata IEEE Trans Syst Man Cybernet 15:360-375

Barto AG, Anderson CW (1985) Structural learning in connectionist systems Proc 7th Ann Conf Cogn Sci Soc

Barto AG, Jordan MI (1987) Gradient following without back-propagation in layered networks Proc 1st IEEE Ann Conf Neural Networks, Caudill M:Butler C, ed. pp.629

Baxter J, Bartlett PL (2001) Infinite-horizon policy-gradient estimation J Artif Intell Res 15:319-350

Baxter J, Bartlett PL, Weaver L (2001) Experiments with infinite-horizon, policy-gradient estimation J Artif Intel Res 15:351-381

Baxter J, Weaver L, Bartlett PL (1999) Direct gradient-based reinforcement learning: II. Gradient ascent algorithms and experiments Tech Rep Australian National University, Research School of Information Sciences and Engineering

Bell AJ, Parra LC (2005) Maximising sensitivity in a spiking network Advances in neural information processing systems, Saul LK:Weiss Y:Bottou L, ed. pp.121

Bell CC, Han VZ, Sugawara Y, Grant K (1997) Synaptic plasticity in a cerebellum-like structure depends on temporal order. Nature 387:278-81 [Journal] [PubMed]

Bi GQ, Poo MM (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18:10464-72 [PubMed]

Bohte SM (2004) The evidence for neural information processing with precise spike-times: A survey Natural Comput 3:195-206

Bohte SM, Mozer MC (2005) Reducing spike train variability: A computational theory of spike-timing dependent plasticity Advances in neural information processing systems, Saul LK:Weiss Y:Bottou L, ed. pp.201

Chechik G (2003) Spike-timing-dependent plasticity and relevant mutual information maximization. Neural Comput 15:1481-510 [Journal] [PubMed]

Cudmore RH, Turrigiano GG (2004) Long-term potentiation of intrinsic excitability in LV visual cortical neurons. J Neurophysiol 92:341-8 [Journal] [PubMed]

Dan Y, Poo MM (1992) Hebbian depression of isolated neuromuscular synapses in vitro. Science 256:1570-3 [PubMed]

Dan Y, Poo MM (2004) Spike timing-dependent plasticity of neural circuits. Neuron 44:23-30 [Journal] [PubMed]

Daoudal G, Debanne D (2003) Long-term plasticity of intrinsic excitability: learning rules and mechanisms. Learn Mem 10:456-65 [Journal] [PubMed]

Dauce E, Soula H, Beslon G (2005) Learning methods for dynamic neural networks Proc 2005 Intl Symp Nonlinear Theory and Its Applications

Egger V, Feldmeyer D, Sakmann B (1999) Coincidence detection and changes of synaptic efficacy in spiny stellate neurons in rat barrel cortex. Nat Neurosci 2:1098-105 [Journal] [PubMed]

Farries MA, Fairhall AL (2005) Reinforcement learning with modulated spike timing-dependent plasticity Poster presented at the Computational and Systems Neuroscience Conference (COSYNE 2005). Available online at http:--www.cosyne.org-climages-d-dy-COSYNE05 Abstracts.pdf

Farries MA, Fairhall AL (2005) Reinforcement learning with modulated spike timing-dependent plasticity Program No. 384.3. 2005 Abstract Viewer-Itinerary Planner

Florian RV (2005) A reinforcement learning algorithm for spiking neural networks Proc 7th Intl Symp Symbolic and Numeric Algorithms for Scientific Computing, Zaharie D:Petcu D:Negru V:Jebelean T:Biobanu G:Cicortas A:Abraham A:Paprzycki M, ed. pp.299

Florian RV, Muresan RC (2006) Phase precession and recession with STDP and anti-STDP Proc 16th Intl Conf Artif Neural Networks

Froemke RC, Dan Y (2002) Spike-timing-dependent synaptic modification induced by natural spike trains. Nature 416:433-8 [Journal] [PubMed]

Ganguly K, Kiss L, Poo M (2000) Enhancement of presynaptic neuronal excitability by correlated presynaptic and postsynaptic spiking. Nat Neurosci 3:1018-26 [Journal] [PubMed]

Gerstner W (2001) A framework for spiking neuron models: The spike response method The Handbook of Biological Physics, Gielen S et al, ed. pp.469

Gerstner W, Kistler WM (2002) Spiking neuron models

Gütig R, Aharonov R, Rotter S, Sompolinsky H (2003) Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity. J Neurosci 23:3697-714 [PubMed]

Han VZ, Grant K, Bell CC (2000) Reversible associative depression and nonassociative potentiation at a parallel fiber synapse. Neuron 27:611-22 [PubMed]

Hopfield JJ, Brody CD (2004) Learning rules and network repair in spike-timing-based computation networks. Proc Natl Acad Sci U S A 101:337-42 [Journal] [PubMed]

Huang YY, Simpson E, Kellendonk C, Kandel ER (2004) Genetic evidence for the bidirectional modulation of synaptic plasticity in the prefrontal cortex by D1 receptors. Proc Natl Acad Sci U S A 101:3236-41 [Journal] [PubMed]

Kempter R, Gerstner W, van Hemmen JL (2001) Intrinsic stabilization of output rates by spike-based Hebbian learning. Neural Comput 13:2709-41 [Journal] [PubMed]

Kempter R, Gerstner W, van_Hemmen JL (1999) Hebbian learning and spiking neurons Physical Review E 59:4498-4514 [Journal]

Legenstein R, Naeger C, Maass W (2005) What can a neuron learn with spike-timing-dependent plasticity? Neural Comput 17:2337-82 [Journal] [PubMed]

Li CY, Lu JT, Wu CP, Duan SM, Poo MM (2004) Bidirectional modification of presynaptic neuronal excitability accompanying spike timing-dependent synaptic plasticity. Neuron 41:257-68 [PubMed]

Lin YW, Min MY, Chiu TH, Yang HW (2003) Enhancement of associative long-term potentiation by activation of beta-adrenergic receptors at CA1 synapses in rat hippocampal slices. J Neurosci 23:4173-81 [PubMed]

Marbach P, Tsitsiklis JN (1999) Simulation-based optimization of Markov reward processes: Implementation issues Proc 38th Conf Decision and Control

Marbach P, Tsitsiklis JN (2000) Approximate gradient methods in policy-space optimization of Markov reward processes Discrete Event Dynamic Systems: Theory and Applications 13:111-148

Markram H, Lübke J, Frotscher M, Sakmann B (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275:213-5 [PubMed]

Mazzoni P, Andersen RA, Jordan MI (1991) A more biologically plausible learning rule for neural networks. Proc Natl Acad Sci U S A 88:4433-7 [PubMed]

Neutze R (1995) Ring interferometer with angular acceleration. Phys Rev A 51:5039-5042 [PubMed]

Nick TA, Ribera AB (2000) Synaptic activity modulates presynaptic excitability. Nat Neurosci 3:142-9 [Journal] [PubMed]

Pfister JP, Toyoizumi T, Barber D, Gerstner W (2006) Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning. Neural Comput 18:1318-48 [Journal] [PubMed]

Pouget A, Deffayet C, Sejnowski TJ (1995) Reinforcement learning predicts the site of plasticity for auditory remapping in the barn owl Advances in neural information processing systems, Fritzke B:Tesauro G:Touretzky DS:Leen TK, ed.

Press WH, Teukolsky SA, Vellerling WT, Flannery BP (1992) Numerical Recipes In C: The Art Of Scientific Computing

Rao RP, Sejnowski TJ (2001) Spike-timing-dependent Hebbian plasticity as temporal difference learning. Neural Comput 13:2221-37 [Journal] [PubMed]

Roberts PD (1999) Computational consequences of temporally asymmetric learning rules: I. Differential hebbian learning. J Comput Neurosci 7:235-46 [Journal] [PubMed]

Roberts PD, Bell CC (2002) Spike timing dependent synaptic plasticity in biological systems. Biol Cybern 87:392-403 [Journal] [PubMed]

Schultz W (2002) Getting formal with dopamine and reward. Neuron 36:241-63 [PubMed]

Seamans JK, Yang CR (2004) The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Prog Neurobiol 74:1-58 [Journal] [PubMed]

Seung HS (2003) Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40:1063-73 [PubMed]

Song S, Miller KD, Abbott LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 3:919-26 [Journal] [PubMed]

Soula H, Alwan A, Beslon G (2004) Obstacle avoidance learning in a spiking neural network Poster presented at Last Minute Results of Simulation of Adaptive Behavior

Soula H, Alwan A, Beslon G (2005) Learning at the edge of chaos: Temporal coupling of spiking neuron controller of autonomous robotic Proc AAAI Spring Symposia on Developmental Robotics

Stassinopoulos D, Bak P (1996) Democratic reinforcement: Learning via self organization Available online at http:--arxiv.org-abs-cond-mat-9601113

Strosslin T, Gerstner W (2003) Reinforcement learning in continuous state and action space Available online at http:--lenpe7.epfl.ch-stroessl-publications-StrosslinGe03.pdf

Sutton RS (1988) Learning to predict by the method of temporal diferences Machine Learning 3:9-44

Sutton RS, Barto AG (1998) Reinforcement learning: an introduction [Journal]

Takita K, Hagiwara M (2002) A pulse neural network learning algorithm for POMDP environment Proc 2002 Intl Joint Conf Neural Networks :1643-1648

Takita K, Hagiwara M (2005) A pulse neural network reinforcement learning algorithm for partially observable Markov decision processes Systems And Computers In Japan 36:42-52

Takita K, Osana Y, Hagiwara M (2001) Reinforcement learning algorithm with network extension for pulse neural network Trans Institute Electrical Engineers Of Japan 121:1634-1640

Thiel CM, Friston KJ, Dolan RJ (2002) Cholinergic modulation of experience-dependent plasticity in human auditory cortex. Neuron 35:567-74 [PubMed]

Toyoizumi T, Pfister JP, Aihara K, Gerstner W (2005) Spike-timing dependent plasticity and mutual information maximization for a spiking neuron model Advances in neural information processing systems, Saul LK:Weiss Y:Bottou L, ed. pp.1409

Turrigiano GG, Nelson SB (2004) Homeostatic plasticity in the developing nervous system. Nat Rev Neurosci 5:97-107 [Journal] [PubMed]

Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning Mach Learn 8:229-256

Xie X, Seung HS (2004) Learning in neural networks by reinforcement of irregular spiking. Phys Rev E Stat Nonlin Soft Matter Phys 69:041909 [Journal] [PubMed]

Xie X, Seung S (2000) Spike-based learning rules and stabilization of persistent neural activity. Advances in neural information processing systems, Solla SA, Leen T, Mu ller K-R, ed. pp.199

Zhang W, Linden DJ (2003) The other side of the engram: experience-driven changes in neuronal intrinsic excitability. Nat Rev Neurosci 4:885-900 [Journal] [PubMed]

Brzosko Z, Zannone S, Schultz W, Clopath C, Paulsen O (2017) Sequential neuromodulation of Hebbian plasticity offers mechanism for effective reward-based navigation. Elife [Journal] [PubMed]

Chadderdon GL, Neymotin SA, Kerr CC, Lytton WW (2012) Reinforcement learning of targeted movement in a spiking neuronal model of motor cortex. PLoS One 7:e47251 [Journal] [PubMed]

Legenstein R, Pecevski D, Maass W (2008) A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput Biol 4:e1000180 [Journal] [PubMed]

Mozafari M, Kheradpisheh SR, Masquelier T, Nowzari-Dalini A, Ganjtabesh M (2018) First-Spike-Based Visual Categorization Using Reward-Modulated STDP IEEE Transactions on Neural Networks and Learning Systems :1-13 [Journal]

Nakano T, Otsuka M, Yoshimoto J, Doya K (2015) A spiking neural network model of model-free reinforcement learning with high-dimensional sensory input and perceptual ambiguity. PLoS One 10:e0115620 [Journal] [PubMed]

Neymotin SA, Chadderdon GL, Kerr CC, Francis JT, Lytton WW (2013) Reinforcement learning of two-joint virtual arm reaching in a computer model of sensorimotor cortex. Neural Comput 25:3263-93 [Journal] [PubMed]

Richmond P, Buesing L, Giugliano M, Vasilaki E (2011) Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations. PLoS One 6:e18539 [Journal] [PubMed]

Rivest F, Kalaska JF, Bengio Y (2010) Alternative time representation in dopamine models. J Comput Neurosci 28:107-30 [Journal] [PubMed]

Zannone S, Brzosko Z, Paulsen O, Clopath C (2018) Acetylcholine-modulated plasticity in reward-driven navigation: a computational study. Sci Rep 8:9486 [Journal] [PubMed]