Sensorimotor cortex reinforcement learning of 2-joint virtual arm reaching (Neymotin et al. 2013)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:150245
"... We developed a model of sensory and motor neocortex consisting of 704 spiking model-neurons. Sensory and motor populations included excitatory cells and two types of interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a 2-joint virtual arm to reach to a fixed target. ... "
Reference:
1 . Neymotin SA, Chadderdon GL, Kerr CC, Francis JT, Lytton WW (2013) Reinforcement learning of 2-joint virtual arm reaching in a computer model of sensorimotor cortex Neural Computation 25(12):3263-93 [PubMed]
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex V1 pyramidal corticothalamic L6 cell; Neocortex U1 pyramidal intratelencephalic L2-5 cell; Neocortex V1 interneuron basket PV cell; Neocortex fast spiking (FS) interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON;
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing;
Implementer(s): Neymotin, Sam [samn at neurosim.downstate.edu]; Chadderdon, George [gchadder3 at gmail.com];
Search NeuronDB for information about:  Neocortex V1 pyramidal corticothalamic L6 cell; Neocortex V1 interneuron basket PV cell; Neocortex U1 pyramidal intratelencephalic L2-5 cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
/
a2dmodeldb
readme.html
drspk.mod *
infot.mod *
intf6_.mod *
misc.mod *
nstim.mod *
stats.mod *
vecst.mod *
arm.hoc
basestdp.hoc
col.hoc
colors.hoc *
declist.hoc *
decmat.hoc *
decnqs.hoc *
decvec.hoc *
default.hoc *
drline.hoc *
filtutils.hoc *
geom.hoc
grvec.hoc *
hinton.hoc *
infot.hoc *
init.hoc
labels.hoc *
misc.h *
mosinit.hoc
network.hoc
nload.hoc
nqs.hoc *
nqsnet.hoc *
nrnoc.hoc *
params.hoc
python.hoc
pywrap.hoc *
run.hoc
samutils.hoc *
screenshot.png
sense.hoc *
setup.hoc *
simctrl.hoc *
stats.hoc *
stim.hoc
syncode.hoc *
trainedplast.nqs
units.hoc *
xgetargs.hoc *
                            
// $Id: infot.hoc,v 1.43 2009/12/04 01:25:55 samn Exp $ 

print "Loading infot.hoc..."

if(!installed_infot) {install_infot()}

//* tentropsig(v1,v2,nshuf,nbins,twoway,[xpast,ypast,hval])
//significance test of transfer entropy using shuffling
//returns (te - tes) / sds , where te is transfer entropy, tes is transfer entropy
//of shuffled data, sds is std-dev of transfer entrop of shuffled data
//should only accept as significanat values > 4-6
func tentropsig () { local nshuf,i,xp,yp,hv,nbins,te localobj v1,v2,vo
  v1=new Vector() v2=new Vector() vo=new Vector(1)
  v1.copy($o1) v2.copy($o2) nshuf=$3 nbins=$4
  if(numarg()>4) xp=$5 else xp=1
  if(numarg()>5) yp=$6 else yp=2
  if(numarg()>6) hv=$7 else hv=0
  te=v1.tentrop(v2,nbins,xp,yp,nshuf,vo,hv)
  if(1||verbose_infot>2) printf("te=%g,sig=%g\n",te,vo.x(0))
  return vo.x(0)
}

//** mutinfbshufv(v1,v2,[nshuf,nbins])
//return vector with mutual information from shuffled v1,v2
//used for significance test , i.e. : ((miorig - mishufmean) / mishufstdev) > 2
obfunc mutinfbshufv () { local nshuf,nbins,i localobj v1,v2,ve
  v1=new Vector() v2=new Vector() ve=new Vector()
  v1.copy($o1) v2.copy($o2)
  if(numarg()>2) nshuf=$3 else nshuf=20
  if(numarg()>3) nbins=$4 else nbins=10
  for i=0,nshuf-1 {
    v1.shuffle() v2.shuffle()
    ve.append(v1.mutinfb(v2,nbins))
  }
  return ve
}

//** mutinfbsig(v1,v2,[nshuf,nbins])
//get significance of mutual information, should be at least > 2
func mutinfbsig () { local nshuf,nbins,st localobj ve
  if(numarg()>2) nshuf=$3 else nshuf=20
  if(numarg()>3) nbins=$4 else nbins=10
  ve=mutinfbshufv($o1,$o2,nshuf,nbins)
  st=ve.stdev
  if(st<=0) st=1
  return ($o1.mutinfb($o2,nbins) - ve.mean) / st
}

//** tentropspksig(v1,v2,nshuffles)
//get significance of tentropspks using shuffling
//returns (TE - AvgTEShuffle) / StdDevTEShuffle
func tentropspksig () { local nshuf,i,xp,yp,hv,nbins,te,sd localobj v1,v2,ve
  v1=new Vector() v2=new Vector() ve=new Vector()
  v1.copy($o1) v2.copy($o2) nshuf=$3
  te=$o1.tentropspks($o2)
  for i=0,nshuf-1 {
    v1.shuffle()     ve.append(v1.tentropspks(v2))
  }
  if(verbose_infot>2) printf("te=%g,ve.mean=%g,ve.stdev=%g\n",te,ve.mean,ve.stdev)
  if(verbose_infot>2) ve.printf
  sd=ve.stdev()
  if(sd<=0)sd=1
  return (te-ve.mean)/sd
}

//* normte() get normalized transfer entropy using tentropspks in output vector vo
//vo.x(0)=transfer entropy of $o1->$o2
//vo.x(1)=H($o2Future|$o2Past)
//vo.x(2)=normalized transfer entropy in 0,1 range
//$3==number of shuffles
//$o1,$o2 should both have same size and non-negative values. this func is meant for time-binned spike train data
obfunc normte () { local a localobj ve,vo
  a=allocvecs(ve) vo=new Vector()
  nshuf=0
  nshuf=$3 vrsz(3+nshuf,vo) 
  te=$o1.tentropspks($o2,vo,nshuf)
  if(verbose_infot>2) vo.printf
  if(vo.x(1)<=0 && verbose_infot>0){printf("WARNING H(X2F|X2P)==%g<=0\n",vo.x(1)) vo.x(1)=1 }
  if (nshuf>0) {
    ve.copy(vo,3,vo.size-1)
    vo.resize(4)
    if (ve.mean!=vo.x[2]) printf("normte ERRA\n")
    vo.append(ve.stdev)
  } 
  vo.x[2]=te
  dealloc(a)
  return vo
}

//* GetTENQ() get an nqs with useful transfer entropy info
obfunc GetTENQ () { local te01,te10,pf01,pf10 localobj nqte,vo1,vo2
  if(numarg()>3) nqte=$o4
  if(nqte==nil) {
    nqte=new NQS("from","to","TE","NTE","HX2|X2P","prefdir","TEshufavg","TEshufstd","sig")
  } else nqte.clear()
  vo1=normte($o1,$o2,$3)
  vo2=normte($o2,$o1,$3)
  te01=vo1.x(2)
  te10=vo2.x(2)
  if(vo1.x(4)<=0)vo1.x(4)=1
  if(vo2.x(4)<=0)vo2.x(4)=1
  if(te01>0 || te10>0) {
    pf01=(te01-te10)/(te01+te10)
    pf10=(te10-te01)/(te01+te10)
  } else {
    pf01=pf10=0
  }
nqte.append(0,1,vo1.x(0),te01,vo1.x(1),pf01,vo1.x(3),vo1.x(4),(vo1.x(0)-vo1.x(3))/vo1.x(4))
nqte.append(1,0,vo2.x(0),te10,vo2.x(1),pf10,vo2.x(3),vo2.x(4),(vo2.x(0)-vo2.x(3))/vo2.x(4))
  return nqte
}

//** prefdte() get preferred direction of transfer entropy
//$o1=vec 1, $o2=vec 2, $3 = # of times to shuffle
func prefdte () { local nshuf,a,te01,te10,pfd localobj v1,v2,vtmp
  a=allocvecs(v1,v2,vtmp)
  v1.copy($o1) v2.copy($o2) nshuf=$3 vtmp.resize(3)
  v1=normte($o1,$o2,nshuf)
  v2=normte($o2,$o1,nshuf)
  te01=v1.x(2)
  te10=v2.x(2)
  pfd=(te01-te10)/(te01+te10)
  dealloc(a)
  return pfd
}

//** mkchist() averages entries in window into disc values and returns in new output vec
//$o1=input vec,$2=win size
obfunc mkchist () { local idx,eidx,wsz localobj vin,vout
  vin=$o1 wsz=$2 vout=new Vector() 
  vout.resize(1+vin.size/wsz) vout.resize(0)
  for(idx=0;idx<=vin.size;idx+=wsz) {
    eidx=idx+wsz-1
    if(eidx>=vin.size)eidx=vin.size-1
    if(eidx>idx) vout.append( int(vin.mean(idx,eidx)) )
  }
  return vout
}

//get magnitude of difference in a preferred direction - just abs of diff, but if theyre both neg, return 0
//$1 = nTE_X->Y
//$2 = nTE_Y->X
func prefdmag () { local n1,n2,s
  n1=$1 n2=$2
  if(n1>0 && n2<=0) return n1-n2 //n1 is relatively strong
  if(n2>0 && n1<=0) return n2-n1 //n2 is relatively strong
  if(n2<0 && n1<0) return 0      //both are weak
  return abs(n1-n2)              //both are weak positive
}


//** simple test for nte
{declare("vb","o[2]","vs","o[2]")}
for i=0,1 {
  vb[i]=new Vector()
  vs[i]=new Vector()
}

//mkspktrain(Random,rate,tmax) -- make a spike train with specified rate,tmax
//Random obj must be initialized
obfunc mkspktrain () { local tmax,rate,t,dt,intt localobj rdp,vs
  rdp=$o1 rate=$2 tmax=$3
  intt=1e3/rate
  t = 0
  vs=new Vector()
  while(t<=tmax) {
    dt = rdp.poisson(intt)
    t += dt
    vs.append(t)
  }
  return vs
}

//make random spikes with frequency $1, tmax=$2, offset for spikes=$3, alpha=$4 -- ratio of spikes from
//vs[0] that get placed in vs[1] 
//spikes in vs[0] are randomly picked, spikes in vs[1] are same as in vs[0] but shifted forward by $3 offset
//so vs[0] 'drives' vs[1], or can be used to predict it, but vs[1] cant be used to predict vs[0]
proc mkspks () { local tmax,rate,t,dt,intt,off,i,alpha localobj rdp
  rate=$1 tmax=$2 off=$3 
  if(numarg()>3)alpha=$4 else alpha=1
  intt=1e3/rate
  rdp=new Random()
  rdp.ACG(1234)
  rdp.poisson(intt)
  for i=0,1 vs[i].resize(0)
  vs[0]=mkspktrain(rdp,rate,tmax)
  if(alpha < 1.0) {
    for vtr(&t,vs[0]) if(rdp.uniform(0,1) <= alpha) vs[1].append(t+off)
  } else {
    vs[1].copy(vs[0])
    vs[1].add(off)
  }
}
//test nTE : nTE of X0 -> X1 should be much higher than nTE of X1 -> X0
//optional $1=offset == offset to shift spikes by, in ms
//optional $2=rate == rate of spikes, in Hz
//optional $3=bin size , in ms
//optional $4=alpha == ratio of spikes of X0 that get placed in X1 with offset
//optional $5=max time, in ms
func testnte () { local a,i,bisv,maxt,alpha,off,rate,binsz,dur  localobj nqt,nqout
  if(numarg()>0)off=$1 else off=10
  if(numarg()>1)rate=$2 else rate=50
  if(numarg()>2)binsz=$3 else binsz=10
  if(numarg()>3)alpha=$4 else alpha=1
  if(numarg()>4)dur=$5 else dur=10000
  bisv=binmin_infot binmin_infot=0
  print "output should be close to:\n\t0 1 0.6707 0.9975 0.6707 0.9407 0.003435 0.001672 399.1"
  print "\t1 0 0.02183 0.03049 0.6685 -0.9407 0.002828 0.001442 13.17"
  mkspks(rate,dur,off,alpha)
  maxt=vs[1].max
  printf("maxt=%g\n",maxt)
  if(vs[1].max>maxt)maxt=vs[1].max
  for i=0,1 vb[i].hist(vs[i],0,(maxt+binsz-1)/binsz,binsz)
  nqt=GetTENQ(vb[0],vb[1],200) 
nqout=new NQS("X1","X2")
nqout.odec("X1")
nqout.odec("X2")
batch_flag=1
nqout.append(vb[0],vb[1])
nqout.sv("/u/samn/bpftest/data/09dec17.func.testnte.nqs")
batch_flag=0
  nqt.pr
  nqsdel(nqt)
  binmin_infot=bisv
  return 1
}

//get kernel smoothed prob distrib in an nqs
//$o1=input vector
//$2=increment in x , smaller values mean finer resolution
//$3=bandwidth - higher means smoother output
// $4=min value in output, $5=max value in output
obfunc khist () { local min,max,inc,h,x,i,s localobj vx,vy,nq,vin
  vin=$o1 
  if(numarg()>1)inc=$2 else inc=0.1
  if(numarg()>2)h=$3 else h=vin.getbandwidth()
  if(numarg()>3)min=$4 else min=vin.min()
  if(numarg()>4)max=$5 else max=vin.max()
  {vx=new Vector() vy=new Vector()}
  vx.indgen(min,max,inc)
  vy.copy(vx)
  for vtr(&x,vx,&i) vy.x(i) = vin.kprob1D(h,x)
  s=vy.sum 
  if(s!=0) vy.div(vy.sum)
  nq=new NQS("x","y")
  nq.v[0]=vx
  nq.v[1]=vy
  return nq
}

Neymotin SA, Chadderdon GL, Kerr CC, Francis JT, Lytton WW (2013) Reinforcement learning of 2-joint virtual arm reaching in a computer model of sensorimotor cortex Neural Computation 25(12):3263-93[PubMed]

References and models cited by this paper

References and models that cite this paper

Afshar A, Santhanam G, Yu BM, Ryu SI, Sahani M, Shenoy KV (2011) Single-trial neural correlates of arm movement preparation. Neuron 71:555-64

Almassy N, Edelman GM, Sporns O (1998) Behavioral constraints in the development of neuronal properties: a cortical model embedded in a real-world device. Cereb Cortex 8:346-61 [PubMed]

Bannister AP (2005) Inter- and intra-laminar connections of pyramidal cells in the neocortex. Neurosci Res 53:95-103 [PubMed]

Bedau MA (2005) Artificial life: more than just building and studying computational systems. Artif Life 11:1-3 [PubMed]

Berthier N (2011) The syntax of human infant reaching 8th International Conference on Complex Systems :1477-1487

Berthier NE, Clifton RK, McCall DD, Robin DJ (1999) Proximodistal structure of early reaching in human infants. Exp Brain Res 127:259-69

Carnevale NT, Hines ML (2006) The NEURON Book

Chadderdon GL, Neymotin SA, Kerr CC, Lytton WW (2012) Reinforcement learning of targeted movement in a spiking neuronal model of motor cortex PLoS ONE 2012 7(10):e47251 [Journal]

   Reinforcement learning of targeted movement (Chadderdon et al. 2012) [Model]

Cools R (2006) Dopaminergic modulation of cognitive function-implications for L-DOPA treatment in Parkinson's disease. Neurosci Biobehav Rev 30:1-23 [PubMed]

Corbetta D, Snapp-Childs W (2009) Seeing and touching: the role of sensory-motor experience on the development of infant reaching. Infant Behav Dev 32:44-58

De Schutter E (2008) Why are computational neuroscience and systems biology so separate? PLoS Comput Biol 4:e1000078 [Journal] [PubMed]

Dyhrfjeld-Johnsen J, Santhakumar V, Morgan RJ, Huerta R, Tsimring L, Soltesz I (2007) Topological determinants of epileptogenesis in large-scale structural and functional models of the dentate gyrus derived from experimental data. J Neurophysiol 97:1566-87 [Journal] [PubMed]

   Dentate gyrus (Morgan et al. 2007, 2008, Santhakumar et al. 2005, Dyhrfjeld-Johnsen et al. 2007) [Model]

Edelman GM (1987) Neural Darwinism: The Theory of Neural Group Selection

Edelman GM (2004) Wider than the sky: The phenomenal gift of consciousness

Engel A, Konig P, Kreiter A, Gray C, Singer W (1991) Temporal coding by coherent oscillations as a potential solution to the binding problem: physiological evidence Nonlinear dynamics and neural networks, Schuster HG, ed.

Evans RC, Morera-Herreras T, Cui Y, Du K, Sheehan T, Kotaleski JH, Venance L, Blackwell KT (2012) The effects of NMDA subunit composition on calcium influx and spike timing-dependent plasticity in striatal medium spiny neurons. PLoS Comput Biol 8:e1002493 [Journal] [PubMed]

   NMDA subunit effects on Calcium and STDP (Evans et al. 2012) [Model]

Farries MA, Fairhall AL (2007) Reinforcement learning with modulated spike timing dependent synaptic plasticity. J Neurophysiol 98:3648-65 [PubMed]

Fenton AA, Lytton WW, Barry JM, Lenck-Santini PP, Zinyuk LE, Kubik S, Bures J, Poucet B, Mull (2010) Attention-like modulation of hippocampus place cell discharge. J Neurosci 30:4613-25

Florian RV (2007) Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Comput 19:1468-502 [PubMed]

Frank MJ, O'reilly RC (2006) A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci 120:497-517 [PubMed]

Frank MJ, Seeberger LC, O`Reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940-3 [Journal] [PubMed]

   Dynamic dopamine modulation in the basal ganglia: Learning in Parkinson (Frank et al 2004,2005) [Model]

Gourevitch B, Eggermont JJ (2007) Evaluating information transfer between auditory cortical neurons. J Neurophysiol 97:2533-43 [PubMed]

Graybiel AM, Aosaki T, Flaherty AW, Kimura M (1994) The basal ganglia and adaptive motor control. Science 265:1826-31 [PubMed]

Hikosaka O, Nakamura K, Sakai K, Nakahara H (2002) Central mechanisms of motor skill learning. Curr Opin Neurobiol 12:217-22

Hosp JA, Pekanovic A, Rioult-Pedotti MS, Luft AR (2011) Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J Neurosci 31:2481-7 [PubMed]

Houk JC, Wise SP (2004) Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb Cortex 5:95-110 [PubMed]

Izhikevich EM (2007) Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling. Cereb Cortex 17(10):2443-2452 [Journal] [PubMed]

   Linking STDP and Dopamine action to solve the distal reward problem (Izhikevich 2007) [Model]

Jones SR, Kerr CE, Wan Q, Pritchett DL, Hamalainen M, Moore CI (2010) Cued spatial attention drives functionally relevant modulation of the mu rhythm in primary somatosensory cortex. J Neurosci 30:13760-5 [PubMed]

Kelemen E, Fenton AA (2010) Dynamic grouping of hippocampal neural activity during cognitive control of two spatial frames. PLoS Biol 8:e1000403 [PubMed]

Kerr CC, Neymotin SA, Chadderdon GL, Fietkiewicz CT, Francis JT, Lytton WW (2012) Electrostimulation as a prosthesis for repair of information flow in a computer model of neocortex IEEE Transactions on Neural Systems & Rehabilitation Engineering 20(2):153-60 [Journal] [PubMed]

   Prosthetic electrostimulation for information flow repair in a neocortical simulation (Kerr 2012) [Model]

Kerr CC, Van Albada SJ, Neymotin SA, Chadderdon GL, Robinson PA, Lytton WW (2013) Cortical information flow in Parkinson's disease: a composite network-field model. Front Comput Neurosci 7:39:1-14 [Journal] [PubMed]

   Composite spiking network/neural field model of Parkinsons (Kerr et al 2013) [Model]

Kubikova L, Kostal L (2010) Dopaminergic system in birdsong learning and maintenance. J Chem Neuroanat 39:112-23

Le Novere N (2007) The long journey to a Systems Biology of neuronal function. BMC Syst Biol 1:28-23

Luft AR, Schwarz S (2009) Dopaminergic signals in primary motor cortex. Int J Dev Neurosci 27:415-21 [PubMed]

Lungarella M, Sporns O (2006) Mapping information flow in sensorimotor networks. PLoS Comput Biol 2:e144 [PubMed]

Lytton WW (2008) Computer modelling of epilepsy. Nat Rev Neurosci 9:626-37 [Journal] [PubMed]

Lytton WW, Neymotin SA, Hines ML (2008) The virtual slice setup. J Neurosci Methods 171:309-15 [Journal] [PubMed]

   The virtual slice setup (Lytton et al. 2008) [Model]

Lytton WW, Omurtag A (2007) Tonic-clonic transitions in computer simulation. J Clin Neurophysiol 24:175-81 [PubMed]

   Tonic-clonic transitions in a seizure simulation (Lytton and Omurtag 2007) [Model]

Lytton WW, Omurtag A, Neymotin SA, Hines ML (2008) Just in time connectivity for large spiking networks Neural Comput 20(11):2745-56 [Journal] [PubMed]

   JitCon: Just in time connectivity for large spiking networks (Lytton et al. 2008) [Model]

Lytton WW, Stewart M (2005) A rule-based firing model for neural networks Int J Bioelectromagn 7:47-50

Lytton WW, Stewart M (2006) Rule-based firing for network simulations. Neurocomputing 69:1160-1164

Mahmoudi B, Sanchez JC (2011) A symbiotic brain-machine interface through value-based decision making. PLoS One 6:e14760-23

Marsh B, Tarigoppula A, Francis J (2011) Correlates of reward expectation in the primary motor cortex: Developing an actor-critic model in macaques for a brain computer interface Society for Neuroscience Abstracts, 41

Mo J, Schroeder CE, Ding M (2011) Attentional modulation of alpha oscillations in macaque inferotemporal cortex. J Neurosci 31:878-82

Molina-Luna K, Pekanovic A, Rohrich S, Hertler B, Schubring-Giese M, Rioult-Pedotti MS, Luft (2009) Dopamine in motor cortex is necessary for skill learning and synaptic plasticity. PLoS One 4:e7082-21 [PubMed]

Neymotin S, Kerr C, Francis J, Lytton W (2011) Training oscillatory dynamics with spike-timing-dependent plasticity in a computer model of neocortex Signal Processing in Medicine and Biology Symposium (SPMB), IEEE :1-6

Neymotin SA, Jacobs KM, Fenton AA, Lytton WW (2011) Synaptic information transfer in computer models of neocortical columns. J Comput Neurosci. 30(1):69-84 [Journal] [PubMed]

   Synaptic information transfer in computer models of neocortical columns (Neymotin et al. 2010) [Model]

Neymotin SA, Lazarewicz MT, Sherif M, Contreras D, Finkel LH, Lytton WW (2011) Ketamine disrupts theta modulation of gamma in a computer model of hippocampus Journal of Neuroscience 31(32):11733-11743 [Journal] [PubMed]

   Ketamine disrupts theta modulation of gamma in a computer model of hippocampus (Neymotin et al 2011) [Model]

Neymotin SA, Lee H, Park E, Fenton AA, Lytton WW (2011) Emergence of physiological oscillation frequencies in a computer model of neocortex. Front Comput Neurosci 5:19-75 [Journal] [PubMed]

   Emergence of physiological oscillation frequencies in neocortex simulations (Neymotin et al. 2011) [Model]

Pastalkova E, Serrano P, Pinkhasova D, Wallace E, Fenton AA, Sacktor TC (2006) Storage of spatial information by the maintenance mechanism of LTP. Science 313:1141-4 [PubMed]

Peterson BE, Healy MD, Nadkarni PM, Miller PL, Shepherd GM (1996) ModelDB: an environment for running and storing computational models and their results applied to neuroscience. J Am Med Inform Assoc 3:389-98 [Journal] [PubMed]

Potjans W, Morrison A, Diesmann M (2009) A spiking neural network model of an actor-critic learning agent. Neural Comput 21:301-39 [PubMed]

Qiu S, Anderson CT, Levitt P, Shepherd GM (2011) Circuit-specific intracortical hyperconnectivity in mice with deletion of the autism-associated Met receptor tyrosine kinase. J Neurosci 31:5855-64

Reid RC (2012) From functional architecture to functional connectomics. Neuron 75:209-17

Reynolds JN, Wickens JR (2005) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 15:507-21 [PubMed]

Roberts PD, Bell CC (2002) Spike timing dependent synaptic plasticity in biological systems. Biol Cybern 87:392-403 [PubMed]

Rowan MS,Neymotin SA (2013) Synaptic Scaling Balances Learning in a Spiking Model of Neocortex Adaptive and Natural Computing Algorithms, Tomassini M, Antonioni A, Daolio F, Buesser P, ed. pp.20 [Journal]

   Synaptic scaling balances learning in a spiking model of neocortex (Rowan & Neymotin 2013) [Model]

Sanes JN (2003) Neocortical mechanisms in motor learning. Curr Opin Neurobiol 13:225-31

Seung HS (2003) Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40:1063-73 [PubMed]

Shadmehr R, Krakauer JW (2008) A computational neuroanatomy for motor control. Exp Brain Res 185:359-81 [PubMed]

Shadmehr R, Wise S (2005) The computational neurobiology of reaching and pointing: a foundation for motor learning

Shen W, Flajolet M, Greengard P, Surmeier DJ (2008) Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321:848-51 [PubMed]

Shepherd G (2004) The synaptic organization of the brain, Shepherd GM, ed.

Sober SJ, Brainard MS (2009) Adult birdsong is actively maintained by error correction. Nat Neurosci 12:927-31 [PubMed]

Song S, Miller KD, Abbott LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 3:919-26 [PubMed]

Sporns O, Tononi G, Kotter R (2005) The human connectome: A structural description of the human brain. PLoS Comput Biol 1:e42-308

Thomson AM, Lamy C (2007) Functional maps of neocortical local circuitry. Front Neurosci 1:19-42 [PubMed]

Thomson AM, West DC, Wang Y, Bannister AP (2002) Synaptic connections and small circuits involving excitatory and inhibitory neurons in layers 2-5 of adult rat and cat neocortex: triple intracellular recordings and biocytin labelling in vitro. Cereb Cortex 12:936-53 [PubMed]

Thorndike E (1911) Animal intelligence

Tiesinga PH, Sejnowski TJ (2004) Rapid temporal modulation of synchrony by competition in cortical interneuron networks. Neural Comput 16:251-75 [PubMed]

Uhlhaas PJ, Singer W (2006) Neural synchrony in brain disorders: relevance for cognitive dysfunctions and pathophysiology. Neuron 52:155-68 [PubMed]

von der Malsburg C, Schneider W (1986) A neural cocktail-party processor. Biol Cybern 54:29-40 [PubMed]

von Kraus LM, Sacktor TC, Francis JT (2010) Erasing sensorimotor memories via PKMzeta inhibition. PLoS One 5:e11125-81

Von_hofsten C (1979) Development of visually directed reaching: The approach phase Department Of Psychology, University Of Uppsala [psykologiska Inst , Uppsala Univ ]

Webb B (2000) What does robotics offer animal behaviour? Anim Behav 60:545-558

Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring behavior via inverse neurocontroller in a lesioned cortical spiking model driving a virtual arm. Front. Neurosci. Neuroprosthetics 10:28 [Journal]

   Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015) [Model]

Dura-Bernal S, Neymotin SA, Kerr CC, Sivagnanam S, Majumdar A, Francis JT, Lytton WW (2017) Evolutionary algorithm optimization of biological learning parameters in a biomimetic neuroprosthesis. IBM Journal of Research and Development (Computational Neuroscience special issue) 61(2/3):6:1-6:14 [Journal]

   Motor system model with reinforcement learning drives virtual arm (Dura-Bernal et al 2017) [Model]

Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [Journal] [PubMed]

   Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015) [Model]

Eguchi A, Neymotin SA and Stringer SM (2014) Color opponent receptive fields self-organize in a biophysical model of visual cortex via spike-timing dependent plasticity 8:16. doi: Front. Neural Circuits 8:16 [Journal]

   Simulated cortical color opponent receptive fields self-organize via STDP (Eguchi et al., 2014) [Model]

(79 refs)