Sensorimotor cortex reinforcement learning of 2-joint virtual arm reaching (Neymotin et al. 2013)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:150245
"... We developed a model of sensory and motor neocortex consisting of 704 spiking model-neurons. Sensory and motor populations included excitatory cells and two types of interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a 2-joint virtual arm to reach to a fixed target. ... "
Reference:
1 . Neymotin SA, Chadderdon GL, Kerr CC, Francis JT, Lytton WW (2013) Reinforcement learning of 2-joint virtual arm reaching in a computer model of sensorimotor cortex Neural Computation 25(12):3263-93 [PubMed]
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex V1 pyramidal corticothalamic L6 cell; Neocortex U1 pyramidal intratelencephalic L2-5 cell; Neocortex V1 interneuron basket PV cell; Neocortex fast spiking (FS) interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON;
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing;
Implementer(s): Neymotin, Sam [samn at neurosim.downstate.edu]; Chadderdon, George [gchadder3 at gmail.com];
Search NeuronDB for information about:  Neocortex V1 pyramidal corticothalamic L6 cell; Neocortex V1 interneuron basket PV cell; Neocortex U1 pyramidal intratelencephalic L2-5 cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
/
a2dmodeldb
readme.html
drspk.mod *
infot.mod *
intf6_.mod *
misc.mod *
nstim.mod *
stats.mod *
vecst.mod *
arm.hoc
basestdp.hoc
col.hoc
colors.hoc *
declist.hoc *
decmat.hoc *
decnqs.hoc *
decvec.hoc *
default.hoc *
drline.hoc *
filtutils.hoc *
geom.hoc
grvec.hoc *
hinton.hoc *
infot.hoc *
init.hoc
labels.hoc *
misc.h *
mosinit.hoc
network.hoc
nload.hoc
nqs.hoc *
nqsnet.hoc *
nrnoc.hoc *
params.hoc
python.hoc
pywrap.hoc *
run.hoc
samutils.hoc *
screenshot.png
sense.hoc *
setup.hoc *
simctrl.hoc *
stats.hoc *
stim.hoc
syncode.hoc *
trainedplast.nqs
units.hoc *
xgetargs.hoc *
                            
// $Id: nrnoc.hoc,v 1.74 2007/11/20 07:51:52 billl Exp $

proc nrnoc () {}

// Users should not edit nrnoc.hoc or default.hoc.  Any local 
// changes to these files should be made in local.hoc.

// key '*&*' is picked up by to indicate command for emacs
proc elisp () { printf("*&* %s\n",$s1) }
// if (not exists(simname)) { strdef simname, output_file, datestr, comment }

// Simctrl.hoc will automatically load stdgraph.hoc which automatically
// loads stdrun.hoc
strdef temp_string_, user_string_  // needed for simctrl
/* Global variable default values.  NOTE that stdrun.hoc, stdgraph.hoc
and simctrl.hoc all contain variable definitions and thus default.hoc
should be loaded after these files */
load_file("default.hoc")      /* Load default.hoc */

/* Allows arrays of strings */
objref hoc_obj_[2]
load_file("stdgui.hoc") // don't want to encounter other String tempate defs
load_file("simctrl.hoc")

proc run () {
  running_ = 1
  stdinit()
  continueRun(tstop)
  finish()
}

proc continueRun () { local rt, rtstart, ts
  if (numarg()==1) ts=$1 else ts=t+1e3
  realtime = 0  rt = screen_update_invl  rtstart = startsw()
  eventcount=0
  eventslow=1
  stoprun = 0
  if (using_cvode_) {
    if (cvode.use_local_dt || (cvode.current_method()%10) == 0) {
      cvode.solve(ts)
      flushPlot()
      realtime = startsw() - rtstart
      return
    }
  } else {
    ts -= dt/2
  }
  while (t<ts && stoprun==0) {
    step()
    realtime = startsw() - rtstart
    if (realtime >= rt) {
      //                        if (!stdrun_quiet) fastflushPlot()
      screen_update()
      //really compute for at least screen_update_invl
      realtime = startsw() - rtstart
      rt = realtime + screen_update_invl
    }
  }
  if (using_cvode_ && stoprun == 0) { // handle the "tstop" event
    step() // so all recordings take place at tstop
  }
  flushPlot()
  realtime = startsw() - rtstart
}

proc stdinit() {
        cvode_simgraph()
        realtime = 0
        setdt()
        init()
        initPlot()
}

proc init () {
  cvode_simgraph()
  initMech()
  initMisc1()

  // Initialize state vars then calculate currents
  // If user hand-set v in initMisc1() then v_init should be > 1000,
  // else all compartments will be set to v_init
  if (v_init < 1000) {
    finitialize(v_init)
  } else {
    finitialize()
  }

  // Set ca pump and leak channel for steady state
  setMemb()

  initMisc2()
  if (cvode_active()) cvode.re_init() else fcurrent()
  frecord_init()
}

// Initialization of mechanism variables
// NOTE: if any changes are made to the NEURON block of any local mod
// file, the user must add the necessary inits to initMisc1()
proc initMech () { 
  forall {
    if ((!ismembrane("pas")) && (!ismembrane("Passive"))) { 
      // Allow for either pas or Passive mod file usage
      // errorMsg("passive not inserted") 
    }

    if (ismembrane("na_ion")) { 
      nai = na_init
      nai0_na_ion = na_init
    }
    
    if (ismembrane("k_ion")) {
      ki = k_init
      ki0_k_ion = k_init
    }
    
    if (ismembrane("ca_ion")) { 
      cai = ca_init
      cai0_ca_ion = ca_init
    }
  }
}

//* setMemb complex -- multiple names for passive mech
//** declarations
iterator scase() { local i
  for i = 1, numarg() { temp_string_ = $si iterator_statement }}
objref paslist,pasvars[3],XO
double pasvals[2],x[1]
paslist = new List()
for ii=0,2 pasvars[ii]= new String()
for scase("fastpas","pas","Pass","Passive") paslist.append(new String(temp_string_))

//** getval(),setval() -- return/set the hoc value of a string
func retval () { return getval($s1) }
func getval () { 
  sprint(temp_string2_,"x=%s",$s1)
  execute(temp_string2_)
  return x
}
proc setval () { 
  sprint(temp_string2_,"%s=%g",$s1,$2)
  execute(temp_string2_)
}

//** findpas()
// assumes that we are starting in a live section since looks for pass mech there
qx_=0
proc findpas () {
  for ii=0,paslist.count-1 {
    XO=paslist.object(ii)
    if (ismembrane(XO.s)) {
      // print XO.s,"found"
      pasvars[2].s=XO.s
      sprint(pasvars[0].s,"g_%s(qx_)",XO.s)
      for scase("e","erev","XXXX") {  // look for the proper prefix
        sprint(temp_string_,"%s_%s",temp_string_,XO.s)
        if (name_declared(temp_string_)==1) break
      }
      if (name_declared(temp_string_)==0) { // not found
        printf("SetMemb() in nrnoc.hoc: Can't find proper 'erev' prefix for %s\n",XO.s)
      } else {
        sprint(pasvars[1].s,"%s(qx_)",temp_string_)
      }
    }
  }
}

proc setMemb () {
  if (!secp()) return
  findpas() // assume that passive name is the same in all sections
  forall for (qx_,0) {  // will eventually want 'for (x)' to handle all the segments
    if (ismembrane(pasvars[2].s)) {
        for ii=0,1 pasvals[ii]=getval(pasvars[ii].s)
        setmemb2()
        for ii=0,1 setval(pasvars[ii].s,pasvals[ii])
    }
  }
}

// secp() determine whether any sections exist
func secp () { local n
  n=0
  forall n+=1
  if (n>0) return 1 else return 0
}

func setother () {return 0} // callback stub
proc setmemb2 () { local iSum, ii, epas, gpas
  if (!secp()) return
  gpas=pasvals[0] epas=pasvals[1]
  // Setup steady state voltage using leak channel
  iSum = 0.0
  if (ismembrane("na_ion")) { iSum += ina(qx_) }
  if (ismembrane("k_ion"))  { iSum += ik(qx_)  }
  if (ismembrane("ca_ion")) { iSum += ica(qx_) }
  iSum += setother()

  if (iSum == 0) {        // Passive cmp so set e_pas = v
    epas = v
  } else {
    if (gpas > 0) {    // Assume g set by user, calc e
      epas = v + iSum/gpas

    } else {            // Assume e set by user, calc g
      if (epas != v) {
        gpas = iSum/(epas - v)
      } else { gpas=0 }
    }
    if (gpas < 0) errorMsg("bad g", gpas)
    if (epas < -100 || epas > 0) {
      printf(".")
      // printf("%s erev: %g %g %g\n",secname(),e_pas,ina,ik)
    }
  }
  pasvals[0]=gpas pasvals[1]=epas
}

proc finish () {
  /* Called following completion of continueRun() */

finishMisc()

if (graph_flag == 1) {
  if (iv_flag == 1) {
    flushPlot()
    doEvents()
  } else {
    graphmode(-1)
    plt(-1)
  }
}

if (print_flag == 1) {
  wopen("")
}
}

/*------------------------------------------------------------
User definable GRAPHICS and PRINTING routines
------------------------------------------------------------*/

proc outputData() {
  // Default procedure - if outputData() doesn't exist in the run file

  if (graph_flag == 1) {
    if (iv_flag == 1) {
      Plot()
      rt = stopsw()
      if (rt > realtime) {
        realtime = rt
        fastflushPlot()
        doNotify()
        if (realtime == 2 && eventcount > 50) {
          eventslow = int(eventcount/50) + 1
        }
        eventcount = 0
      }else{
        eventcount = eventcount + 1
        if ((eventcount%eventslow) == 0) {
          doEvents()
        }
      }

    } else {
      graph(t)
    }
  }

  if (print_flag == 1) { 
    if (t%printStep <= printStep) { printOut() }
  }
}

proc printOut() {
  /* Default procedure - if printOut() doesn't exist in the run file */
}

proc initGraph() {
  /* Default procedure - if initGraph() doesn't exist in the run file */

graph()
}

proc initPrint() {
  /* Default procedure - if initPrint() doesn't exist in the run file */

wopen(output_file)
}

/*------------------------------------------------------------
User definable BATCH RUN routines
------------------------------------------------------------*/

proc nextrun() {
  // Called from finishmisc() following completion of batch in an autorun
  wopen("")   
  runnum = runnum + 1
  sprint(output_file,"data/b%s.%02d", datestr, runnum)
}                       

// commands for emacs
proc update_runnum() { 
  runnum = $1
  sprint(output_file,"data/%s.%02d", datestr, runnum)
  print "^&^ (progn (sim-index-revert)(setq sim-runnum ",runnum,"))" }
proc nrn_write_index() { printf("&INDEX& %s\n",$s1) }
proc nrn_update () { elisp("nrn-update") }
proc nrn_message () { printf("!&! %s\n",$s1) } 

/*------------------------------------------------------------
User definable INITIALIZATION and FINISH routines
------------------------------------------------------------*/

// Default procedure - if initMisc1() doesn't exist in the run file 
// Initializations performed prior to finitialize() 
// This should contain point process inits and inits for any changes 
//        made to the NEURON block of any local mod file 
proc initMisc1() { }

// Default procedure - if initMisc2() doesn't exist in the run file 
// Initializations performed after finitialize() 
proc initMisc2() { }

// Default procedure - if finishMisc() doesn't exist in the run file 
proc finishMisc() { }

/*------------------------------------------------------------
Miscellaneous routines
------------------------------------------------------------*/

proc errorMsg() {
  /* Print warning, assumes arg1 is string and arg2 if present is a
  variable value */

sectionname(section)

if (numarg() == 0) {
  printf("ERROR in errorMsg(): Needs at least 1 argument.\n")
} else if (numarg() == 1) {
  printf("ERROR: %s in section %s.\n", $s1, section)
} else {
  printf("ERROR: %s in section %s (var=%g).\n", $s1, section, $2)
}
}

proc clear() {
  /* Clear non-interviews plot window */
plt(-3)
}

func mod() { local x, y
  /* Mod function for non-integers */

x=$1
y=$2

return (x/y - int(x/y))
}

proc whatSection() { print secname() }

proc print_pp_location() { local x //arg1 must be a point process
   x = $o1.get_loc()
   sectionname(temp_string_)
   printf("%s located at %s(%g)\n", $o1, temp_string_, x)
   pop_section()
}

//* set method with method()
proc method () { local prc
  if (numarg()==0) {
    if (cvode_active() && cvode_local()) { printf("\tlocal atol=%g\n",cvode.atol)
    } else if (cvode_active()) { printf("\tglobal atol=%g\n",cvode.atol)
    } else if (secondorder==2) { printf("\tCrank-Nicholson dt=%g\n",dt)
    } else if (secondorder==0) { printf("\timplicit dt=%g\n",dt)
    } else { printf("\tMethod unrecognized\n") }
    return
  }
  if (numarg()==2) prc = $2 else prc=0
  finitialize()
  if (strcmp($s1,"global")==0) {
    cvode_active(1)
    cvode.condition_order(2)
    if (prc) cvode.atol(prc)
  } else if (strcmp($s1,"local")==0) {
    cvode_local(1)
    cvode.condition_order(2)
    if (prc) cvode.atol(prc)
  } else if (strcmp($s1,"implicit")==0) {
    secondorder=0
    cvode_active(1)
    cvode_active(0)
    if (prc) dt=prc
  } else if (strcmp($s1,"CN")==0) {
    secondorder=2
    cvode_active(1) // this turns off local
    cvode_active(0)
    if (prc) dt=prc
  } else {
    printf("Integration method %s not recognized\n",$s1)
  }
}

//* Load local modifications to nrnoc.hoc and default.hoc
load_file("local.hoc")

if (xwindows && graph_flag) { nrnmainmenu() } // pwman_place(50,50)

print "Init complete.\n"

Neymotin SA, Chadderdon GL, Kerr CC, Francis JT, Lytton WW (2013) Reinforcement learning of 2-joint virtual arm reaching in a computer model of sensorimotor cortex Neural Computation 25(12):3263-93[PubMed]

References and models cited by this paper

References and models that cite this paper

Afshar A, Santhanam G, Yu BM, Ryu SI, Sahani M, Shenoy KV (2011) Single-trial neural correlates of arm movement preparation. Neuron 71:555-64

Almassy N, Edelman GM, Sporns O (1998) Behavioral constraints in the development of neuronal properties: a cortical model embedded in a real-world device. Cereb Cortex 8:346-61 [PubMed]

Bannister AP (2005) Inter- and intra-laminar connections of pyramidal cells in the neocortex. Neurosci Res 53:95-103 [PubMed]

Bedau MA (2005) Artificial life: more than just building and studying computational systems. Artif Life 11:1-3 [PubMed]

Berthier N (2011) The syntax of human infant reaching 8th International Conference on Complex Systems :1477-1487

Berthier NE, Clifton RK, McCall DD, Robin DJ (1999) Proximodistal structure of early reaching in human infants. Exp Brain Res 127:259-69

Carnevale NT, Hines ML (2006) The NEURON Book

Chadderdon GL, Neymotin SA, Kerr CC, Lytton WW (2012) Reinforcement learning of targeted movement in a spiking neuronal model of motor cortex PLoS ONE 2012 7(10):e47251 [Journal]

   Reinforcement learning of targeted movement (Chadderdon et al. 2012) [Model]

Cools R (2006) Dopaminergic modulation of cognitive function-implications for L-DOPA treatment in Parkinson's disease. Neurosci Biobehav Rev 30:1-23 [PubMed]

Corbetta D, Snapp-Childs W (2009) Seeing and touching: the role of sensory-motor experience on the development of infant reaching. Infant Behav Dev 32:44-58

De Schutter E (2008) Why are computational neuroscience and systems biology so separate? PLoS Comput Biol 4:e1000078 [Journal] [PubMed]

Dyhrfjeld-Johnsen J, Santhakumar V, Morgan RJ, Huerta R, Tsimring L, Soltesz I (2007) Topological determinants of epileptogenesis in large-scale structural and functional models of the dentate gyrus derived from experimental data. J Neurophysiol 97:1566-87 [Journal] [PubMed]

   Dentate gyrus (Morgan et al. 2007, 2008, Santhakumar et al. 2005, Dyhrfjeld-Johnsen et al. 2007) [Model]

Edelman GM (1987) Neural Darwinism: The Theory of Neural Group Selection

Edelman GM (2004) Wider than the sky: The phenomenal gift of consciousness

Engel A, Konig P, Kreiter A, Gray C, Singer W (1991) Temporal coding by coherent oscillations as a potential solution to the binding problem: physiological evidence Nonlinear dynamics and neural networks, Schuster HG, ed.

Evans RC, Morera-Herreras T, Cui Y, Du K, Sheehan T, Kotaleski JH, Venance L, Blackwell KT (2012) The effects of NMDA subunit composition on calcium influx and spike timing-dependent plasticity in striatal medium spiny neurons. PLoS Comput Biol 8:e1002493 [Journal] [PubMed]

   NMDA subunit effects on Calcium and STDP (Evans et al. 2012) [Model]

Farries MA, Fairhall AL (2007) Reinforcement learning with modulated spike timing dependent synaptic plasticity. J Neurophysiol 98:3648-65 [PubMed]

Fenton AA, Lytton WW, Barry JM, Lenck-Santini PP, Zinyuk LE, Kubik S, Bures J, Poucet B, Mull (2010) Attention-like modulation of hippocampus place cell discharge. J Neurosci 30:4613-25

Florian RV (2007) Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Comput 19:1468-502 [PubMed]

Frank MJ, O'reilly RC (2006) A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci 120:497-517 [PubMed]

Frank MJ, Seeberger LC, O`Reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940-3 [Journal] [PubMed]

   Dynamic dopamine modulation in the basal ganglia: Learning in Parkinson (Frank et al 2004,2005) [Model]

Gourevitch B, Eggermont JJ (2007) Evaluating information transfer between auditory cortical neurons. J Neurophysiol 97:2533-43 [PubMed]

Graybiel AM, Aosaki T, Flaherty AW, Kimura M (1994) The basal ganglia and adaptive motor control. Science 265:1826-31 [PubMed]

Hikosaka O, Nakamura K, Sakai K, Nakahara H (2002) Central mechanisms of motor skill learning. Curr Opin Neurobiol 12:217-22

Hosp JA, Pekanovic A, Rioult-Pedotti MS, Luft AR (2011) Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J Neurosci 31:2481-7 [PubMed]

Houk JC, Wise SP (2004) Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb Cortex 5:95-110 [PubMed]

Izhikevich EM (2007) Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling. Cereb Cortex 17(10):2443-2452 [Journal] [PubMed]

   Linking STDP and Dopamine action to solve the distal reward problem (Izhikevich 2007) [Model]

Jones SR, Kerr CE, Wan Q, Pritchett DL, Hamalainen M, Moore CI (2010) Cued spatial attention drives functionally relevant modulation of the mu rhythm in primary somatosensory cortex. J Neurosci 30:13760-5 [PubMed]

Kelemen E, Fenton AA (2010) Dynamic grouping of hippocampal neural activity during cognitive control of two spatial frames. PLoS Biol 8:e1000403 [PubMed]

Kerr CC, Neymotin SA, Chadderdon GL, Fietkiewicz CT, Francis JT, Lytton WW (2012) Electrostimulation as a prosthesis for repair of information flow in a computer model of neocortex IEEE Transactions on Neural Systems & Rehabilitation Engineering 20(2):153-60 [Journal] [PubMed]

   Prosthetic electrostimulation for information flow repair in a neocortical simulation (Kerr 2012) [Model]

Kerr CC, Van Albada SJ, Neymotin SA, Chadderdon GL, Robinson PA, Lytton WW (2013) Cortical information flow in Parkinson's disease: a composite network-field model. Front Comput Neurosci 7:39:1-14 [Journal] [PubMed]

   Composite spiking network/neural field model of Parkinsons (Kerr et al 2013) [Model]

Kubikova L, Kostal L (2010) Dopaminergic system in birdsong learning and maintenance. J Chem Neuroanat 39:112-23

Le Novere N (2007) The long journey to a Systems Biology of neuronal function. BMC Syst Biol 1:28-23

Luft AR, Schwarz S (2009) Dopaminergic signals in primary motor cortex. Int J Dev Neurosci 27:415-21 [PubMed]

Lungarella M, Sporns O (2006) Mapping information flow in sensorimotor networks. PLoS Comput Biol 2:e144 [PubMed]

Lytton WW (2008) Computer modelling of epilepsy. Nat Rev Neurosci 9:626-37 [Journal] [PubMed]

Lytton WW, Neymotin SA, Hines ML (2008) The virtual slice setup. J Neurosci Methods 171:309-15 [Journal] [PubMed]

   The virtual slice setup (Lytton et al. 2008) [Model]

Lytton WW, Omurtag A (2007) Tonic-clonic transitions in computer simulation. J Clin Neurophysiol 24:175-81 [PubMed]

   Tonic-clonic transitions in a seizure simulation (Lytton and Omurtag 2007) [Model]

Lytton WW, Omurtag A, Neymotin SA, Hines ML (2008) Just in time connectivity for large spiking networks Neural Comput 20(11):2745-56 [Journal] [PubMed]

   JitCon: Just in time connectivity for large spiking networks (Lytton et al. 2008) [Model]

Lytton WW, Stewart M (2005) A rule-based firing model for neural networks Int J Bioelectromagn 7:47-50

Lytton WW, Stewart M (2006) Rule-based firing for network simulations. Neurocomputing 69:1160-1164

Mahmoudi B, Sanchez JC (2011) A symbiotic brain-machine interface through value-based decision making. PLoS One 6:e14760-23

Marsh B, Tarigoppula A, Francis J (2011) Correlates of reward expectation in the primary motor cortex: Developing an actor-critic model in macaques for a brain computer interface Society for Neuroscience Abstracts, 41

Mo J, Schroeder CE, Ding M (2011) Attentional modulation of alpha oscillations in macaque inferotemporal cortex. J Neurosci 31:878-82

Molina-Luna K, Pekanovic A, Rohrich S, Hertler B, Schubring-Giese M, Rioult-Pedotti MS, Luft (2009) Dopamine in motor cortex is necessary for skill learning and synaptic plasticity. PLoS One 4:e7082-21 [PubMed]

Neymotin S, Kerr C, Francis J, Lytton W (2011) Training oscillatory dynamics with spike-timing-dependent plasticity in a computer model of neocortex Signal Processing in Medicine and Biology Symposium (SPMB), IEEE :1-6

Neymotin SA, Jacobs KM, Fenton AA, Lytton WW (2011) Synaptic information transfer in computer models of neocortical columns. J Comput Neurosci. 30(1):69-84 [Journal] [PubMed]

   Synaptic information transfer in computer models of neocortical columns (Neymotin et al. 2010) [Model]

Neymotin SA, Lazarewicz MT, Sherif M, Contreras D, Finkel LH, Lytton WW (2011) Ketamine disrupts theta modulation of gamma in a computer model of hippocampus Journal of Neuroscience 31(32):11733-11743 [Journal] [PubMed]

   Ketamine disrupts theta modulation of gamma in a computer model of hippocampus (Neymotin et al 2011) [Model]

Neymotin SA, Lee H, Park E, Fenton AA, Lytton WW (2011) Emergence of physiological oscillation frequencies in a computer model of neocortex. Front Comput Neurosci 5:19-75 [Journal] [PubMed]

   Emergence of physiological oscillation frequencies in neocortex simulations (Neymotin et al. 2011) [Model]

Pastalkova E, Serrano P, Pinkhasova D, Wallace E, Fenton AA, Sacktor TC (2006) Storage of spatial information by the maintenance mechanism of LTP. Science 313:1141-4 [PubMed]

Peterson BE, Healy MD, Nadkarni PM, Miller PL, Shepherd GM (1996) ModelDB: an environment for running and storing computational models and their results applied to neuroscience. J Am Med Inform Assoc 3:389-98 [Journal] [PubMed]

Potjans W, Morrison A, Diesmann M (2009) A spiking neural network model of an actor-critic learning agent. Neural Comput 21:301-39 [PubMed]

Qiu S, Anderson CT, Levitt P, Shepherd GM (2011) Circuit-specific intracortical hyperconnectivity in mice with deletion of the autism-associated Met receptor tyrosine kinase. J Neurosci 31:5855-64

Reid RC (2012) From functional architecture to functional connectomics. Neuron 75:209-17

Reynolds JN, Wickens JR (2005) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 15:507-21 [PubMed]

Roberts PD, Bell CC (2002) Spike timing dependent synaptic plasticity in biological systems. Biol Cybern 87:392-403 [PubMed]

Rowan MS,Neymotin SA (2013) Synaptic Scaling Balances Learning in a Spiking Model of Neocortex Adaptive and Natural Computing Algorithms, Tomassini M, Antonioni A, Daolio F, Buesser P, ed. pp.20 [Journal]

   Synaptic scaling balances learning in a spiking model of neocortex (Rowan & Neymotin 2013) [Model]

Sanes JN (2003) Neocortical mechanisms in motor learning. Curr Opin Neurobiol 13:225-31

Seung HS (2003) Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40:1063-73 [PubMed]

Shadmehr R, Krakauer JW (2008) A computational neuroanatomy for motor control. Exp Brain Res 185:359-81 [PubMed]

Shadmehr R, Wise S (2005) The computational neurobiology of reaching and pointing: a foundation for motor learning

Shen W, Flajolet M, Greengard P, Surmeier DJ (2008) Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321:848-51 [PubMed]

Shepherd G (2004) The synaptic organization of the brain, Shepherd GM, ed.

Sober SJ, Brainard MS (2009) Adult birdsong is actively maintained by error correction. Nat Neurosci 12:927-31 [PubMed]

Song S, Miller KD, Abbott LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 3:919-26 [PubMed]

Sporns O, Tononi G, Kotter R (2005) The human connectome: A structural description of the human brain. PLoS Comput Biol 1:e42-308

Thomson AM, Lamy C (2007) Functional maps of neocortical local circuitry. Front Neurosci 1:19-42 [PubMed]

Thomson AM, West DC, Wang Y, Bannister AP (2002) Synaptic connections and small circuits involving excitatory and inhibitory neurons in layers 2-5 of adult rat and cat neocortex: triple intracellular recordings and biocytin labelling in vitro. Cereb Cortex 12:936-53 [PubMed]

Thorndike E (1911) Animal intelligence

Tiesinga PH, Sejnowski TJ (2004) Rapid temporal modulation of synchrony by competition in cortical interneuron networks. Neural Comput 16:251-75 [PubMed]

Uhlhaas PJ, Singer W (2006) Neural synchrony in brain disorders: relevance for cognitive dysfunctions and pathophysiology. Neuron 52:155-68 [PubMed]

von der Malsburg C, Schneider W (1986) A neural cocktail-party processor. Biol Cybern 54:29-40 [PubMed]

von Kraus LM, Sacktor TC, Francis JT (2010) Erasing sensorimotor memories via PKMzeta inhibition. PLoS One 5:e11125-81

Von_hofsten C (1979) Development of visually directed reaching: The approach phase Department Of Psychology, University Of Uppsala [psykologiska Inst , Uppsala Univ ]

Webb B (2000) What does robotics offer animal behaviour? Anim Behav 60:545-558

Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring behavior via inverse neurocontroller in a lesioned cortical spiking model driving a virtual arm. Front. Neurosci. Neuroprosthetics 10:28 [Journal]

   Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015) [Model]

Dura-Bernal S, Neymotin SA, Kerr CC, Sivagnanam S, Majumdar A, Francis JT, Lytton WW (2017) Evolutionary algorithm optimization of biological learning parameters in a biomimetic neuroprosthesis. IBM Journal of Research and Development (Computational Neuroscience special issue) 61(2/3):6:1-6:14 [Journal]

   Motor system model with reinforcement learning drives virtual arm (Dura-Bernal et al 2017) [Model]

Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [Journal] [PubMed]

   Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015) [Model]

Eguchi A, Neymotin SA and Stringer SM (2014) Color opponent receptive fields self-organize in a biophysical model of visual cortex via spike-timing dependent plasticity 8:16. doi: Front. Neural Circuits 8:16 [Journal]

   Simulated cortical color opponent receptive fields self-organize via STDP (Eguchi et al., 2014) [Model]

(79 refs)