Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:183014
We developed a 3-layer sensorimotor cortical network of consisting of 704 spiking model-neurons, including excitatory, fast-spiking and low-threshold spiking interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a virtual musculoskeletal human arm, with realistic anatomical and biomechanical properties, to reach a target. Virtual arm position was used to simultaneously control a robot arm via a network interface.
Reference:
1 . Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [PubMed]
2 . Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring Behavior via Inverse Neurocontroller in a Lesioned Cortical Spiking Model Driving a Virtual Arm. Front Neurosci 10:28 [PubMed]
Citations  Citation Browser
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 interneuron basket PV GABA cell; Neocortex fast spiking (FS) interneuron; Neostriatum fast spiking interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON; Python (web link to model);
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing; Motor control; Touch;
Implementer(s): Neymotin, Sam [samn at neurosim.downstate.edu]; Dura, Salvador [ salvadordura at gmail.com];
Search NeuronDB for information about:  Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 interneuron basket PV GABA cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
/
arm2dms_modeldb
mod
msarm
stimdata
README.html
analyse_funcs.py
analysis.py
armGraphs.py
arminterface_pipe.py
basestdp.hoc
bicolormap.py
boxes.hoc *
bpf.h *
col.hoc
colors.hoc *
declist.hoc *
decmat.hoc *
decnqs.hoc *
decvec.hoc *
default.hoc *
drline.hoc *
filtutils.hoc *
grvec.hoc
hinton.hoc *
hocinterface.py
infot.hoc *
init.hoc
intfsw.hoc *
labels.hoc
load.hoc
load.py
local.hoc *
main.hoc
main_demo.hoc
main_neurostim.hoc
misc.h *
misc.py *
msarm.hoc
network.hoc
neuroplot.py *
neurostim.hoc
nload.hoc
nqs.hoc *
nqsnet.hoc *
nrnoc.hoc
params.hoc
perturb.hoc
python.hoc
pywrap.hoc *
run.hoc
runbatch_neurostim.py
runsim_neurostim
samutils.hoc *
saveoutput.hoc
saveoutput2.hoc
setup.hoc *
sim.hoc
sim.py
sim_demo.py
simctrl.hoc *
stats.hoc *
stim.hoc
syncode.hoc *
units.hoc *
vector.py
xgetargs.hoc *
                            
# hocinterface.py -- package of helper functions to allow interface with 
#   various hoc utilities and data structures our models use
#
# Usage:
#   Within the desired place where you want to use the functions do
#     from hocinterface import *
#
# Last update: 12/7/12 (georgec)

# Import pylab here, so it's not required in the functions.
import pylab

# Import neuron here.
from neuron import h

#
# Miscellaneous hoc interface functions
#

# Convert a hoc variable (passed as string) into a numpy array.
def hv2narr (hocvar='vec'):
   exec('%s = h.%s.to_python()' % (hocvar, hocvar))
   exec('%s = pylab.array(%s)' % (hocvar, hocvar))
   exec('x = %s' % hocvar)
   return x

# Convert a numpy array into a hoc variable (passed as string).
def narr2hv (hocvar,narr):
   h('objref %s' % hocvar)
   h('%s = new Vector()' % hocvar)
   h('objref tmp')
   h.tmp = narr
   h('{%s.from_python(tmp)}' % hocvar)

# Get the CTYP number from the string (e.g. 'E2').
def get_ctyp_num (ctyp_str):
   ctyp_ind = -1
   for ii in range(int(h.CTYPi)):
      if (h.CTYP.o(ii).s == ctyp_str):
         ctyp_ind = ii
   return ctyp_ind

# Get the CTYP string from the CTYP number.
def get_ctyp_str (ctyp_num):
   return h.CTYP.o(ctyp_num).s

# Get the STYP number from the string (e.g. 'AM2').
def get_styp_num (styp_str):
   styp_ind = -1
   for ii in range(int(h.STYPi)):
      if (h.STYP.o(ii).s == styp_str):
         styp_ind = ii
   return styp_ind

# Get the STYP string from the STYP number.
def get_styp_str (styp_num):
   return h.STYP.o(styp_num).s

#
# NQS interface functions
#

# Do a gethdrs() for an NQS table.
def shownqshdr (nqsvar='col[0].cellsnq'):
   print nqsvar
   h('%s.gethdrs()' % nqsvar)
 
# Do a pr(numrows) for an NQS table.
def nqspr (nqsvar='col[0].cellsnq', numrows=10):
   print nqsvar    
   h('%s.pr(%d)' % (nqsvar, numrows))
   
# Convert a hoc NQS column into a numpy array.
def nqscol2narr (nqsvar='col[0].cellsnq', colstr='col'):
   h('objref tmpv')
   h('tmpv = new Vector()')
   h('tmpv = %s.getcol("%s")' % (nqsvar, colstr))
   tmpv = h.tmpv.to_python()
   tmpv = pylab.array(tmpv)
   return tmpv
     
#
# grvec functions
#

# Load saved grvec simulation info.
# NOTE: Both the file and its dot-prefixed equivalent need to be present for gvnew() to succeed.
def ldgrvec (grvec_fname):
   h.gvnew(grvec_fname)

# Look at the grvec printlist for the current sim or a saved grvec file.
def lookgrveclist (gvcobjnum=0): 
   if (gvcobjnum == 0):
      gvcstr = 'current simulation'
   else:
      gvcstr = h.panobjl.o(gvcobjnum).filename
   print 'GRVEC List #%d (%s)' % (gvcobjnum, gvcstr)
   print '--------------------------------------------'
   if (gvcobjnum == 0):
      for ii in range(int(h.printlist.count())):
         prstr = '%d %s %d ' % (ii, h.printlist.o(ii).name, \
            h.printlist.o(ii).vec.size())
         print prstr,
         if (h.printlist.o(ii).tvec == None):
            print '(vec only)'
         else:
            print '\n',
   else:
      for ii in range(int(h.panobjl.o(gvcobjnum).printlist.count())):
         print '%d %s %d' % (ii, h.panobjl.o(gvcobjnum).printlist.o(ii).name, \
            h.panobjl.o(gvcobjnum).printlist.o(ii).size)

# Get tvec and vec (in numpy form) from grvec (gvcobjnum=0 means current 
# sim; >0 means saved grvec file)
def getgrvecdat (gvcobjnum=0, vecname='C0_X0_Y0_SPKS'):
   found_ind = -1
   for ii in range(int(h.panobjl.o(gvcobjnum).printlist.count())):
       if (h.panobjl.o(gvcobjnum).printlist.o(ii).name == vecname):
          found_ind = ii
   if (found_ind == -1):
      print 'No such array is on the printlist.'
      return None, None
   elif (gvcobjnum == 0):
      vec = h.printlist.o(found_ind).vec.to_python()    
      tvec = h.printlist.o(found_ind).tvec
      if (tvec == None):
         tvec = pylab.linspace(0,len(vec)-1,len(vec))
      else:
         tvec = pylab.array(tvec.to_python())
   else:
      h('goodread = panobjl.o(%d).rv_readvec(%d,tvec,vec)' % (gvcobjnum, found_ind))
      if (not h.goodread):
         h('panobjl.o(%d).rv_readvec(%d,vec)' % (gvcobjnum, found_ind))
         vec = h.vec.to_python()
         tvec = pylab.linspace(0,len(vec)-1,len(vec))
      else:         
         tvec = pylab.array(h.tvec.to_python())
         vec = h.vec.to_python()
   vec = pylab.array(vec)
   return tvec, vec

# Plot tvec and vec (in numpy form) from grvec (gvcobjnum=0 means current 
# sim; >0 means saved grvec file)
def plotgrvecdat (gvcobjnum=0, vecname='C0_X0_Y0_SPKS'):
   tvec,vec = getgrvecdat(gvcobjnum, vecname)
   if (tvec != None):
      pylab.plot(tvec,vec)