Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:183014
We developed a 3-layer sensorimotor cortical network of consisting of 704 spiking model-neurons, including excitatory, fast-spiking and low-threshold spiking interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a virtual musculoskeletal human arm, with realistic anatomical and biomechanical properties, to reach a target. Virtual arm position was used to simultaneously control a robot arm via a network interface.
Reference:
1 . Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [PubMed]
2 . Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring Behavior via Inverse Neurocontroller in a Lesioned Cortical Spiking Model Driving a Virtual Arm. Front Neurosci 10:28 [PubMed]
Citations  Citation Browser
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 interneuron basket PV GABA cell; Neocortex fast spiking (FS) interneuron; Neostriatum fast spiking interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON; Python (web link to model);
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing; Motor control; Touch;
Implementer(s): Neymotin, Sam [Samuel.Neymotin at nki.rfmh.org]; Dura, Salvador [ salvadordura at gmail.com];
Search NeuronDB for information about:  Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 interneuron basket PV GABA cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
/
arm2dms_modeldb
mod
msarm
stimdata
README.html
analyse_funcs.py
analysis.py
armGraphs.py
arminterface_pipe.py
basestdp.hoc
bicolormap.py
boxes.hoc *
bpf.h *
col.hoc
colors.hoc *
declist.hoc *
decmat.hoc *
decnqs.hoc *
decvec.hoc *
default.hoc *
drline.hoc *
filtutils.hoc *
grvec.hoc
hinton.hoc *
hocinterface.py
infot.hoc *
init.hoc
intfsw.hoc *
labels.hoc
load.hoc
load.py
local.hoc *
main.hoc
main_demo.hoc
main_neurostim.hoc
misc.h *
misc.py *
msarm.hoc
network.hoc
neuroplot.py *
neurostim.hoc
nload.hoc
nqs.hoc *
nqsnet.hoc *
nrnoc.hoc
params.hoc
perturb.hoc
python.hoc
pywrap.hoc *
run.hoc
runbatch_neurostim.py
runsim_neurostim
samutils.hoc *
saveoutput.hoc
saveoutput2.hoc
setup.hoc *
sim.hoc
sim.py
sim_demo.py
simctrl.hoc *
stats.hoc *
stim.hoc
syncode.hoc *
units.hoc *
vector.py
xgetargs.hoc *
                            
// $Id: nqsnet.hoc,v 1.73 2011/11/01 01:49:40 samn Exp $
// xopen("nqsnet.hoc")

//* params controlling sim from testrf.hoc

//* mkcp0() pre-id  post-id  pre#  post#   distance weight  syn-id   nc ptr  wt1 (eg AMPA+NMDA)
objref nq[2],sq[CTYPi][CTYPi],cp
obfunc mkcp0 () { localobj lo
  lo = new NQS("PRID","POID","STYP","PIJ","DIV","CONV","NSYN","NPRE")
  lo.useslist("PRID",CTYP) lo.useslist("POID",CTYP) lo.useslist("STYP",STYP)
  return lo
}

// CODE: PRID,POID,INCOL,COL1,COL2
obfunc mksp () { localobj lo
  lo=new NQS("CODE","PR","PO","DEL","WT0","WT1") // CODE==PRID(1),POID(2),COLA(3),COLB(4)
  lo.coddec("CODE")
  // lo.useslist("PRID",CTYP) lo.useslist("POID",CTYP) 
  return lo
}
sp=mksp()

//* Numbers and connectivity params

// layer return layer location with 'sublayer' defined by Inhib (+0.5) or other suffix
// E or I should be 1st letter of name, suffix letter will ideally dichotomize into late
// alphabet or early alphabet
func layer () { local x,in,la
  la=0
  if ($1 >= CTYP.count) return -1
  if (sscanf(CTYP.o($1).s,"%c%d%c",&in,&x,&la)<2) return -1
  if (x==23) x=3 // layer 2/3
  if (in==73) x+=0.5 // ascii 73 is 'I'
  if (la>77) x+=0.2 // <='M'
  return x
}

//** getlayerz -- get average z location of a layer, based on frana and tononi
// returns value in micrometers. 0 is at top (layer 1), max val
// is at layer 6 ... actual #s don't matter, only distances between
// the layers matters...
func getlayerz () {
  if($1 == 2 || $1 == 3){
    return 1740 // average of 1540+1940 from frana
  } else if($1 == 4 ) { // this is just midpoint btwn L2/3 and L5
    return 1435
  } else if($1 == 5 ){
    return 1130
  } else if($1 == 6 ){
    return 488
  } else {
    printf("getz ERRA: invalid layer %d!\n",$1)
    return -1
  }
}

//* routines
//** styp() sets synapse type based on presynaptic cell
func styp () { local pr,po
  pr=$1 po=$2
  if (pr==IN && po==IN) { return GA 
  } else if (pr==IN) { return IX
  } else if (pr==SU || pr==DP) { return EX
  } else if (pr==SM) { return AM
  } else if (strm(CTYP.o[pr].s,"^E")) { return EX
  } else if (strm(CTYP.o[pr].s,"^I")) { return IX
  } else printf("styp ERR %s->%s not classified",CTYP.object(pr).s,CTYP.object(po).s)
}

//** ellfld() place the cells inside an ellipse
// r for an ellipse = a*b/sqrt((a*sin(theta))^2 + (b*cos(theta))^2)
proc ellfld () { local a,b,ii,jj,p,seed localobj xv,yv,xo
  seed=239023229
  a=1 b=2
  p=allocvecs(xv,yv) vrsz(allcells*10,xv,yv)
  xv.setrnd(4,2*a,seed) yv.setrnd(4,2*b) xv.sub(a) yv.sub(b)
  jj=0
  for vtr2(&x,&y,xv,yv,&ii) {
    if (a*x^2+b*y^2<1) { ce.o(jj).xloc=x ce.o(jj).yloc=y jj+=1 }
    if (jj==ce.count) break
  }
  print ii,jj
  if (jj!=ce.count) print "Not filled"
  dealloc(p)
}