Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:183014
We developed a 3-layer sensorimotor cortical network of consisting of 704 spiking model-neurons, including excitatory, fast-spiking and low-threshold spiking interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a virtual musculoskeletal human arm, with realistic anatomical and biomechanical properties, to reach a target. Virtual arm position was used to simultaneously control a robot arm via a network interface.
Reference:
1 . Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [PubMed]
2 . Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring Behavior via Inverse Neurocontroller in a Lesioned Cortical Spiking Model Driving a Virtual Arm. Front Neurosci 10:28 [PubMed]
Citations  Citation Browser
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 interneuron basket PV GABA cell; Neocortex fast spiking (FS) interneuron; Neostriatum fast spiking interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON; Python (web link to model);
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing; Motor control; Touch;
Implementer(s): Neymotin, Sam [Samuel.Neymotin at nki.rfmh.org]; Dura, Salvador [ salvadordura at gmail.com];
Search NeuronDB for information about:  Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 interneuron basket PV GABA cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
/
arm2dms_modeldb
mod
msarm
stimdata
README.html
analyse_funcs.py
analysis.py
armGraphs.py
arminterface_pipe.py
basestdp.hoc
bicolormap.py
boxes.hoc *
bpf.h *
col.hoc
colors.hoc *
declist.hoc *
decmat.hoc *
decnqs.hoc *
decvec.hoc *
default.hoc *
drline.hoc *
filtutils.hoc *
grvec.hoc
hinton.hoc *
hocinterface.py
infot.hoc *
init.hoc
intfsw.hoc *
labels.hoc
load.hoc
load.py
local.hoc *
main.hoc
main_demo.hoc
main_neurostim.hoc
misc.h *
misc.py *
msarm.hoc
network.hoc
neuroplot.py *
neurostim.hoc
nload.hoc
nqs.hoc *
nqsnet.hoc *
nrnoc.hoc
params.hoc
perturb.hoc
python.hoc
pywrap.hoc *
run.hoc
runbatch_neurostim.py
runsim_neurostim
samutils.hoc *
saveoutput.hoc
saveoutput2.hoc
setup.hoc *
sim.hoc
sim.py
sim_demo.py
simctrl.hoc *
stats.hoc *
stim.hoc
syncode.hoc *
units.hoc *
vector.py
xgetargs.hoc *
                            
declare("nsmist",new List()) // list of netstims for mist
declare("nvmist",new List()) // list of netstims for mist
declare("ncmist",new List()) // list of NetCons from nsmist to cells
declare("vecTmist",new List()) // list of vectors with mist times for each cell
declare("vecISImist",new List()) // list of vectors with mist ISIs for each cell

objref xo,mistdatafile, mistTimes, mistTimesCell, mistRates, mistIntervals, mistCells, tvecMist, idvecMist, temp, indices   // Initialize things for mist stimuli

tvecMist= new Vector()
idvecMist = new Vector()

//** mkNSs() create and locate the Stimulators -- these are NetStims -- NSLOCs since have loc -- used for MiSt
proc mkMistNSs () { local nmist,ii localobj xo,nc
  nmist=512
  for ii=0,nmist-1 {
      nsmist.append(xo = new NSLOC()) // create the NSs 
  }
  
  for ltr(xo,nsmist,&ii) { // initialize netstims
	  xo.id=ii
	  xo.start = -1
	  xo.interval = 10000
	  xo.number = 1000000
	  xo.noise = 0
	  
	  ncmist.append(nc=new NetCon(xo, col[0].ce.o(ii))) // create netcon between each netstim and cell in the model
	  nc.delay=0
	  nc.weight[mistSynapse]=0.0001
	  
	  nc.record(tvecMist, idvecMist, xo.id) // record spikes at netcon
  }
}

if (userandommist) {
	print "Creating mist stimulus..."
	// Read in stimulus from file
	strdef mistinputfile
	mistdatafile = new File()
	sprint(mistinputfile,"%s-in.txt",filestem)
	mistdatafile.ropen(mistinputfile) // Open file  for reading
	if (mistdatafile.isopen()) {
		mistCells = new Vector()
		mistTimes = new Vector()
		mistRates = new Vector()
		mistIntervals = new Vector()
		mistTimesCell = new Vector()
		temp = new Vector()
		indices = new Vector()
		mkMistNSs() // make microstimulation vec stims
		while (!mistdatafile.eof()) { // Read from the data file
			mistCells.append(mistdatafile.scanvar()) // Read cell
			mistTimes.append(mistdatafile.scanvar()) // Read time point
			mistRates.append(mistdatafile.scanvar()) // Read rate
			}
		mistdatafile.close() // Close the input file
		
		mistRates.add(0.0001) // to avoid division by 0
		mistRates.pow(-1) // invert to obtain intervals
		mistRates.mul(1000) //convert to ms
		
		for i=0, mistCells.size()-1 {
			if (indices.contains(mistCells.x[i])==0) { // only do if new cell index
				indices.append(mistCells.x[i])
				temp.indvwhere(mistCells,"[]",mistCells.x[i],mistCells.x[i]) // find indices of all entries in file for current cell
				vecISImist.append(mistRates.ind(temp)) // append the mist ISIs to list
				vecTmist.append(mistTimes.ind(temp)) // appeend the mist times to list
				ncmist.o(mistCells.x[i]).weight[mistSynapse] = mistWeight // set netcon weight
				xo = nsmist.o(mistCells.x[i]) // select netstim object
				xo.start = mistTimes.ind(temp).x[0] // set start time of netstim
				print xo.start
				vecISImist.o(vecISImist.count()-1).play(&xo.interval, vecTmist.o(vecTmist.count()-1)) // play mist vector on netstim
			}
		}	
	}
}