Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:183014
We developed a 3-layer sensorimotor cortical network of consisting of 704 spiking model-neurons, including excitatory, fast-spiking and low-threshold spiking interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a virtual musculoskeletal human arm, with realistic anatomical and biomechanical properties, to reach a target. Virtual arm position was used to simultaneously control a robot arm via a network interface.
References:
1 . Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [PubMed]
2 . Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring behavior via inverse neurocontroller in a lesioned cortical spiking model driving a virtual arm. Front. Neurosci. Neuroprosthetics 10:28
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex M1 pyramidal pyramidal tract L5B cell; Neocortex M1 pyramidal intratelencephalic L2-5 cell; Neocortex M1 interneuron basket PV cell; Neocortex fast spiking (FS) interneuron; Neostriatum fast spiking interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON; Python (web link to model);
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing; Motor control;
Implementer(s): Neymotin, Sam [samn at neurosim.downstate.edu]; Dura, Salvador [ salvadordura at gmail.com];
Search NeuronDB for information about:  Neocortex M1 pyramidal intratelencephalic L2-5 cell; Neocortex M1 pyramidal pyramidal tract L5B cell; Neocortex M1 interneuron basket PV cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
/
arm2dms_modeldb
mod
msarm
stimdata
README.html
analyse_funcs.py
analysis.py
armGraphs.py
arminterface_pipe.py
basestdp.hoc
bicolormap.py
boxes.hoc *
bpf.h *
col.hoc
colors.hoc *
declist.hoc *
decmat.hoc *
decnqs.hoc *
decvec.hoc *
default.hoc *
drline.hoc *
filtutils.hoc *
grvec.hoc
hinton.hoc *
hocinterface.py
infot.hoc *
init.hoc
intfsw.hoc *
labels.hoc
load.hoc
load.py
local.hoc *
main.hoc
main_demo.hoc
main_neurostim.hoc
misc.h *
misc.py *
msarm.hoc
network.hoc
neuroplot.py *
neurostim.hoc
nload.hoc
nqs.hoc *
nqsnet.hoc *
nrnoc.hoc
params.hoc
perturb.hoc
python.hoc
pywrap.hoc *
run.hoc
runbatch_neurostim.py
runsim_neurostim
samutils.hoc *
saveoutput.hoc
saveoutput2.hoc
setup.hoc *
sim.hoc
sim.py
sim_demo.py
simctrl.hoc *
stats.hoc *
stim.hoc
syncode.hoc *
units.hoc *
vector.py
xgetargs.hoc *
                            
#!/bin/bash
# This program runs the Ubermodel in batch mode.
#
# Instructions:
#   runsim [filename] [duration] [simsize] etc. 
# 
# Example: runsim foo 120 8
#   which runs a 4000-cell sim for 120 s and saves to foo-spk.txt etc.
#
# Version: 2012jun19

tic=`date +%s` # Get current time
cd `dirname $0` # Make sure we're n the right directory

# Handle input arguments
   filestem=${1}; if [ -z $filestem  ]; then filestem="stimdata/testsim"; fi # Name of file stem

    param1=${2}; if [ -z $param1 ]; then param1=0; 		fi 	# parameter 1
	param2=${3}; if [ -z $param2  ]; then param2=0; 		fi # parameter 2
    param3=${4}; if [ -z $param3 ]; then param3=0; 		fi 	# parameter 3
	param4=${5}; if [ -z $param4  ]; then param4=0; 		fi # parameter 4
	param5=${6}; if [ -z $param5 ]; then param5=0; 		fi 	# parameter 5
    param6=${7}; if [ -z $param6 ]; then param6=0; 		fi 	# parameter 6
	param7=${8}; if [ -z $param7  ]; then param7=0; 		fi # parameter 7
    param8=${9}; if [ -z $param8 ]; then param8=0; 		fi 	# parameter 8
	param9=${10}; if [ -z $param9  ]; then param9=0; 		fi # parameter 9
	param10=${11}; if [ -z $param10 ]; then param10=0; 		fi 	# parameter 10
	param11=${12}; if [ -z $param11 ]; then param11=0; 		fi 	# parameter 11
	param12=${13}; if [ -z $param12  ]; then param12=0; 		fi # parameter 12
    param13=${14}; if [ -z $param13 ]; then param13=0; 		fi 	# parameter 13
	param14=${15}; if [ -z $param14  ]; then param14=0; 		fi # parameter 14
	param15=${16}; if [ -z $param15 ]; then param15=0; 		fi 	# parameter 15
	param16=${17}; if [ -z $param16 ]; then param16=0; 		fi 	# parameter 16
	param17=${18}; if [ -z $param17  ]; then param17=0; 		fi # parameter 17
    param18=${19}; if [ -z $param18 ]; then param18=0; 		fi 	# parameter 18
	param19=${20}; if [ -z $param19  ]; then param19=0; 		fi # parameter 19
	param20=${21}; if [ -z $param20 ]; then param20=0; 		fi 	# parameter 20
	param21=${22}; if [ -z $param21 ]; then param21=0; 		fi 	# parameter 21

	target=${23}; if [ -z $target ]; then target=0; 	fi # target
	inputseed=${24}; if [ -z $inputseed ]; then inputseed=1; 	fi # Input seed
	wireseed=${25}; if [ -z $wireseed  ]; then wireseed=1; 		fi # Wiring seed
	calcErr=${26}; if [ -z $calcErr  ]; then calcErr=0; 		fi # calcerrseed
	mistparam1=${27}; if [ -z $mistparam1 ]; then mistparam1=""; 		fi # mistparam1

# Find NEURON
NEURON=nrniv

# Run the simulation
echo $NEURON -c "inputseed=$inputseed" -c "dvseed=$wireseed" -c "strdef filestem" -c "filestem=\"$filestem\"" -c "param1 = $param1" -c "param2 = $param2" -c "param3 = $param3" -c "param4 = $param4" -c "param5 = $param5" -c "param6 = $param6" -c "param7 = $param7" -c "param8 = $param8" -c "param9 = $param9" -c "param10 = $param10" -c "param11 = $param11" -c "param12 = $param12" -c "param13 = $param13" -c "param14 = $param14" -c "param15 = $param15" -c "param16 = $param16" -c "param17 = $param17" -c "param18 = $param18" -c "param19 = $param19" -c "param20 = $param20" -c "param21 = $param21" -c "ptarget = $target" -c "calcErr = $calcErr" -c "strdef mistparam1" -c "mistparam1=\"$mistparam1\"" main_stim.hoc #>> $codefile # Set and run
$NEURON -c "inputseed=$inputseed" -c "dvseed=$wireseed" -c "strdef filestem" -c "filestem=\"$filestem\"" -c "param1 = $param1" -c "param2 = $param2" -c "param3 = $param3" -c "param4 = $param4" -c "param5 = $param5" -c "param6 = $param6" -c "param7 = $param7" -c "param8 = $param8" -c "param9 = $param9" -c "param10 = $param10" -c "param11 = $param11" -c "param12 = $param12" -c "param13 = $param13" -c "param14 = $param14" -c "param15 = $param15" -c "param16 = $param16" -c "param17 = $param17" -c "param18 = $param18" -c "param19 = $param19" -c "param20 = $param20" -c "param21 = $param21" -c "ptarget = $target" -c "calcErr = $calcErr" -c "strdef mistparam1" -c "mistparam1=\"$mistparam1\"" main_stim.hoc #>> $codefile # Set and run

echo 'Simulation complete...'

elapsed=$(( `date +%s` - tic )) # Find out elapsed time
echo "runsim: done; elapsed time: $elapsed s." # | tee -a $codefile # Send elapsed time to display, and also save to sim file

Loading data, please wait...