Motor system model with reinforcement learning drives virtual arm (Dura-Bernal et al 2017)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:194897
"We implemented a model of the motor system with the following components: dorsal premotor cortex (PMd), primary motor cortex (M1), spinal cord and musculoskeletal arm (Figure 1). PMd modulated M1 to select the target to reach, M1 excited the descending spinal cord neurons that drove the arm muscles, and received arm proprioceptive feedback (information about the arm position) via the ascending spinal cord neurons. The large-scale model of M1 consisted of 6,208 spiking Izhikevich model neurons [37] of four types: regular-firing and bursting pyramidal neurons, and fast-spiking and low-threshold-spiking interneurons. These were distributed across cortical layers 2/3, 5A, 5B and 6, with cell properties, proportions, locations, connectivity, weights and delays drawn primarily from mammalian experimental data [38], [39], and described in detail in previous work [29]. The network included 486,491 connections, with synapses modeling properties of four different receptors ..."
Reference:
1 . Dura-Bernal S, Neymotin SA, Kerr CC, Sivagnanam S, Majumdar A, Francis JT, Lytton WW (2017) Evolutionary algorithm optimization of biological learning parameters in a biomimetic neuroprosthesis. IBM Journal of Research and Development (Computational Neuroscience special issue) 61(2/3):6:1-6:14
Citations  Citation Browser
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Abstract Izhikevich neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; GabaB; NMDA; AMPA;
Gene(s):
Transmitter(s): Glutamate; Gaba;
Simulation Environment: NEURON; Python;
Model Concept(s): Learning; Reinforcement Learning; Reward-modulated STDP; STDP; Motor control; Sensory processing;
Implementer(s): Dura-Bernal, Salvador [salvadordura at gmail.com]; Kerr, Cliff [cliffk at neurosim.downstate.edu];
Search NeuronDB for information about:  GabaA; GabaB; AMPA; NMDA; Gaba; Glutamate;
import matplotlib
# matplotlib.use('Agg') # needed for hpc batch sims

import sys
from numpy import mean, zeros
from time import time
import pickle
from neuron import h# Import NEURON
import shared as s

###############################################################################
### Update model parameters from command-line arguments
###############################################################################

for argv in sys.argv[1:]: # Skip first argument, which is file name
    arg = argv.replace(' ','').split('=') # ignore spaces and find varname and value
    harg = arg[0].split('.')+[''] # Separate out variable name; '' since if split fails need to still have an harg[1]
    if len(arg)==2:
        if hasattr(s,arg[0]) or hasattr(s,harg[1]): # Check that variable exists
            if arg[0] == 'outfilestem':
                exec('s.'+arg[0]+'="'+arg[1]+'"') # Actually set variable
                if s.rank==0: # messages only come from Master
                    print(('  Setting %s=%s' %(arg[0],arg[1])))
            else:
                exec('s.'+argv) # Actually set variable
                if s.rank==0: # messages only come from Master
                    print(('  Setting %s=%r' %(arg[0],eval(arg[1]))))
        else:
            sys.tracebacklimit=0
            raise Exception('Reading args from commandline: Variable "%s" not found' % arg[0])
    elif argv=='-mpi':   ismpi = True
    else: pass # ignore -python


###############################################################################
### Run model
###############################################################################
import network

network.runTrainTest2targets()
#network.runTrainTest2targetsOptim()

#from pylab import show; show()  # needed for hpc batch sims