Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:183014
We developed a 3-layer sensorimotor cortical network of consisting of 704 spiking model-neurons, including excitatory, fast-spiking and low-threshold spiking interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a virtual musculoskeletal human arm, with realistic anatomical and biomechanical properties, to reach a target. Virtual arm position was used to simultaneously control a robot arm via a network interface.
References:
1 . Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [PubMed]
2 . Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring Behavior via Inverse Neurocontroller in a Lesioned Cortical Spiking Model Driving a Virtual Arm. Front Neurosci 10:28 [PubMed]
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 interneuron basket PV GABA cell; Neocortex fast spiking (FS) interneuron; Neostriatum fast spiking interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON; Python (web link to model);
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing; Motor control; Touch;
Implementer(s): Neymotin, Sam [Samuel.Neymotin at nki.rfmh.org]; Dura, Salvador [ salvadordura at gmail.com];
Search NeuronDB for information about:  Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 interneuron basket PV GABA cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
<Simulation name="human">
	<System name="3DGaitModel2392Arms">
		<AdditionalFileDir name="./Geometry" />
		<AdditionalFileDir name="./obj_textures" />
		<OsimFile name="SUNY_arm_2DOFs_horizon.osim">
			<LoadForces load_all="true" muscle_apply_force="true" need_compute_actu="true" need_load="true" sys_active="true" use_fatigue="false">
			</LoadForces> 
			<LoadGeometry need_load="true" />
		</OsimFile>	
		<ScaleMuscleToRelax name="Muscles" />
		
        <ForceSubsystems>
            <JointLimitActutorForceSubsystem>
                <Coefficients>2500 1000 500 1500</Coefficients>
            </JointLimitActutorForceSubsystem>        
        </ForceSubsystems>

		<EventHandlers>
			<OsgEventHandler>
				<Interval> 0.01</Interval>			
				<Camera graphics_context="true" preset="4">
					<ViewUpDirection value="240 -70 200" />
					<ViewPosition relative="false" value="50 -60 100" />
					<ViewCenter relative="false" value="0 0 0" />
				</Camera>
				<Screen background_color="1.0 1.0 1.0 1.0" position="100 100" resolution="800 600" />
				<ShowBodies value="true" />
				<ShowBodyBaseLines value="false">
					<Scale value="8e-1" />
				</ShowBodyBaseLines>
				<ShowBodyFrames value="false">
					<Scale value="4e-2" />
				</ShowBodyFrames>
				<ShowJoints value="false" />
				<ShowForces value="false" />
				<ShowContactProxyGroups value="false" />
				<ShowContactGroups value="false" />
				<ShowForceSubsystem name="Muscles">
					<DefaultColor> 0.0 0.0 1.0 1.0 </DefaultColor>
					<ColorValueRange> 0.0 1.0 </ColorValueRange>
					<ColorOption value="6" />
					<Radius value="0.0025" />
				</ShowForceSubsystem>
			</OsgEventHandler>
	
			<MuscleStatusEventHandler>
				<Interval> 10e-3</Interval>
				<PntOutput name="SUNY_arm_2DOFs_horizon_static_muscle_status.pnt" /> 
				<LOAMuscleForceSubsystem name="Muscles" />
				<MuscleNames all="false">
					DELT1  DELT2 DELT3 Infraspinatus Latissimus_dorsi_1 Latissimus_dorsi_2 Latissimus_dorsi_3 Teres_minor PECM1 PECM2 PECM3 Coracobrachialis TRIlong TRIlat TRImed BIClong BICshort BRA  
				</MuscleNames>		
				<MuscleVars all="false">	
                    excitation
					activation
					force
				</MuscleVars>		
				<PortSend2>32001</PortSend2>					
			</MuscleStatusEventHandler>
			
			<MuscleExcitationSetterEventHandler>
				<Interval> 10e-3</Interval>
				<LOAMuscleForceSubsystem name="Muscles" />
				<MuscleNames all="true">
					DELT1  DELT2 DELT3 Infraspinatus Latissimus_dorsi_1 Latissimus_dorsi_2 Latissimus_dorsi_3 Teres_minor PECM1 PECM2 PECM3 Coracobrachialis TRIlong TRIlat TRImed BIClong BICshort BRA  
				</MuscleNames>						
				<Body name="SUNY_2DOF_Arm_Model" />
				<CoordinateNames all="true">
					arm_flex   elbow_flex
				</CoordinateNames>	
				<PortReceive>31000</PortReceive>		
			</MuscleExcitationSetterEventHandler>

			<CoordinateOutputEventHandler>
				<Interval> 10e-3</Interval>
				<Body name="SUNY_2DOF_Arm_Model" />
				<CoordinateNames all="true">
					arm_flex   elbow_flex
				</CoordinateNames>					
				<PntOutput name="SUNY_arm_2DOFs_horizon_static_coordinate_status.pnt" /> 
				<PortSend1>32000</PortSend1>		
			</CoordinateOutputEventHandler>
		</EventHandlers>
	</System>
	
	<Dynamics>
		<PntOutput name="SUNY_arm_2DOFs_horizon_static_state.pnt" /> 
		<TimeStep> 1.0e-4 </TimeStep>
		<TimeStepper> 
			<SynTimeStepper />
		</TimeStepper>
		<Integrator>
			<EulerIntegrator />
			<LSODAIntegrator2 max_substeps="500" />
			<RungeKuttaImplicitIntegrator />
		</Integrator>
	</Dynamics>

	<EndTime>10.0</EndTime>
	
</Simulation>

Loading data, please wait...