Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:183014
We developed a 3-layer sensorimotor cortical network of consisting of 704 spiking model-neurons, including excitatory, fast-spiking and low-threshold spiking interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a virtual musculoskeletal human arm, with realistic anatomical and biomechanical properties, to reach a target. Virtual arm position was used to simultaneously control a robot arm via a network interface.
Reference:
1 . Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [PubMed]
2 . Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring Behavior via Inverse Neurocontroller in a Lesioned Cortical Spiking Model Driving a Virtual Arm. Front Neurosci 10:28 [PubMed]
Citations  Citation Browser
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 interneuron basket PV GABA cell; Neocortex fast spiking (FS) interneuron; Neostriatum fast spiking interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON; Python (web link to model);
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing; Motor control; Touch;
Implementer(s): Neymotin, Sam [Samuel.Neymotin at nki.rfmh.org]; Dura, Salvador [ salvadordura at gmail.com];
Search NeuronDB for information about:  Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 interneuron basket PV GABA cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
/
arm2dms_modeldb
mod
drspk.mod *
infot.mod *
intf6.mod *
intfsw.mod *
misc.mod *
mySTDP.mod
nsloc.mod
nstim.mod *
place.mod *
stats.mod *
updown.mod *
vecst.mod *
parameters.multi
                            
: $Id: expsynstdp.mod,v 1.3 2012/02/17 15:45:45 samn Exp $ 
:
: basic STDP exponential synapse
: from http://www.neuron.yale.edu/neuron/static/news/stdp.mod
:

NEURON {
	POINT_PROCESS mySTDP : at a point
	RANGE tau, e, i, d, p, dtau, ptau : those values beong to this process and are not global
	NONSPECIFIC_CURRENT i : transmembrane current not necessarily calcium or specific ion
        GLOBAL verbose : all instances of this class can print you things if you put it to 1
}

UNITS {
	(nA) = (nanoamp)
	(mV) = (millivolt)
	(uS) = (microsiemens)
}

PARAMETER {
	tau = 0.1 (ms) <1e-9,1e9>
	e = 0	(mV)
	d = 0.2 <0,1>: depression factor (multiplicative to prevent < 0)
	p = 0.2 : potentiation factor (multiplicative)
	dtau = 34 (ms) : depression effectiveness time constant
	ptau = 17 (ms) : Bi & Poo (1998, 2001)
        verbose = 0 
}

ASSIGNED {
	v (mV)
	i (nA)
	tpost (ms)
}

STATE {
	g (uS)
}

INITIAL {
	g=0
	tpost = -1e9
	net_send(0, 1)
}

BREAKPOINT { : the value is updated at every time step, neuron automatically does the update for v so don't do it
	SOLVE state METHOD cnexp
	i = g*(v - e)
}

DERIVATIVE state { : put all your derivatives, they will be called at every time step
	g' = -g/tau
}

NET_RECEIVE(w (uS), A, tpre (ms)) { : presynaptic spike events are received and handled here, they always have flag 0
	INITIAL { A = 1  tpre = -1e9 }
	if (flag == 0) { : presynaptic spike  (after last post so depress)
          if(verbose) {printf("entry flag=%g t=%g w=%g A=%g tpre=%g tpost=%g\n", flag, t, w, A, tpre, tpost)}
		g = g + w*A
		tpre = t
		A = A * (1 - d*exp((tpost - t)/dtau))
	}else if (flag == 2) { : postsynaptic spike
          if(verbose) {printf("entry flag=%g t=%g tpost=%g\n", flag, t, tpost)}
		tpost = t : tracs the time of the post spike
		FOR_NETCONS(w1, A1, tp) { : also can hide NET_RECEIVE args, goes through all the sources, you can connect many spike sources to a single synapse
                  if(verbose) {printf("entry FOR_NETCONS w1=%g A1=%g tp=%g\n", w1, A1, tp)}
			A1 = A1*(1 + p*exp((tp - t)/ptau))
		}
	} else { : flag == 1 from INITIAL block
          if(verbose) {printf("entry flag=%g t=%g\n", flag, t)}
		WATCH (v > -20) 2 : if there is a postsynaptic spike, send an event with flag 2
	}
}