Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:183014
We developed a 3-layer sensorimotor cortical network of consisting of 704 spiking model-neurons, including excitatory, fast-spiking and low-threshold spiking interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a virtual musculoskeletal human arm, with realistic anatomical and biomechanical properties, to reach a target. Virtual arm position was used to simultaneously control a robot arm via a network interface.
References:
1 . Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [PubMed]
2 . Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring behavior via inverse neurocontroller in a lesioned cortical spiking model driving a virtual arm. Front. Neurosci. Neuroprosthetics 10:28
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex M1 pyramidal pyramidal tract L5B cell; Neocortex M1 pyramidal intratelencephalic L2-5 cell; Neocortex M1 interneuron basket PV cell; Neocortex fast spiking (FS) interneuron; Neostriatum fast spiking interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON; Python (web link to model);
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing; Motor control;
Implementer(s): Neymotin, Sam [samn at neurosim.downstate.edu]; Dura, Salvador [ salvadordura at gmail.com];
Search NeuronDB for information about:  Neocortex M1 pyramidal intratelencephalic L2-5 cell; Neocortex M1 pyramidal pyramidal tract L5B cell; Neocortex M1 interneuron basket PV cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
/
arm2dms_modeldb
mod
drspk.mod *
infot.mod *
intf6.mod *
intfsw.mod *
misc.mod *
mySTDP.mod
nsloc.mod
nstim.mod *
place.mod *
stats.mod *
updown.mod *
vecst.mod *
parameters.multi
                            
: $Id: nstim.mod,v 1.24 2006/04/03 19:18:18 billl Exp $

NEURON	{ 
  ARTIFICIAL_CELL NStim
  RANGE interval, number, start, end
  RANGE noise,type,id
}

PARAMETER {
  interval	= 10 (ms) <1e-9,1e9>: time between spikes (msec)
  number	= 10 <0,1e9>	: number of spikes
  start		= 50 (ms)	: start of first spike
  noise		= 0 <0,1>	: amount of randomeaness (0.0 - 1.0)
  end		= 1e9 (ms)	: time to terminate train
}

ASSIGNED {
  event (ms)
  on
  endt (ms)
  type
  id
}

CONSTRUCTOR {
  VERBATIM 
  { if (ifarg(1)) { id= *getarg(1); } else { id= -1; }
    if (ifarg(2)) { type= *getarg(2); } else { type= 1; }
  }
  ENDVERBATIM
}

PROCEDURE seed (x) {
  set_seed(x)
}

INITIAL {
  on = 0
  if (noise < 0) { noise = 0 }
  if (noise > 1) { noise = 1 }
  if (interval <= 0.) { interval = .01 (ms) }
  if (start>=0 && number>0 && end>0) {
    event = start + invl(interval) - interval*(1. - noise)
    if (event < 0) { event = 0 }
    net_send(event, 3)
  }
}	

PROCEDURE init_sequence (t(ms)) {
  if (number > 0) {
    on = 1
    event = t
    endt = t + 1e-6 + interval*(number-1)
  }
}

FUNCTION invl (mean (ms)) (ms) {
  if (noise == 0) {
    invl = mean
  } else {
    invl = (1. - noise)*mean + noise*mean*exprand(1)
  }
}

NET_RECEIVE (w) {
  if (flag == 0) { : external event
    if (w > 0 && on == 0) { : turn on spike sequence
      init_sequence(t)
      net_send(0, 1)
    } else if (w < 0 && on == 1) { : turn off spiking
      on = 0
    }
  }
  if (flag == 3) { : from INITIAL
    if (on == 0) {
      init_sequence(t)
      net_send(0, 1)
    }
  }
  if (flag == 1 && on == 1) {
    net_event(t)
    event = event + invl(interval)
    if (event > endt || event > end) {
      on = 0
    } else {
      net_send(event - t, 1)
    }
  }
}

FUNCTION fflag () { fflag=1 }

Loading data, please wait...