Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:183014
We developed a 3-layer sensorimotor cortical network of consisting of 704 spiking model-neurons, including excitatory, fast-spiking and low-threshold spiking interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a virtual musculoskeletal human arm, with realistic anatomical and biomechanical properties, to reach a target. Virtual arm position was used to simultaneously control a robot arm via a network interface.
Reference:
1 . Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [PubMed]
2 . Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring Behavior via Inverse Neurocontroller in a Lesioned Cortical Spiking Model Driving a Virtual Arm. Front Neurosci 10:28 [PubMed]
Citations  Citation Browser
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 interneuron basket PV GABA cell; Neocortex fast spiking (FS) interneuron; Neostriatum fast spiking interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON; Python (web link to model);
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing; Motor control; Touch;
Implementer(s): Neymotin, Sam [Samuel.Neymotin at nki.rfmh.org]; Dura, Salvador [ salvadordura at gmail.com];
Search NeuronDB for information about:  Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 interneuron basket PV GABA cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
/
arm2dms_modeldb
mod
drspk.mod *
infot.mod *
intf6.mod *
intfsw.mod *
misc.mod *
mySTDP.mod
nsloc.mod
nstim.mod *
place.mod *
stats.mod *
updown.mod *
vecst.mod *
parameters.multi
                            
: $Id: drspk.mod,v 1.27 2011/11/22 04:23:40 samn Exp $

UNITS {
    (mV) = (millivolt)
    (nA) = (nanoamp)
}

NEURON {
  POINT_PROCESS DRSPK
  GLOBAL refrac, vrefrac, rdmthresh
  RANGE drive,rand,inrefrac,fflag,qq,thresh
  NONSPECIFIC_CURRENT i
}

PARAMETER {
  refrac = 5 (ms)
  vrefrac = 0 (mV)
  drive = 0
  rand = 1
  fflag=1
  i = 0 (nA)
  rdmthresh = 0
}

ASSIGNED {
  : i (nA)
  v (mV)
  inrefrac
  qq
  thresh
}

CONSTRUCTOR {
  VERBATIM 
  ENDVERBATIM
}

INITIAL {
  net_send(0, 3)
  i=0
  drive=0
  qq=0
  if (rdmthresh) {
    rand=1
    thresh=1
  }
  inrefrac=0
}

BREAKPOINT {
  if (inrefrac) {
    qq = 0
  } else {
    qq = drive
    thresh = rand
  }
  i = -qq
}

NET_RECEIVE(w) {
  if (flag == 1 && !inrefrac) {
    net_event(t)
    net_send(refrac, 2)
    v = vrefrac
    inrefrac=1
    qq = 0
  } else if (flag == 2) {
    inrefrac=0
    if(v > thresh) { net_send(0,1) }
  } else if (flag == 3) {
    WATCH (v>thresh) 1
  }	
}

:** vers gives version
PROCEDURE vers () {
  printf("$Id: drspk.mod,v 1.27 2011/11/22 04:23:40 samn Exp $\n")
}