Sensorimotor cortex reinforcement learning of 2-joint virtual arm reaching (Neymotin et al. 2013)

 Download zip file   Auto-launch 
Help downloading and running models
Accession:150245
"... We developed a model of sensory and motor neocortex consisting of 704 spiking model-neurons. Sensory and motor populations included excitatory cells and two types of interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a 2-joint virtual arm to reach to a fixed target. ... "
Reference:
1 . Neymotin SA, Chadderdon GL, Kerr CC, Francis JT, Lytton WW (2013) Reinforcement learning of 2-joint virtual arm reaching in a computer model of sensorimotor cortex Neural Computation 25(12):3263-93 [PubMed]
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex V1 pyramidal corticothalamic L6 cell; Neocortex U1 pyramidal intratelencephalic L2-5 cell; Neocortex V1 interneuron basket PV cell; Neocortex fast spiking (FS) interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Channel(s):
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Gene(s):
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON;
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing;
Implementer(s): Neymotin, Sam [samn at neurosim.downstate.edu]; Chadderdon, George [gchadder3 at gmail.com];
Search NeuronDB for information about:  Neocortex V1 pyramidal corticothalamic L6 cell; Neocortex V1 interneuron basket PV cell; Neocortex U1 pyramidal intratelencephalic L2-5 cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
/
a2dmodeldb
readme.html
drspk.mod *
infot.mod *
intf6_.mod *
misc.mod *
nstim.mod *
stats.mod *
vecst.mod *
arm.hoc
basestdp.hoc
col.hoc
colors.hoc *
declist.hoc *
decmat.hoc *
decnqs.hoc *
decvec.hoc *
default.hoc *
drline.hoc *
filtutils.hoc *
geom.hoc
grvec.hoc *
hinton.hoc *
infot.hoc *
init.hoc
labels.hoc *
misc.h *
mosinit.hoc
network.hoc
nload.hoc
nqs.hoc *
nqsnet.hoc *
nrnoc.hoc *
params.hoc
python.hoc
pywrap.hoc *
run.hoc
samutils.hoc *
screenshot.png
sense.hoc *
setup.hoc *
simctrl.hoc *
stats.hoc *
stim.hoc
syncode.hoc *
trainedplast.nqs
units.hoc *
xgetargs.hoc *
                            
// $Id: decnqs.hoc,v 1.38 2011/03/01 19:06:15 billl Exp $

print "Loading decnqs.hoc..."

{load_file("nqs.hoc")}
objref nq[10],pq[10]

//** prl2nqs(NQS[,min,max,nointerp]) -- transfer printlist to NQS
proc rename () {}
// eg proc rename () { sprint($s1,"P%d",objnum($s1)) }
obfunc prl2nqs () { local tstep localobj st,oq
  st=new String2()
  oq=new NQS()
  if (numarg()>=1) min=$1 else min=0
  if (numarg()>=2) max=$2 else max=printlist.count-1
  if (numarg()>=3) interp=$3 else interp=0 // no interp when looking at spk times
  if (interp) oq.resize(max-min+2)
  if (interp) {
    tstep=0.1 // 0.1 ms step size for interpolation
    oq.s[0].s="time"
    oq.v[0].indgen(0,printlist.object(0).tvec.max,tstep)
    for ii=min,max {
      XO=printlist.object(ii)
      oq.s[ii+1-min].s = XO.var
      rename(oq.s[ii+1-min].s)
      oq.v[ii+1-min].resize(oq.v.size)
      oq.v[ii+1-min].interpolate(oq.v[0],XO.tvec,XO.vec)
    }
  } else {
    for ii=min,max {
      XO=printlist.object(ii)
      st.s=XO.name
      sprint(st.t,"%s-time",XO.name)
      rename(st.s) rename(st.t)
      oq.resize(st.s,st.t)
      oq.setcols(XO.vec,XO.tvec)
    }
  }
  return oq
}

//** pvp2nqs(NQS) -- transfer grvec data file to NQS
obfunc pvp2nqs () { local min,max,interp,gvnum,ii,jj,n localobj oq,po,st,xo
  interp=min=max=gvnum=0
  if (argtype(1)==2) { po=gvnew($s1) gvnum=panobjl.count()-1 }
  if (argtype(1)==0) gvnum=$1
  if (numarg()>=2) min=$2
  if (numarg()>=3) max=$3
  if (numarg()>=4) interp=$4 // no interp when looking at spk times
  if (po==nil) po=panobjl.o(gvnum)
  oq=new NQS() st=new String2()
  if (gvnum>0) {
    if (max==0) max=po.llist.count()-1
    for ii=min,max {
      xo=po.llist.object(ii)
      po.tmpfile.seek(xo.loc)
      if (xo.num==-2) {
        sprint(st.s,"%s-time",xo.name)
        jj=oq.resize(st.s)-1
        oq.v[jj].vread(po.tmpfile)
      }
      jj=oq.resize(xo.name)-1
      oq.v[jj].vread(po.tmpfile)
    }
  } else { // from printlist
    if (max==0) max=printlist.count()-1
    for ii=min,max {
      xo=printlist.o(ii)
      jj=oq.resize(xo.name)-1
      oq.v[jj].copy(xo.vec)
      if (xo.pstep==0) {
        sprint(st.s,"%s-time",xo.name)
        jj=oq.resize(st.s)-1
        oq.v[jj].copy(xo.tvec)
      }
    }
  }
  return oq
}

//** veclist2nqs(nqs[,STR1,STR2,...])
proc veclist2nqs () { local flag,i
  if (numarg()==0) {printf("veclist2nqs(nqs[,STR1,STR2,...])\n") return}
  $o1.resize(veclist.count)
  if (numarg()==1+$o1.m) flag=1 else flag=0
  for ltr(XO,veclist) {
    $o1.v[i1].copy(XO)
    if (flag) {i=i1+2 $o1.s[i1].s=$si} else {sprint(tstr,"v%d",i1) $o1.s[i1].s=tstr}
  }
  $o1.cpout()
}

// fudup(vec[,nq,#CUTS,LOGCUT,MIN]) -- use updown() to find spikes
// LOC(0) PEAK(1) WIDTH(2) BASE(3) HEIGHT(4) START(5) SLICES(6) SHARP(7) INDEX(8) FILE(9) NESTED(10)
// other options
pos_fudup=1 // set to 1 to move whole curve up above 0
maxp_fudup=0.95 // draw top sample at 95% of max 
minp_fudup=0.05 // draw bottom sample at 5% of max
over_fudup=1    // turn over and try again if nothing found
allover_fudup=0 // turn over and add these locs (not debugged)
verbose_fudup=0 // give messages, can also turn on DEBUG_VECST for messages from updown()
obfunc fudup () { local a,i,ii,npts,logflag,min,x,sz localobj bq,cq,v1,v2,v3,bb,tl,v5,eq
  if (verbose_fudup) printf("MAXTIME appears to be %g (dt=%g)\n",$o1.size*dt,dt)
  logflag=0  npts=10 // n sample locations by default
  bq=new NQS("LOC","PEAK","WIDTH","BASE","HEIGHT","START","SLICES","SHARP","INDEX","FILE","NESTED") 
  i=2
  if (argtype(i)==1) {i+=1 if ($o2==nil) {cq=new NQS() $o2=cq} else cq=$o2} else cq=new NQS()
  if (cq.m!=11) { cq.resize(0) 
  cq.resize("LOC","PEAK","WIDTH","BASE","HEIGHT","START","SLICES","SHARP","INDEX","FILE","NESTED")}
  if (argtype(i)==0){ npts=$i i+=1
    if (argtype(i)==0){ logflag=$i i+=1
      if (argtype(i)==1) { v5=$oi i+=1
        if (npts!=v5.size) printf("Correcting npts from %d to %d\n",npts,npts=v5.size)
        if (v5.ismono(1)) v5.reverse
        if (! v5.ismono(-1)) {printf("fudup: final arg (%s) must be monotonic\n",v5) return}
      }
    }
  }
  bq.listvecs(bb)
  bq.pad(5000)
  eq=new NQS(-2,npts) a=allocvecs(v1,v2,v3)
  tl=eq.vl
  eq.clear(2e4) vrsz(2e4,v1,v2,v3)
  v1.copy($o1)
  if (pos_fudup) {
    min=v1.min
    v1.sub(min) // make it uniformly positive
  } else min=0
  if (numarg()>4) v2.copy(v5) else {
    v2.indgen(2,2+npts-1,1)   // sampling at npts points, start at 2 to avoid log(1)=0
    if (logflag) v2.log() // log sampling
    v2.scale(-maxp_fudup*v1.max,-minp_fudup*v1.max) v2.mul(-1)
  }
  v1.updown(v2,tl,bb)
  if (pos_fudup) { bq.v[1].add(min) bq.v[3].add(min) }
  cq.append(bq)
  sz=bq.size(1)
  if (allover_fudup) { // do it upside down as well
    v1.mul(-1) // v2 will be upside-down
    if (pos_fudup) {min=v1.min v1.sub(min)}
    if (0) {  // can't see a rationale to recalc sampling points
      v2.indgen(2,2+npts-1,1)   // sampling at npts points
      if (logflag) v2.log() // log sampling
      v2.scale(-0.95*v1.max,-0.05*v1.max) v2.mul(-1)
    }
    v1.updown(v2,tl,bb)
    bq.v[8].add(sz) bq.v[4].mul(-1) // turn HEIGHT upside down
    cq.append(bq)
  } else if (over_fudup && sz==0) { // turn it over an try again
    print "fudup() checking upside-down"
    v1.mul(-1) // v2 will be upside-down
    v1.updown(v2,tl,bb)
  } 
  for case(&x,0,2,5) cq.v[x].mul(dt)
  nqsdel(bq,eq)
  dealloc(a)
  return cq
}

//** listsort(LIST[,START,REV]) sorts list of strings numerically
// optional start gives a regexp to start at
proc listsort () { local x,rev localobj nq,st,xo
  if (numarg()==0) { 
    print "listsort(LIST[,RXP,REV]) numerically, optional RXP starts after there" return}
  if (numarg()==3) if ($3) rev=-1 else rev=0
  nq=new NQS("STR","NUM") nq.strdec("STR")
  st=new String()
  for ltr(xo,$o1) {
    if (numarg()>=2) sfunc.tail(xo.s,$s2,st.s) else st.s=xo.s
    if (sscanf(st.s,"%g",&x)!=1) print "listsort ERR: num not found in ",st.s
    nq.append(xo.s,x)
  } 
  nq.sort("NUM",rev)
  $o1.remove_all
  for nq.qt(st.s,"STR") $o1.append(new String(st.s))
}

// stat(VEC) print stats for the vector
// stat(VEC1,VEC2) append stats of VEC1 on VEC2
// stat(VEC1,NQS) append stats of VEC1 on NQS (create if necessary)
proc stat () { local sz
  sz=$o1.size
  if (sz<=1) {printf("decnqs::stat() WARN: %s size %d\n",$o1,$o1.size) return}
  if (numarg()==1 && sz>1) {
    printf("Sz:%d\tmax=%g; min=%g; mean=%g; stdev=%g\n",$o1.size,$o1.max,$o1.min,$o1.mean,$o1.stdev)
  } else {
    if (!isassigned($o2)) { $o2=new NQS("SIZE","MAX","MIN","MEAN","STDEV")
    } else if (isobj($o2,"NQS")) { 
      if ($o2.m!=5) {$o2.resize(0) $o2.resize("SIZE","MAX","MIN","MEAN","STDEV")}
    } else if (isobj($o2,"Vector")) revec($o2)
    if (sz>2) {$o2.append($o1.size,$o1.max,$o1.min,$o1.mean,$o1.stdev) // .append for Vector or NQS
    } else   $o2.append($o1.size,$o1.max,$o1.min,$o1.min,0) // no sdev
  }
}

//* fil2nqs(FILE,NQS) reads lines of file and places all numbers in NQS
func fil2nqs () { local a,n localobj v1
  $o2.clear
  a=allocvecs(v1)
  tmpfile.ropen($s1)
  for (n=1;tmpfile.gets(tstr)!=-1;n+=1) {
    if (n%1e3==0) printf("%d ",n)
    parsenums(tstr,v1)
    if (v1.size!=$o2.m) {
      printf("Wrong size at line %d (%d)  ",n,v1.size)  vlk(v1)
      return
    }
    $o2.append(v1)
  }
  dealloc(a)
  return $o2.size(1)
}

//** plnqs(file,NQS) reads output of txt2num.pl
// format ascii 'rows cols' then binary contents
proc plnqs () { local a,rows,cols localobj v1,v2
  a=allocvecs(v1,v2)
  tmpfile.ropen($s1)
  tmpfile.gets(tstr)
  sscanf(tstr,"%d %d",&rows,&cols)
  printf("%s: %d rows x %d cols\n",$s1,rows,cols)
  v1.fread(tmpfile,rows*cols) // could now use .transpose
  v2.indgen(0,rows*cols,cols)
  $o2.resize(cols,rows)
  for ii=0,cols-1 {
    v2.add(ii)
    $o2.v[ii].index(v1,v2)
  }
  dealloc(a)
}
    
// DEST=maxem(SRC,MIN,WIDTH) -- keep looking for maxima till get down to min
obfunc maxem () { local a,min,wid,ii,ix,beg,end localobj v1,aq
  a=allocvecs(v1)
  aq=new NQS("max","loc") aq.clear(v1.size/2)
  v1.copy($o1)
  min=$2 wid=$3
  while(v1.max>min) {
   aq.append(v1.max,ix=v1.max_ind)
   beg=ix-wid if (beg<0) beg=0
   end=ix+wid if (end>=v1.size) end=v1.size-1
   for ii=beg,end v1.x[ii]=-1e9
  }
  dealloc(a)
  return aq
}

// nqo=percl(nq,"COLA", ..) generates NQS of percentile values (10..90) for these cols
// nqo=percl(nq,min,max,step,"COLA", ..) -- eg percl(nq,50,70,5,"COLA","COLB")
obfunc percl () {  local i,ii,a,p localobj v1,v2,v3,aq,xo
  a=allocvecs(v1,v2,v3)
  aq=new NQS(numarg())
  if (argtype(2)==0) {
    v3.indgen($2,$3,$4) 
    aq.resize(aq.size(1)-3)
    i=5 j=4 // start at arg i and aq col #j
  } else {
    v3.indgen(10,90,10)
    i=2 j=1
  }
  aq.setcol(0,"PERCL",v3)
  for (;i<=numarg();i+=1) {
    $o1.getcol($si,v1)
    v1.sort()
    v2.resize(0)
    for vtr(&ii,v3) {ii/=100 v2.append(v1.x[round(ii*v1.size)])}
    aq.setcol(i-j,$si,v2)
  }
  dealloc(a)
  return aq
}

// pqunq(NQS) returns columns of sorted nique values corresponding to the arg
// NB: does not produce a rectangular array
obfunc pqunq () { local a localobj v1,v2,aq
  aq=new NQS()
  a=allocvecs(v1,v2,1e5)
  aq.sethdrs($o1)
  aq.resize(-2) 
  for ii=0,aq.m-1 {
    $o1.getcol(ii,v1)
    v1.sort 
    v2.redundout(v1)
    aq.v[ii].copy(v2)
  }
  dealloc(a)
  return aq
}

//** aa=seqind(ind) -- find beginning and end of sequential indices with 
obfunc seqind () { local a,n,skip,ii,x,last localobj vi,oq
  vi=$o1
  if (numarg()>=2) skip=$2+1 else skip=1
  if (numarg()>=3) oq=$o3
  if (!isassigned(oq)) {oq=new NQS() if (numarg()>=3) $o3=oq}
  if (oq.m!=3) { oq.resize(0) oq.resize("beg","end","diff") }
  oq.clear()
  n=last=0
  for ii=1,vi.size(1)-1 {
    if (vi.x[ii]-vi.x[ii-1]>skip) {
      if (n>0) oq.append(vi.x[last],vi.x[ii-1],0)
      last=ii
      n=0
    } else n+=1
  }
  if (n>0) oq.append(vi.x[last],vi.x[ii-1],0)
  oq.pad()
  oq.calc("<diff>.copy(<end>.c.sub(<beg>))")
  return oq
}

//** list_transpose
proc list_transpose () { localobj aq,mat,xo,inlist,outlist
  aq=new NQS() inlist=$o1 outlist=$o2
  if (!isojt(outlist,inlist)) {outlist=new List() $o2=outlist}
  for ltr(xo,inlist) aq.resize("",xo)
  mat=aq.tomat(1) // transpose
  aq.frmat(mat)
  outlist.remove_all
  aq.listvecs(outlist)
  delnqs(aq)
}

//* Sam's additions -- moved from nqs_utils.hoc
//get row of Vectors
//$o1 = nqs
//$2 = row number
//$s3 - $snumarg() - name of cols to get values for
//returns list with associated Vectors
obfunc getobjrow(){ local i,rowid localobj nq,vt,ls
  nq=$o1 rowid=$2
  ls=new List()
  vt=new Vector()
  for(i=3;i<=numarg();i+=1){
    nq.get($si,rowid,vt)
    ls.append(vt)
  }
  return ls
}

//get column of objects as Vector using oform
//$o1 = nqs
//$s2 = col name
//$3=iff==1 return list of Vectors in column, else return vector of oform of each row in column
obfunc getobjcol(){ local idx,getl localobj nq,vt,vt2,ls
  nq=$o1
  if(numarg()>2) getl=$3 else getl=0
  if(getl){
    ls=new List()
    vt=new Vector()
    for idx=0,nq.size-1{
      nq.get($s2,idx,vt)
      ls.append(vt)
    }
    return ls
  } else {
    vt=new Vector(nq.size)
    vt.resize(0)
    vt2=new Vector()
    for idx=0,nq.size-1{
      nq.get($s2,idx,vt2)
      vt.append(oform(vt2))
    }
    return vt
  }
}

//get correlation between 2 columns of an NQS
//$o1=nqs
//$s2=column name
//$s3=column name
//$4=pearson correlation iff == 1 (default), otherwise spearman
func nqcor(){ local pc localobj nq1,v1,v2
  if(numarg()>3) pc=$4 else pc=1
  nq1=$o1
  v1=nq1.getcol($s2)  v2=nq1.getcol($s3)
  if (pc) return v1.pcorrel(v2) else return v1.scorrel(v2)
}

//func nqgrslice(){ local startidx,endidx localobj nq,vtmp
//  nq=$o1 startidx=$2 endidx=$3
//  vtmp=new Vector($3-$2+1)
//  gg(
//}

func MIN(){ if($1<$2)return $1 else return $2 }

//get correlation matrix/nqs of all columns to all columns
//$o1 = NQS
//$2 = num columns 0 - NQS.m
//$3 = start row/index
//$4 = end row/index
//$5 = index increment
//$6 = window size , iff <= 0, do full columns against each other
obfunc nqcolcor(){ local startidx,endidx,inct,wint,c1,c2,ncol localobj vhr,vhl,nqc,nqf
 if(numarg()<1){
   printf("nqcolcor usage: \n\t$o1 = NQS\
                           \n\t$2 = num columns 0 - NQS.m\
                           \n\t$3 = start row/index\
                           \n\t$4 = end row/index\
                           \n\t$5 = index increment\
                           \n\t$6 = window size , iff <= 0, do full columns against each other\n")
   return nil
 }
 nqf=$o1
 if(numarg()>1) ncol=$2 else ncol=nqf.m
 if(numarg()>2) startidx=$3 else startidx=0
 if(numarg()>3) endidx=$4 else endidx=nqf.size
 if(numarg()>4) inct=$5 else inct=50*2//50ms
 if(numarg()>5) wint=$6 else wint=100*2//50ms

 vhr=new Vector(wint) vhl=new Vector(wint)

 if(wint<=0){ //full column cross-correlation
   nqc=new NQS("ID0","ID1","cor") 
   for c1=0,ncol-1{
     for c2=c1+1,ncol-1{
       nqc.append(c1,c2,nqf.v[c1].pcorrel(nqf.v[c2]))
     }
   }
 } else { //cross correlation using slices of column
   nqc=new NQS("ID0","ID1","start","end","cor") 
   for c1=0,ncol-1{
     for c2=c1+1,ncol-1{
       for(startidx=0;startidx<endidx;startidx+=inct){
         vhl.copy(nqf.v[c1],startidx,MIN(startidx+wint,endidx-1))
         vhr.copy(nqf.v[c2],startidx,MIN(startidx+wint,endidx-1))
         nqc.append(c1,c2,startidx,startidx+wint,vhl.pcorrel(vhr))
       }
     }
   }
 }

 return nqc
}

 
//read wmf ascii file (just skips header and calls rdcol)
//$s1 = wmf file path
//$2 = # of columns
// obfunc rdwmf(){ local idx,jdx,hdrlines localobj nq,myf,myftmp,strf,str,strtmp,lcols
// myf=new File() myftmp=new File() strf=new StringFunctions() str=new String() lcols=new List()
// strtmp=new String() hdrlines=6
// myf.ropen($s1)
// if(!myf.is_open()){
//  printf("rdwmf ERRA: couldn't open wmf file %s for read\n",$s1)
//  return nil
// }
// for idx=0,hdrlines-1{
//  if(myf.gets(str.s)==-1){
//    printf("rdwmf ERRB: corrupt header\n")
//    return nil
//  } else if(idx==2){
//    jdx=strf.tail(str.s,"",strtmp.s)            
//  }
// }
// myf.close()
// return nq
// }

//draw regression line
//$o1 = nqs, $s2 = column 1, $s3 = column 2
// or
//$o1 = Vector 1 , $o2 = Vector 2
//returns vo
obfunc drawregline(){ local x0,y0,x1,y1,gvtmp,r localobj vo,nq,v1,v2,vx,vy,str
  vo=new Vector(5)
  if(numarg()==3){
    nq = $o1
    v1=new Vector()  v2=new Vector()
    nq.getcol($s2,v1)
    nq.getcol($s3,v2)
  } else {
    v1=$o1 v2=$o2
  }
  v1.vstats(v2,vo)
  x0 = v2.min
  y0 = x0*vo.x(0)+vo.x(1)
  x1 = v2.max
  y1 = x1*vo.x(0)+vo.x(1)
  vx=new Vector(2)
  vx.x(0)=x0
  vx.x(1)=x1
  vy=new Vector(2)
  vy.x(0)=y0
  vy.x(1)=y1
  gvtmp=gvmarkflag
  gvmarkflag=0
  gg(vy,vx)
  gvmarkflag=gvtmp
  str=new String()
  r=v1.pcorrel(v2)
  if(name_declared("rpval_stats")){
    sprint(str.s,"r = %.2f, p = %g, N = %d",r,rpval_stats(v1.size,r),v1.size)
    g.label(0,0,str.s)
  }
  return vo
}

// select from a vector handled as a matrix -- see matrix.mod
halfmat=0
obfunc mindsel () { local a,x,r,c localobj vm,vi,oq
  vm=$o1 
  a=allocvecs(vi)
  if (numarg()==4) vi.indvwhere(vm,$s2,$3,$4) else vi.indvwhere(vm,$s2,$3)
  oq=new NQS("row","col","val") 
  oq.clear(vi.size)
  for vtr(&x,vi) {
    r=int(x/COLS) c=x-r*COLS
    if (!halfmat || c>r) oq.append(r,c,vm.x[x])
  }
  dealloc(a)
  print oq.size(1)
  return oq
}

//* return row $2 of nqs $o1 
obfunc nqrow () { local row,col localobj vout,nq
  nq=$o1
  row=$2
  vout=new Vector(nq.m)
  for col=0,nq.m-1 vout.x(col)=nq.v[col].x(row)
  return vout
}

//* find row $o2 (vector) in $o1 (nqs) and return index
//  if not there return -1
func nqfindrow () { local idx,jdx,sz localobj nq,vf,vrow
  nq=$o1  vf=$o2
  sz=nq.size(-1)
  for idx=0,sz-1 {
    vrow = nqrow(nq,idx)
    if(vrow.eq(vf)) return idx
  }
  return -1
}

//* nquniq(NQS) -- return a new NQS with unique rows in $o1
obfunc nquniq () { local sz,idx,outrow,jdx localobj nqin,nqout,vrow
  nqin=$o1
  nqout=new NQS()
  for idx=0,nqin.m-1{
    nqout.resize(nqin.s[idx].s)
    nqout.v[nqout.m-1].resize(nqin.v[idx].size)
  }
  sz=nqin.size(-1)
  jdx=0
  outrow=0
  for idx=0,sz-1{
    vrow=nqrow(nqin,idx)
    if(nqfindrow(nqout,vrow)==-1){
      for jdx=0,nqin.m-1 nqout.v[jdx].x(outrow) = vrow.x(jdx)
      outrow += 1
    }
  }
  for idx=0,nqout.m-1 nqout.v[idx].resize(outrow)
  return nqout
}

Neymotin SA, Chadderdon GL, Kerr CC, Francis JT, Lytton WW (2013) Reinforcement learning of 2-joint virtual arm reaching in a computer model of sensorimotor cortex Neural Computation 25(12):3263-93[PubMed]

References and models cited by this paper

References and models that cite this paper

Afshar A, Santhanam G, Yu BM, Ryu SI, Sahani M, Shenoy KV (2011) Single-trial neural correlates of arm movement preparation. Neuron 71:555-64

Almassy N, Edelman GM, Sporns O (1998) Behavioral constraints in the development of neuronal properties: a cortical model embedded in a real-world device. Cereb Cortex 8:346-61 [PubMed]

Bannister AP (2005) Inter- and intra-laminar connections of pyramidal cells in the neocortex. Neurosci Res 53:95-103 [PubMed]

Bedau MA (2005) Artificial life: more than just building and studying computational systems. Artif Life 11:1-3 [PubMed]

Berthier N (2011) The syntax of human infant reaching 8th International Conference on Complex Systems :1477-1487

Berthier NE, Clifton RK, McCall DD, Robin DJ (1999) Proximodistal structure of early reaching in human infants. Exp Brain Res 127:259-69

Carnevale NT, Hines ML (2006) The NEURON Book

Chadderdon GL, Neymotin SA, Kerr CC, Lytton WW (2012) Reinforcement learning of targeted movement in a spiking neuronal model of motor cortex PLoS ONE 2012 7(10):e47251 [Journal]

   Reinforcement learning of targeted movement (Chadderdon et al. 2012) [Model]

Cools R (2006) Dopaminergic modulation of cognitive function-implications for L-DOPA treatment in Parkinson's disease. Neurosci Biobehav Rev 30:1-23 [PubMed]

Corbetta D, Snapp-Childs W (2009) Seeing and touching: the role of sensory-motor experience on the development of infant reaching. Infant Behav Dev 32:44-58

De Schutter E (2008) Why are computational neuroscience and systems biology so separate? PLoS Comput Biol 4:e1000078 [Journal] [PubMed]

Dyhrfjeld-Johnsen J, Santhakumar V, Morgan RJ, Huerta R, Tsimring L, Soltesz I (2007) Topological determinants of epileptogenesis in large-scale structural and functional models of the dentate gyrus derived from experimental data. J Neurophysiol 97:1566-87 [Journal] [PubMed]

   Dentate gyrus (Morgan et al. 2007, 2008, Santhakumar et al. 2005, Dyhrfjeld-Johnsen et al. 2007) [Model]

Edelman GM (1987) Neural Darwinism: The Theory of Neural Group Selection

Edelman GM (2004) Wider than the sky: The phenomenal gift of consciousness

Engel A, Konig P, Kreiter A, Gray C, Singer W (1991) Temporal coding by coherent oscillations as a potential solution to the binding problem: physiological evidence Nonlinear dynamics and neural networks, Schuster HG, ed.

Evans RC, Morera-Herreras T, Cui Y, Du K, Sheehan T, Kotaleski JH, Venance L, Blackwell KT (2012) The effects of NMDA subunit composition on calcium influx and spike timing-dependent plasticity in striatal medium spiny neurons. PLoS Comput Biol 8:e1002493 [Journal] [PubMed]

   NMDA subunit effects on Calcium and STDP (Evans et al. 2012) [Model]

Farries MA, Fairhall AL (2007) Reinforcement learning with modulated spike timing dependent synaptic plasticity. J Neurophysiol 98:3648-65 [PubMed]

Fenton AA, Lytton WW, Barry JM, Lenck-Santini PP, Zinyuk LE, Kubik S, Bures J, Poucet B, Mull (2010) Attention-like modulation of hippocampus place cell discharge. J Neurosci 30:4613-25

Florian RV (2007) Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Comput 19:1468-502 [PubMed]

Frank MJ, O'reilly RC (2006) A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci 120:497-517 [PubMed]

Frank MJ, Seeberger LC, O`Reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940-3 [Journal] [PubMed]

   Dynamic dopamine modulation in the basal ganglia: Learning in Parkinson (Frank et al 2004,2005) [Model]

Gourevitch B, Eggermont JJ (2007) Evaluating information transfer between auditory cortical neurons. J Neurophysiol 97:2533-43 [PubMed]

Graybiel AM, Aosaki T, Flaherty AW, Kimura M (1994) The basal ganglia and adaptive motor control. Science 265:1826-31 [PubMed]

Hikosaka O, Nakamura K, Sakai K, Nakahara H (2002) Central mechanisms of motor skill learning. Curr Opin Neurobiol 12:217-22

Hosp JA, Pekanovic A, Rioult-Pedotti MS, Luft AR (2011) Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J Neurosci 31:2481-7 [PubMed]

Houk JC, Wise SP (2004) Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb Cortex 5:95-110 [PubMed]

Izhikevich EM (2007) Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling. Cereb Cortex 17(10):2443-2452 [Journal] [PubMed]

   Linking STDP and Dopamine action to solve the distal reward problem (Izhikevich 2007) [Model]

Jones SR, Kerr CE, Wan Q, Pritchett DL, Hamalainen M, Moore CI (2010) Cued spatial attention drives functionally relevant modulation of the mu rhythm in primary somatosensory cortex. J Neurosci 30:13760-5 [PubMed]

Kelemen E, Fenton AA (2010) Dynamic grouping of hippocampal neural activity during cognitive control of two spatial frames. PLoS Biol 8:e1000403 [PubMed]

Kerr CC, Neymotin SA, Chadderdon GL, Fietkiewicz CT, Francis JT, Lytton WW (2012) Electrostimulation as a prosthesis for repair of information flow in a computer model of neocortex IEEE Transactions on Neural Systems & Rehabilitation Engineering 20(2):153-60 [Journal] [PubMed]

   Prosthetic electrostimulation for information flow repair in a neocortical simulation (Kerr 2012) [Model]

Kerr CC, Van Albada SJ, Neymotin SA, Chadderdon GL, Robinson PA, Lytton WW (2013) Cortical information flow in Parkinson's disease: a composite network-field model. Front Comput Neurosci 7:39:1-14 [Journal] [PubMed]

   Composite spiking network/neural field model of Parkinsons (Kerr et al 2013) [Model]

Kubikova L, Kostal L (2010) Dopaminergic system in birdsong learning and maintenance. J Chem Neuroanat 39:112-23

Le Novere N (2007) The long journey to a Systems Biology of neuronal function. BMC Syst Biol 1:28-23

Luft AR, Schwarz S (2009) Dopaminergic signals in primary motor cortex. Int J Dev Neurosci 27:415-21 [PubMed]

Lungarella M, Sporns O (2006) Mapping information flow in sensorimotor networks. PLoS Comput Biol 2:e144 [PubMed]

Lytton WW (2008) Computer modelling of epilepsy. Nat Rev Neurosci 9:626-37 [Journal] [PubMed]

Lytton WW, Neymotin SA, Hines ML (2008) The virtual slice setup. J Neurosci Methods 171:309-15 [Journal] [PubMed]

   The virtual slice setup (Lytton et al. 2008) [Model]

Lytton WW, Omurtag A (2007) Tonic-clonic transitions in computer simulation. J Clin Neurophysiol 24:175-81 [PubMed]

   Tonic-clonic transitions in a seizure simulation (Lytton and Omurtag 2007) [Model]

Lytton WW, Omurtag A, Neymotin SA, Hines ML (2008) Just in time connectivity for large spiking networks Neural Comput 20(11):2745-56 [Journal] [PubMed]

   JitCon: Just in time connectivity for large spiking networks (Lytton et al. 2008) [Model]

Lytton WW, Stewart M (2005) A rule-based firing model for neural networks Int J Bioelectromagn 7:47-50

Lytton WW, Stewart M (2006) Rule-based firing for network simulations. Neurocomputing 69:1160-1164

Mahmoudi B, Sanchez JC (2011) A symbiotic brain-machine interface through value-based decision making. PLoS One 6:e14760-23

Marsh B, Tarigoppula A, Francis J (2011) Correlates of reward expectation in the primary motor cortex: Developing an actor-critic model in macaques for a brain computer interface Society for Neuroscience Abstracts, 41

Mo J, Schroeder CE, Ding M (2011) Attentional modulation of alpha oscillations in macaque inferotemporal cortex. J Neurosci 31:878-82

Molina-Luna K, Pekanovic A, Rohrich S, Hertler B, Schubring-Giese M, Rioult-Pedotti MS, Luft (2009) Dopamine in motor cortex is necessary for skill learning and synaptic plasticity. PLoS One 4:e7082-21 [PubMed]

Neymotin S, Kerr C, Francis J, Lytton W (2011) Training oscillatory dynamics with spike-timing-dependent plasticity in a computer model of neocortex Signal Processing in Medicine and Biology Symposium (SPMB), IEEE :1-6

Neymotin SA, Jacobs KM, Fenton AA, Lytton WW (2011) Synaptic information transfer in computer models of neocortical columns. J Comput Neurosci. 30(1):69-84 [Journal] [PubMed]

   Synaptic information transfer in computer models of neocortical columns (Neymotin et al. 2010) [Model]

Neymotin SA, Lazarewicz MT, Sherif M, Contreras D, Finkel LH, Lytton WW (2011) Ketamine disrupts theta modulation of gamma in a computer model of hippocampus Journal of Neuroscience 31(32):11733-11743 [Journal] [PubMed]

   Ketamine disrupts theta modulation of gamma in a computer model of hippocampus (Neymotin et al 2011) [Model]

Neymotin SA, Lee H, Park E, Fenton AA, Lytton WW (2011) Emergence of physiological oscillation frequencies in a computer model of neocortex. Front Comput Neurosci 5:19-75 [Journal] [PubMed]

   Emergence of physiological oscillation frequencies in neocortex simulations (Neymotin et al. 2011) [Model]

Pastalkova E, Serrano P, Pinkhasova D, Wallace E, Fenton AA, Sacktor TC (2006) Storage of spatial information by the maintenance mechanism of LTP. Science 313:1141-4 [PubMed]

Peterson BE, Healy MD, Nadkarni PM, Miller PL, Shepherd GM (1996) ModelDB: an environment for running and storing computational models and their results applied to neuroscience. J Am Med Inform Assoc 3:389-98 [Journal] [PubMed]

Potjans W, Morrison A, Diesmann M (2009) A spiking neural network model of an actor-critic learning agent. Neural Comput 21:301-39 [PubMed]

Qiu S, Anderson CT, Levitt P, Shepherd GM (2011) Circuit-specific intracortical hyperconnectivity in mice with deletion of the autism-associated Met receptor tyrosine kinase. J Neurosci 31:5855-64

Reid RC (2012) From functional architecture to functional connectomics. Neuron 75:209-17

Reynolds JN, Wickens JR (2005) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 15:507-21 [PubMed]

Roberts PD, Bell CC (2002) Spike timing dependent synaptic plasticity in biological systems. Biol Cybern 87:392-403 [PubMed]

Rowan MS,Neymotin SA (2013) Synaptic Scaling Balances Learning in a Spiking Model of Neocortex Adaptive and Natural Computing Algorithms, Tomassini M, Antonioni A, Daolio F, Buesser P, ed. pp.20 [Journal]

   Synaptic scaling balances learning in a spiking model of neocortex (Rowan & Neymotin 2013) [Model]

Sanes JN (2003) Neocortical mechanisms in motor learning. Curr Opin Neurobiol 13:225-31

Seung HS (2003) Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40:1063-73 [PubMed]

Shadmehr R, Krakauer JW (2008) A computational neuroanatomy for motor control. Exp Brain Res 185:359-81 [PubMed]

Shadmehr R, Wise S (2005) The computational neurobiology of reaching and pointing: a foundation for motor learning

Shen W, Flajolet M, Greengard P, Surmeier DJ (2008) Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321:848-51 [PubMed]

Shepherd G (2004) The synaptic organization of the brain, Shepherd GM, ed.

Sober SJ, Brainard MS (2009) Adult birdsong is actively maintained by error correction. Nat Neurosci 12:927-31 [PubMed]

Song S, Miller KD, Abbott LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 3:919-26 [PubMed]

Sporns O, Tononi G, Kotter R (2005) The human connectome: A structural description of the human brain. PLoS Comput Biol 1:e42-308

Thomson AM, Lamy C (2007) Functional maps of neocortical local circuitry. Front Neurosci 1:19-42 [PubMed]

Thomson AM, West DC, Wang Y, Bannister AP (2002) Synaptic connections and small circuits involving excitatory and inhibitory neurons in layers 2-5 of adult rat and cat neocortex: triple intracellular recordings and biocytin labelling in vitro. Cereb Cortex 12:936-53 [PubMed]

Thorndike E (1911) Animal intelligence

Tiesinga PH, Sejnowski TJ (2004) Rapid temporal modulation of synchrony by competition in cortical interneuron networks. Neural Comput 16:251-75 [PubMed]

Uhlhaas PJ, Singer W (2006) Neural synchrony in brain disorders: relevance for cognitive dysfunctions and pathophysiology. Neuron 52:155-68 [PubMed]

von der Malsburg C, Schneider W (1986) A neural cocktail-party processor. Biol Cybern 54:29-40 [PubMed]

von Kraus LM, Sacktor TC, Francis JT (2010) Erasing sensorimotor memories via PKMzeta inhibition. PLoS One 5:e11125-81

Von_hofsten C (1979) Development of visually directed reaching: The approach phase Department Of Psychology, University Of Uppsala [psykologiska Inst , Uppsala Univ ]

Webb B (2000) What does robotics offer animal behaviour? Anim Behav 60:545-558

Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring behavior via inverse neurocontroller in a lesioned cortical spiking model driving a virtual arm. Front. Neurosci. Neuroprosthetics 10:28 [Journal]

   Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015) [Model]

Dura-Bernal S, Neymotin SA, Kerr CC, Sivagnanam S, Majumdar A, Francis JT, Lytton WW (2017) Evolutionary algorithm optimization of biological learning parameters in a biomimetic neuroprosthesis. IBM Journal of Research and Development (Computational Neuroscience special issue) 61(2/3):6:1-6:14 [Journal]

   Motor system model with reinforcement learning drives virtual arm (Dura-Bernal et al 2017) [Model]

Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [Journal] [PubMed]

   Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015) [Model]

Eguchi A, Neymotin SA and Stringer SM (2014) Color opponent receptive fields self-organize in a biophysical model of visual cortex via spike-timing dependent plasticity 8:16. doi: Front. Neural Circuits 8:16 [Journal]

   Simulated cortical color opponent receptive fields self-organize via STDP (Eguchi et al., 2014) [Model]

(79 refs)