Reward modulated STDP (Legenstein et al. 2008)

 Download zip file 
Help downloading and running models
Accession:116837
"... This article provides tools for an analytic treatment of reward-modulated STDP, which allows us to predict under which conditions reward-modulated STDP will achieve a desired learning effect. These analytical results imply that neurons can learn through reward-modulated STDP to classify not only spatial but also temporal firing patterns of presynaptic neurons. They also can learn to respond to specific presynaptic firing patterns with particular spike patterns. Finally, the resulting learning theory predicts that even difficult credit-assignment problems, where it is very hard to tell which synaptic weights should be modified in order to increase the global reward for the system, can be solved in a self-organizing manner through reward-modulated STDP. This yields an explanation for a fundamental experimental result on biofeedback in monkeys by Fetz and Baker. In this experiment monkeys were rewarded for increasing the firing rate of a particular neuron in the cortex and were able to solve this extremely difficult credit assignment problem. ... In addition our model demonstrates that reward-modulated STDP can be applied to all synapses in a large recurrent neural network without endangering the stability of the network dynamics."
Reference:
1 . Legenstein R, Pecevski D, Maass W (2008) A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput Biol 4:e1000180 [PubMed]
Citations  Citation Browser
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism: Neocortex;
Cell Type(s):
Channel(s):
Gap Junctions:
Receptor(s):
Gene(s):
Transmitter(s):
Simulation Environment: Python; PCSIM;
Model Concept(s): Pattern Recognition; Spatio-temporal Activity Patterns; Reinforcement Learning; STDP; Biofeedback; Reward-modulated STDP;
Implementer(s):
### MATPLOTLIBRC FORMAT

# This is a sample matplotlib configuration file.  It should be placed
# in HOME/.matplotlib/matplotlibrc (unix/linux like systems) and
# C:\Documents and Settings\yourname\.matplotlib (win32 systems)
#
# By default, the installer will overwrite the existing file in the
# install path, so if you want to preserve your's, please move it to
# your HOME dir and set the environment variable if necessary.
#
# This file is best viewed in a editor which supports python mode
# syntax highlighting
#
# Blank lines, or lines starting with a comment symbol, are ignored,
# as are trailing comments.  Other lines must have the format
#
#   key : val   # optional comment
#
# Colors: for the color values below, you can either use
#  - a matplotlib color string, such as r, k, or b
#  - an rgb tuple, such as (1.0, 0.5, 0.0)
#  - a hex string, such as ff00ff  (no '#' symbol)
#  - a scalar grayscale intensity such as 0.75
#  - a legal html color name, eg red, blue, darkslategray

#### CONFIGURATION BEGINS HERE
# the default backend; one of GTK GTKAgg GTKCairo FltkAgg QtAgg TkAgg
#     Agg Cairo GD GDK Paint PS PDF SVG Template
backend      : GTKAgg
numerix      : numpy  # numpy, Numeric or numarray
#interactive  : False      # see http://matplotlib.sourceforge.net/interactive.html
#toolbar      : toolbar2   # None | classic | toolbar2
#timezone     : UTC        # a pytz timezone string, eg US/Central or Europe/Paris

# Where your matplotlib data lives if you installed to a non-default
# location.  This is where the matplotlib fonts, bitmaps, etc reside
#datapath : /home/jdhunter/mpldata


### LINES
# See http://matplotlib.sourceforge.net/matplotlib.lines.html for more
# information on line properties.
#lines.linewidth   : 1.0     # line width in points
#lines.linestyle   : -       # solid line
#lines.color       : blue
#lines.marker      : None    # the default marker
#lines.markeredgewidth  : 0.5     # the line width around the marker symbol
#lines.markersize  : 6            # markersize, in points
#lines.dash_joinstyle : miter        # miter|round|bevel
#lines.dash_capstyle : butt          # butt|round|projecting
#lines.solid_joinstyle : miter       # miter|round|bevel
#lines.solid_capstyle : projecting   # butt|round|projecting
#lines.antialiased : True         # render lines in antialised (no jaggies)

### PATCHES
# Patches are graphical objects that fill 2D space, like polygons or
# circles.  See
# http://matplotlib.sourceforge.net/matplotlib.patches.html for more
# information on patch properties
#patch.linewidth   	: 1.0     # edge width in points
#patch.facecolor  	: blue
#patch.edgecolor  	: black
#patch.antialiased 	: True    # render patches in antialised (no jaggies)

### FONT
#
# font properties used by text.Text.  See
# http://matplotlib.sourceforge.net/matplotlib.font_manager.html for more
# information on font properties.  The 6 font properties used for font
# matching are given below with their default values.
#
# The font.family property has five values: 'serif' (e.g. Times),
# 'sans-serif' (e.g. Helvetica), 'cursive' (e.g. Zapf-Chancery),
# 'fantasy' (e.g. Western), and 'monospace' (e.g. Courier).  Each of
# these font families has a default list of font names in decreasing
# order of priority associated with them.
#
# The font.style property has three values: normal (or roman), italic
# or oblique.  The oblique style will be used for italic, if it is not
# present.
#
# The font.variant property has two values: normal or small-caps.  For
# TrueType fonts, which are scalable fonts, small-caps is equivalent
# to using a font size of 'smaller', or about 83% of the current font
# size.
#
# The font.weight property has effectively 13 values: normal, bold,
# bolder, lighter, 100, 200, 300, ..., 900.  Normal is the same as
# 400, and bold is 700.  bolder and lighter are relative values with
# respect to the current weight.
#
# The font.stretch property has 11 values: ultra-condensed,
# extra-condensed, condensed, semi-condensed, normal, semi-expanded,
# expanded, extra-expanded, ultra-expanded, wider, and narrower.  This
# property is not currently implemented.
#
# The font.size property is the default font size for text, given in pts.
# 12pt is the standard value.
#
font.family         : sans-serif
#font.style          : normal
#font.variant        : normal
#font.weight         : medium
#font.stretch        : normal
# note that font.size controls default text sizes.  To configure
# special text sizes tick labels, axes, labels, title, etc, see the rc
# settings for axes and ticks. Special text sizes can be defined
# relative to font.size, using the following values: xx-small, x-small,
# small, medium, large, x-large, xx-large, larger, or smaller
font.size           : 18.0
#font.serif          : Bitstream Vera Serif, New Century Schoolbook, Century Schoolbook L, Utopia, ITC Bookman, Bookman, Nimbus Roman No9 L, Times New Roman, Times, Palatino, Charter, serif
#font.sans-serif     : Bitstream Vera Sans, Lucida Grande, Verdana, Geneva, Lucid, Arial, Helvetica, Avant Garde, sans-serif
#font.cursive        : Apple Chancery, Textile, Zapf Chancery, Sand, cursive
#font.fantasy        : Comic Sans MS, Chicago, Charcoal, Impact, Western, fantasy
#font.monospace      : Bitstream Vera Sans Mono, Andale Mono, Nimbus Mono L, Courier New, Courier, Fixed, Terminal, monospace

### TEXT
# text properties used by text.Text.  See
# http://matplotlib.sourceforge.net/matplotlib.text.html for more
# information on text properties

#text.color          : black
#text.usetex         : False  # use latex for all text handling.  For more information, see
                             # http://www.scipy.org/Wiki/Cookbook/Matplotlib/UsingTex
#text.dvipnghack     : False  # some versions of dvipng don't handle
                             # alpha channel properly.  Use True to correct and flush
                             # ~/.matplotlib/tex.cache before testing

### AXES
# default face and edge color, default tick sizes,
# default fontsizes for ticklabels, and so on.  See
# http://matplotlib.sourceforge.net/matplotlib.axes.html#Axes
#axes.hold           : True    # whether to clear the axes by default on
#axes.facecolor      : white   # axes background color
#axes.edgecolor      : black   # axes edge color
#axes.linewidth      : 1.0     # edge linewidth
#axes.grid           : False   # display grid or not
axes.titlesize      : larger      # fontsize of the axes title
axes.labelsize      : medium      # fontsize of the x any y labels
#axes.labelcolor     : black
#axes.axisbelow      : False   # whether axis gridlines and ticks are below
                              # the axes elements (lines, text, etc)
#axes.formatter.limits : -7, 7 # use scientific notation if log10
                               # of the axis range is smaller than the
                               # first or larger than the second

#polaraxes.grid      : True    # display grid on polar axes

### TICKS
# see http://matplotlib.sourceforge.net/matplotlib.axis.html#Ticks
#xtick.major.size     : 4      # major tick size in points
#xtick.minor.size     : 2      # minor tick size in points
xtick.major.pad      : 8      # distance to major tick label in points
xtick.minor.pad      : 8      # distance to the minor tick label in points
#xtick.color          : k      # color of the tick labels
xtick.labelsize      :  medium    # fontsize of the tick labels
#xtick.direction      : in     # direction: in or out

#ytick.major.size     : 4      # major tick size in points
#ytick.minor.size     : 2      # minor tick size in points
ytick.major.pad      : 10      # distance to major tick label in points
ytick.minor.pad      : 10       # distance to the minor tick label in points
#ytick.color          : k      # color of the tick labels
ytick.labelsize      :  medium    # fontsize of the tick labels
#ytick.direction      : in     # direction: in or out


### GRIDS
#grid.color       :   black   # grid color
#grid.linestyle   :   :       # dotted
#grid.linewidth   :   0.5     # in points

### Legend
#legend.isaxes        : True
#legend.numpoints     : 2      # the number of points in the legend line
legend.fontsize      : medium
#legend.pad           : 0.2    # the fractional whitespace inside the legend border
#legend.markerscale   : 1.0    # the relative size of legend markers vs. original
# the following dimensions are in axes coords
#legend.labelsep      : 0.010  # the vertical space between the legend entries
#legend.handlelen     : 0.05   # the length of the legend lines
#legend.handletextsep : 0.02   # the space between the legend line and legend text
#legend.axespad       : 0.02   # the border between the axes and legend edge
#legend.shadow        : False

### FIGURE
# See http://matplotlib.sourceforge.net/matplotlib.figure.html#Figure
#figure.figsize   : 8, 6    # figure size in inches
#figure.dpi       : 80      # figure dots per inch
#figure.facecolor : 0.75    # figure facecolor; 0.75 is scalar gray
#figure.edgecolor : white   # figure edgecolor

# The figure subplot parameters.  All dimensions are fraction of the
# figure width or height
figure.subplot.left  : 0.1  # the left side of the subplots of the figure
figure.subplot.right : 0.9    # the right side of the subplots of the figure
#figure.subplot.bottom : 0.1   # the bottom of the subplots of the figure
#figure.subplot.top : 0.9      # the top of the subplots of the figure
#figure.subplot.wspace : 0.2   # the amount of width reserved for blank space between subplots
#figure.subplot.hspace : 0.2   # the amount of height reserved for white space between subplots


### IMAGES
#image.aspect : equal             # equal | auto | a number
#image.interpolation  : bilinear  # see help(imshow) for options
#image.cmap   : jet               # gray | jet etc...
#image.lut    : 256               # the size of the colormap lookup table
#image.origin : upper             # lower | upper


### CONTOUR PLOTS
#contour.negative_linestyle :  6.0, 6.0 # negative contour dashstyle (size in points)

### SAVING FIGURES
# the default savefig params can be different for the GUI backends.
# Eg, you may want a higher resolution, or to make the figure
# background white
#savefig.dpi       : 100      # figure dots per inch
#savefig.facecolor : white    # figure facecolor when saving
#savefig.edgecolor : white    # figure edgecolor when saving

# tk backend params
#tk.window_focus   : False    # Maintain shell focus for TkAgg
#tk.pythoninspect  : False    # tk sets PYTHONINSEPCT

# ps backend params
#ps.papersize      : letter   # auto, letter, legal, ledger, A0-A10, B0-B10
#ps.useafm         : False    # use of afm fonts, results in small files
#ps.usedistiller   : False    # can be: None, ghostscript or xpdf
                                          # Experimental: may produce smaller files.
                                          # xpdf intended for production of publication quality files,
                                          # but requires ghostscript, xpdf and ps2eps
#ps.distiller.res  : 6000      # dpi

# pdf backend params
#pdf.compression   : 6 # integer from 0 to 9
		      # 0 disables compression (good for debugging)

# svg backend params
#svg.image_inline : True       # write raster image data directly into the svg file
#svg.image_noscale : False     # suppress scaling of raster data embedded in SVG

# Set the verbose flags.  This controls how much information
# matplotlib gives you at runtime and where it goes.  Ther verbosity
# levels are: silent, helpful, debug, debug-annoying.  Any level is
# inclusive of all the levels below it.  If you setting is debug,
# you'll get all the debug and helpful messages.  When submitting
# problems to the mailing-list, please set verbose to helpful or debug
# and paste the output into your report.
#
# The fileo gives the destination for any calls to verbose.report.
# These objects can a filename, or a filehandle like sys.stdout.
#
# You can override the rc default verbosity from the command line by
# giving the flags --verbose-LEVEL where LEVEL is one of the legal
# levels, eg --verbose-helpful.
#
# You can access the verbose instance in your code
#   from matplotlib import verbose.
#verbose.level  : silent      # one of silent, helpful, debug, debug-annoying
#verbose.fileo  : sys.stdout  # a log filename, sys.stdout or sys.stderr