First-Spike-Based Visual Categorization Using Reward-Modulated STDP (Mozafari et al. 2018)

 Download zip file 
Help downloading and running models
"...Here, for the first time, we show that (Reinforcement Learning) RL can be used efficiently to train a spiking neural network (SNN) to perform object recognition in natural images without using an external classifier. We used a feedforward convolutional SNN and a temporal coding scheme where the most strongly activated neurons fire first, while less activated ones fire later, or not at all. In the highest layers, each neuron was assigned to an object category, and it was assumed that the stimulus category was the category of the first neuron to fire. ..."
1 . Mozafari M, Kheradpisheh SR, Masquelier T, Nowzari-Dalini A, Ganjtabesh M (2018) First-Spike-Based Visual Categorization Using Reward-Modulated STDP IEEE Transactions on Neural Networks and Learning Systems :1-13
Citations  Citation Browser
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Abstract integrate-and-fire neuron;
Gap Junctions:
Simulation Environment: C#;
Model Concept(s): Reward-modulated STDP; STDP; Winner-take-all; Reinforcement Learning; Temporal Coding; Vision;
Implementer(s): Mozafari, Milad [milad.mozafari at];
The "RLSNN" folder contains a Microsoft Visual Studio solution file which loads two projects. One is named "Core" which contains the source files of the proposed RL-based SNN and the other one is named "Tester" which contains source files to run the network on an image dataset.
We put a small dataset sampled from Caltech face and motorbike images by which you can examine the code. Please note that the program outputs gnuplot scripts for visualization of features and synaptic weights. You need to install gnuplot if you want to execute them (
Besides, depending on your operating system, you need to install the latest Microsoft dotNet framework on Windows, or Mono on Mac/ Linux.

In our paper, we also evaluated CNNs with the same network structure as ours. You can find related python scripts in the "CNN" folder. Those scripts are based on Keras ( and Tensorflow (

Paper details
First-Spike-Based Visual Categorization Using Reward-Modulated STDP