Cortical model with reinforcement learning drives realistic virtual arm (Dura-Bernal et al 2015)

 Download zip file 
Help downloading and running models
We developed a 3-layer sensorimotor cortical network of consisting of 704 spiking model-neurons, including excitatory, fast-spiking and low-threshold spiking interneurons. Neurons were interconnected with AMPA/NMDA, and GABAA synapses. We trained our model using spike-timing-dependent reinforcement learning to control a virtual musculoskeletal human arm, with realistic anatomical and biomechanical properties, to reach a target. Virtual arm position was used to simultaneously control a robot arm via a network interface.
1 . Dura-Bernal S, Zhou X, Neymotin SA, Przekwas A, Francis JT, Lytton WW (2015) Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm. Front Neurorobot 9:13 [PubMed]
2 . Dura-Bernal S, Li K, Neymotin SA, Francis JT, Principe JC, Lytton WW (2016) Restoring Behavior via Inverse Neurocontroller in a Lesioned Cortical Spiking Model Driving a Virtual Arm. Front Neurosci 10:28 [PubMed]
Model Information (Click on a link to find other models with that property)
Model Type: Realistic Network;
Brain Region(s)/Organism:
Cell Type(s): Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 interneuron basket PV GABA cell; Neocortex fast spiking (FS) interneuron; Neostriatum fast spiking interneuron; Neocortex spiking regular (RS) neuron; Neocortex spiking low threshold (LTS) neuron;
Gap Junctions:
Receptor(s): GabaA; AMPA; NMDA;
Transmitter(s): Gaba; Glutamate;
Simulation Environment: NEURON; Python (web link to model);
Model Concept(s): Synaptic Plasticity; Learning; Reinforcement Learning; STDP; Reward-modulated STDP; Sensory processing; Motor control; Touch;
Implementer(s): Neymotin, Sam [Samuel.Neymotin at]; Dura, Salvador [ salvadordura at];
Search NeuronDB for information about:  Neocortex M1 L2/6 pyramidal intratelencephalic GLU cell; Neocortex M1 L5B pyramidal pyramidal tract GLU cell; Neocortex M1 interneuron basket PV GABA cell; GabaA; AMPA; NMDA; Gaba; Glutamate;
#include "CoordinateOutputEventHandler.h"

#include <Multibody/MultibodyDyna.h> 

#include <sys/socket.h>
#include <netinet/in.h>
#include <stdio.h>
#include <fcntl.h>      /* To change socket to nonblocking mode */
#include <arpa/inet.h>  /* For inet_pton() */


namespace mf_mbd
	bool verboseSend = 0;
	//double shoulderAngInput;
	//double elbowAngInput;
	InitFactoryStaticMembersMacro(CoordinateOutputEventHandler, PeriodicEventHandler);

	//class MultibodySystem;

	CoordinateOutputEventHandler::CoordinateOutputEventHandler(MultibodySystem& system)

	CoordinateOutputEventHandler::CoordinateOutputEventHandler(MultibodySystem& system, Real interval)

		if(_out.is_open()) _out.close();

	bool CoordinateOutputEventHandler::handle(Real currTime)
			return false;

		//std::cout << currTime << " ";

		for(int n = 0; n < _coords.size(); ++n) {
			Coordinate& cd = *_coords[n];
			std::cout << " " <<  cd.getQ();// << "\t" << cd.getQd() << "\t" << cd.getQdd(); //velocity and acceleration of coordinate
		std::cout << std::endl;
		// send packets
		if (verboseSend) {
			printf("\nSent joint angles to stdout\n");
		return true;
	void CoordinateOutputEventHandler::initBeforeRun()

		if(!_out.is_open()) {;
			if(!_out.is_open()) CL_ERR("Can not open file " + _outFileName);


	void CoordinateOutputEventHandler::readFromXML(DOMNode* node)
		XMLDOM::DOMElement* tmpNode = NULL;
		tmpNode = XMLDOM::getFirstChildElementByTagName(node,"Interval");
		double interval = XMLDOM::getValueAsType<double>(tmpNode);
		tmpNode = XMLDOM::getFirstChildElementByTagName(node,"PntOutput");
		if(tmpNode) {
			_outFileName = XMLDOM::getAttribute(tmpNode,"name");

		tmpNode = XMLDOM::getFirstChildElementByTagName(node,"Body");
		CHK_ERR(tmpNode, "Can not find Body node");
		std::string bodyname = XMLDOM::getAttribute(tmpNode, "name");

		if(bodyname.empty()) {
			CHK_ERR(tmpNode, "Can not find Body name");

		Body* body = getMultibodySystem()->getMattSubsystem()->getBody(bodyname);
        if(!body) {
                    CL_ERR("Can not find Body with given name " + bodyname);
        _artBody = dynamic_cast<ReducedCoordArtBody*>(body);
        if(!_artBody) {
                    CL_ERR("The found body " + bodyname + " is not a reduced coordiante articulated body ");

		bool needAll = false;
		tmpNode = XMLDOM::getFirstChildElementByTagName(node,"CoordinateNames");
		CHK_ERR(tmpNode, "Can not find CoordinateNames node");
		std::string all = XMLDOM::getAttribute(tmpNode,"all");
		if(!all.empty()) {
			if(all == "true") {
				needAll = true;

		if(needAll) { //read all muscle names
			_coords = _artBody->getCoords();
		else {
			std::vector<Coordinate::Ref>& cods = _artBody->getCoords();
			std::string str = XMLDOM::getTextAsStdStringAndTrim(tmpNode);

			mf_utils::Tokenizer tokens(str);
			for(int i = 0; i < tokens.size(); ++i) {
				std::string name = tokens[i];

				for(int n = 0; n < cods.size(); ++n) {
					if(cods[n]->getName() == name) {


		tmpNode = XMLDOM::getFirstChildElementByTagName(node,"PntOutput");
		if(tmpNode) {
			_outFileName = XMLDOM::getAttribute(tmpNode,"name");

		if(_outFileName.empty()) {
			_outFileName = _artBody->getName() + "_coordates_status.pnt";
		if(!_out.is_open()) CL_ERR("Can not open file " + _outFileName);


} //end namespace 

Loading data, please wait...