Cortex learning models (Weber at al. 2006, Weber and Triesch, 2006, Weber and Wermter 2006/7)

 Download zip file 
Help downloading and running models
A simulator and the configuration files for three publications are provided. First, "A hybrid generative and predictive model of the motor cortex" (Weber at al. 2006) which uses reinforcement learning to set up a toy action scheme, then uses unsupervised learning to "copy" the learnt action, and an attractor network to predict the hidden code of the unsupervised network. Second, "A Self-Organizing Map of Sigma-Pi Units" (Weber and Wermter 2006/7) learns frame of reference transformations on population codes in an unsupervised manner. Third, "A possible representation of reward in the learning of saccades" (Weber and Triesch, 2006) implements saccade learning with two possible learning schemes for horizontal and vertical saccades, respectively.
1 . Weber C, Wermter S, Elshaw M (2006) A hybrid generative and predictive model of the motor cortex. Neural Netw 19:339-53 [PubMed]
2 . Weber C, Triesch J (2006) A possible representation of reward in the learning of saccades Proc. of the Sixth International Workshop on Epigenetic Robots :153-60
3 . Weber C, Wermter S (2006) A self-organizing map of sigma-pi units Neurocomputing 70(13-15):2552-2560
Model Information (Click on a link to find other models with that property)
Model Type: Connectionist Network;
Brain Region(s)/Organism:
Cell Type(s):
Gap Junctions:
Simulation Environment: C or C++ program;
Model Concept(s): Rate-coding model neurons; Reinforcement Learning; Unsupervised Learning; Attractor Neural Network; Winner-take-all; Hebbian plasticity;
Implementer(s): Weber, Cornelius [cweber at]; Elshaw, Mark [mark.elshaw at];

The simulator has been tested on a couple of UNIX/Linux systems
and on Windows with cygwin. It requires something equivalent to
g++ for compilation,
lex and yacc to generate the parser of the language file and
Tcl/Tk for the result viewer look.tcl.

Get the file
Then unzip, which creates the subdirectory coco06/ with the simulator in it.
Type ./make in coco06/, which creates the executable cococo
(and also prae.exe as a pre-processor).

Program call

Start the program in coco04/ with the following call:

./cococo -file v/mylanguagefile

Eg. mylanguagefile = HopfieldContinuous. Make sure a directory for the
activations and weights exists (in the examples, this is: /tmp/coco/).
This will start the program with it reading the language file v/mylanguagefile.
The program will allocate memory for the network you defined,
it will run the activation and training algorithm over and over again,
as you have described in your language file.

Start the ``GUI'', an image file viewer, like:

./look.tcl a w 0 1

Make sure to start this after the first weight- and activation-files have been
written out.
The arguments "a" means display activation files, "w" means display weights.
The numbers specify the areas from which to collect these files.
In the window, click the left mouse button or Return to reload the files,
right mouse button to quit.

Further reading

See the file which is the same as README.pdf or docu/handbook.pdf


With this neural net simulator, you edit your own network and training
algorithm using a built-in ``batch''-language.
In this language, you specify every detail of your algorithm.
You can extend the language by adding and compiling to the simulator
your own functions written in C.
A variety of networks with connectionist neurons can be programmed,
in particular if neuronal updates are local.

This simulator was developed to bridge the gap between pure C-code that
becomes messy over time and simulators (such as SNNS) which restrict the user
too much. It is meant as a scheme to organize new C-code that piles up during
development. New versions are not compatible with older ones, sorry.

The need for it arose from testing new neural network learning algorithms with
different architectures, in order to explain cortico-cortical connections
(hence the name). See the publication: Emergence of modularity within one
sheet of intrinsically active stochastic neurons. C. Weber and K. Obermayer.
Proc. ICONIP, 732-737 (2000).


Currently, the following files are supplied, addressing the following papers:

- v/HopfieldContinuous

  This addresses no paper in particular, but such a network is used in:
  "Self-Organization of Orientation Maps, Lateral Connections, and Dynamic
   Receptive Fields in the Primary Visual Cortex"
   C. Weber, In: Proc. ICANN, 1147-52, (2001).

  Such a network also constitutes the lateral, predictive weights in
  the following:

- v/CortexDocking

  The first produces, the latter analyze, the toy example weights of:
  "A hybrid generative and predictive model of the motor cortex"
   Cornelius Weber, Stefan Wermter and Mark Elshaw
   Neural Networks, 19 (4), 339-353 (2006).

- v/SigmaPi_1dim

   This creates Fig.6a) of:
   "A Self-Organizing Map of Sigma-Pi Units",
    C. Weber and S. Wermter, Neurocomputing (2006/7, in press).

- v/Vert_Horiz_Saccades

   This creates Fig.5 of:
   "A possible representation of reward in the learning of saccades"
   C. Weber and J. Triesch, Proc. Epigenetic Robotics, pp. 153-60 (2006).


Dr. Cornelius Weber, Room 0.318, Frankfurt Institute for Advanced Studies,
Johann Wolfgang Goethe University, Max-von-Laue Str. 1, 60438 Frankfurt am
Main, Germany. Tel: +49 69 798 47536, Fax: +49 69 798 47611.

Weber C, Wermter S, Elshaw M (2006) A hybrid generative and predictive model of the motor cortex. Neural Netw 19:339-53[PubMed]

References and models cited by this paper

References and models that cite this paper

Aboitiz F, Morales D, Montiel J (2003) The evolutionary origin of the mammalian isocortex: towards an integrated developmental and functional approach. Behav Brain Sci 26:535-52; discussion 552-85

Ackley DH, Hinton GE, Sejnowski TJ (1985) A learning algorithm for Bolzmann machines. Cognitive Sci 9:147-69

Bell AJ, Sejnowski TJ (1997) The independent components of natural scenes are edge filters. Vision Res 37:3327-38 [PubMed]

Blundell P, Hall G, Killcross S (2003) Preserved sensitivity to outcome value after lesions of the basolateral amygdala. J Neurosci 23:7702-9

Boyan J, Moore A (1995) Generalization in reinforcement learning: Safely approximating the value function Neural information processing systems, Tesauro G:Touretzky D:Lee T, ed. pp.369

Brashers-Krug T, Shadmehr R, Bizzi E (1996) Consolidation in human motor memory. Nature 382:252-5

Dayan P, Hinton GE, Neal RM, Zemel RS (1995) The Helmholtz machine. Neural Comput 7:889-904 [PubMed]

Demiris Y, Hayes G (2002) Imitation as a dual-route process featuring prediction and learning components: A biologically-plausible computational model Imitation in animals and artifacts, Kautenhaln K:Nehaniv C, ed. pp.327

Deneve S, Latham PE, Pouget A (1999) Reading population codes: a neural implementation of ideal observers. Nat Neurosci 2:740-5 [PubMed]

Deneve S, Latham PE, Pouget A (2001) Efficient computation and cue integration with noisy population codes. Nat Neurosci 4:826-31 [PubMed]

Dickinson A (1985) Actions and habits: The development of behavioural autonomy Phil Trans Royal Soc London B 308:67-78

Doya K (1999) What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? Neural Netw 12:961-974 [PubMed]

Fagg AH, Arbib MA (1998) Modeling parietal-premotor interactions in primate control of grasping. Neural Netw 11:1277-1303

Felleman DJ, Van Essen DC (2003) Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1:1-47

Foster DJ, Morris RG, Dayan P (2000) A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10(1):1-16

Fukushima K, Miyake S, Ito T (1983) Neocognitron: A neural network model for a mechanism of visual pattern recognition IEEE Trans Systems, Man, and Cybernetics 13:826-834

Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119 ( Pt 2):593-609

Gallese V, Goldman A (1998) Mirror neurons and the simulation theory of mind-reading Trends Cogn Sci 2:493-501

Hikosaka O, Nakamura K, Sakai K, Nakahara H (2002) Central mechanisms of motor skill learning. Curr Opin Neurobiol 12:217-22

Hinton G, Dayan P, To A, Neal R (1995) The Helmholtz machine through time Proceedings of the ICANN, Soulie F:Gallinari R, ed. pp.483

Hinton GE, Dayan P, Frey BJ, Neal RM (1995) The fiwake-sleepfi algorithm for unsupervised neural networks. Science 268:1158-61 [PubMed]

Jog MS, Kubota Y, Connolly CI, Hillegaart V, Graybiel AM (1999) Building neural representations of habits. Science 286:1745-9 [PubMed]

Kohler E, Keysers C, Umilta MA, Fogassi L, Gallese V, Rizzolatti G (2002) Hearing sounds, understanding actions: action representation in mirror neurons. Science 297:846-8

Kohonen T (2001) Self-Organizing Maps (3rd edition)

Mccallum A (1995) Reinforcement learning with selective perception and hidden state PhD Thesis

Moore A, Atkeson C (1995) The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces Mach Learn 21:1-36

Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Res 37:3311-25 [PubMed]

Oztop E, Arbib MA (2002) Schema design and implementation of the grasp-related mirror neuron system. Biol Cybern 87:116-40

Oztop E, Wolpert D, Kawato M (2003) Mirror neurons: Key for mental simulation? Twelfth annual computational neuroscience meeting CNS :81

Pasupathy A, Miller EK (2005) Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433:873-6 [PubMed]

Plaut DC, Kello CT (1998) The emergence of phonology from the interplay of speech comprehension and production: A distributed connectionist approach The emergence of Language, MacWhinney B, ed. pp.381

Pulvermuller F (2002) A brain perspective on language mechanisms: from discrete neuronal ensembles to serial order. Prog Neurobiol 67:85-111

Rao RP, Ballard DH (1997) Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Comput 9:721-63 [PubMed]

Ravel S, Richmond BJ (2005) Where did the time go? Nat Neurosci 8:705-7

Riesenhuber M, Poggio T (2002) Neural mechanisms of object recognition. Curr Opin Neurobiol 12:162-8 [PubMed]

Rizzolatti G, Arbib MA (1998) Language within our grasp. Trends Neurosci 21:188-94

Rizzolatti G, Luppino G (2001) The cortical motor system. Neuron 31:889-901

Robinson FR, Cohen JL, May J, Sestokas AK, Glickstein M (1984) Cerebellar targets of visual pontine cells in the cat. J Comp Neurol 223:471-82

Rolls ET (2000) The orbitofrontal cortex and reward. Cereb Cortex 10:284-94

Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation Parallel Distributed Processing, Rumelhart DE:McClelland JL, ed. pp.318

Sanes JN, Dimitrov B, Hallett M (1990) Motor learning in patients with cerebellar dysfunction. Brain 113 ( Pt 1):103-20

Sauser E, Billard A (2005) Three dimensional frames of references transformations using recurrent populations of neurons Neurocomputing 64:5-24

Shadmehr R, Holcomb HH (1997) Neural correlates of motor memory consolidation. Science 277:821-5 [PubMed]

Stringer SM, Rolls ET, Trappenberg TP, de Araujo IE (2003) Self-organizing continuous attractor networks and motor function. Neural Netw 16:161-82

Umilta MA, Kohler E, Gallese V, Fogassi L, Fadiga L, Keysers C, Rizzolatti G (2001) I know what you are doing. a neurophysiological study. Neuron 31:155-65

Uther WTB, Veloso MM (1998) Tree based discretization for continuous state space reinforcement learning Proc. Sixteenth National Conference on Artificial Intelligence (AAAI-98)

Weber C (2001) Self-organization of orientation maps, lateral connections, and dynamic receptive fields in the primary visual cortex Proceedings of the ICANN, Dorffner G:Bischof H:Hornik K, ed. pp.1147

Weber C, Wermter S (2003) Object localization using laterally connected what and where associator networks Proceedings of the ICANN :813-820

Weber C, Wermter S, Zochios A (2004) Robot docking with neural vision and reinforcement. Knowledge-based Systems 17:165-172

Wermter S, Elshaw M (2006) Learning robot actions based on self-organising language memory. Neural Netw 16:691-9

(50 refs)

Weber C, Triesch J (2006) A possible representation of reward in the learning of saccades Proc. of the Sixth International Workshop on Epigenetic Robots :153-60

References and models cited by this paper

References and models that cite this paper

Bahcall DO, Kowler E (2000) The control of saccadic adaptation: implications for the scanning of natural visual scenes. Vision Res 40:2779-96

Bandera C, Vico F, Bravo J, Harmon M, Andbaird L (1996) Residual q-learning applied to visual attention Intl Conf Mach Learn :20-27

Barnes N, Sandini G (2000) Direction control for an active docking behaviour based on the rotational component of log-polar optic flow Proc Europ Conf On Computer Vision 2:167-181

Brecht M, Singer W, Engel AK (2004) Amplitude and direction of saccadic eye movements depend on the synchronicity of collicular population activity. J Neurophysiol 92:424-32 [Journal]

Bronson G (1974) The postnatal growth of visual capacity. Child Dev 45:873-90

Bronson GW (1990) Changes in infants' visual scanning across the 2- to 14-week age period. J Exp Child Psychol 49:101-25

Canfield R, Kirkham N (2001) Infant cortical development and the prospective control of saccadic eye movements Infancy 2:197-211

Csibra G, Tucker LA, Volein A, Johnson MH (2000) Cortical development and saccade planning: the ontogeny of the spike potential. Neuroreport 11:1069-73

Cynader M, Berman N (1972) Receptive-field organization of monkey superior colliculus. J Neurophysiol 35:187-201 [Journal]

Deubel H, Schneider WX, Bridgeman B (2002) Transsaccadic memory of position and form. Prog Brain Res 140:165-80

Dominey P, Arbib M, Joseph JP (1995) A model of corticostriatal plasticity for learning oculomotor associations and sequences J Cogn Neurosci 7:311-336

Duhamel JR, Bremmer F, BenHamed S, Graf W (1997) Spatial invariance of visual receptive fields in parietal cortex neurons. Nature 389:845-8 [PubMed]

Edelman JA, Goldberg ME (2003) Saccade-related activity in the primate superior colliculus depends on the presence of local landmarks at the saccade endpoint. J Neurophysiol 90:1728-36 [Journal]

Frens MA, Van Opstal AJ (1997) Monkey superior colliculus activity during short-term saccadic adaptation. Brain Res Bull 43:473-83

Gancarz G, Grossberg S (1999) A neural model of saccadic eye movement control explains task-specific adaptation. Vision Res 39:3123-43

Gilmore RO, Johnson MH (1997) Body-centered representations for visually-guided action emerge during early infancy. Cognition 65:B1-9

Girard B, Berthoz A (2005) From brainstem to cortex: computational models of saccade generation circuitry. Prog Neurobiol 77:215-51

Groh JM (2001) Converting neural signals from place codes to rate codes. Biol Cybern 85:159-65

Heinen SJ, Skavenski AA (1992) Adaptation of saccades and fixation to bilateral foveal lesions in adult monkey. Vision Res 32:365-73

Hikosaka O, Nakamura K, Nakahara H (2006) Basal ganglia orient eyes to reward. J Neurophysiol 95:567-84 [Journal]

Hopp JJ, Fuchs AF (2004) The characteristics and neuronal substrate of saccadic eye movement plasticity. Prog Neurobiol 72:27-53

Itti L (2005) Quantitative modeling of perceptual salience at human eye position Visual Cognition

Itti L, Baldi P (2005) A principled approach to detecting surprising events in video Proc IEEE Conf Computer Vis Pattern Recog :631-637

Jasso H, Triesch J, Teuscher C, Deak G (2006) A reinforcement learning model explains the development of gaze following Intl Conf Cogn Model

Kojima Y, Iwamoto Y, Yoshida K (2005) Effect of saccadic amplitude adaptation on subsequent adaptation of saccades in different directions. Neurosci Res 53:404-12

Kowler E, Blaser E (1995) The accuracy and precision of saccades to small and large targets. Vision Res 35:1741-54

Lewis A, Garcia R, Zhaoping L (2003) The distribution of visual objects on the retina: connecting eye movements and cone distributions. J Vis 3:893-905

MacAskill MR, Anderson TJ, Jones RD (2002) Saccadic adaptation in neurological disorders. Prog Brain Res 140:417-31

Mercuri E, Atkinson J, Braddick O, Anker S, Cowan F, Rutherford M, Pennock J, Dubowitz L (1997) Basal ganglia damage and impaired visual function in the newborn infant. Arch Dis Child Fetal Neonatal Ed 77:F111-4

Metta G, Panerai F, Manzotti R, Sandini G (2000) Babybot: an artificial developing robotic agent

Moschovakis AK, Kitama T, Dalezios Y, Petit J, Brandi AM, Grantyn AA (1998) An anatomical substrate for the spatiotemporal transformation. J Neurosci 18:10219-29

Nakamura K, Colby CL (2002) Updating of the visual representation in monkey striate and extrastriate cortex during saccades. Proc Natl Acad Sci U S A 99:4026-31

Noto CT, Robinson FR (2001) Visual error is the stimulus for saccade gain adaptation. Brain Res Cogn Brain Res 12:301-5

Orabona F, Metta G, Sandini G (2005) Object-based visual attention: a model for a behaving robot 3rd International Workshop on Attention and Performance in Computational Vision

Oudeyer PFK, Hafner V, Whyte A (2005) The playground experiment: Task-independent development of a curious robot AAI Spring Symposium Workshop on Developmental Robotics

Rao R, Ballard D (1994) Learning saccadic eye movements using multiscale spatial lters Proceedings Of NIPS 7, Tesauro G:Touretzky D:Leen T, ed. pp.893

Robinson FR, Noto CT, Bevans SE (2003) Effect of visual error size on saccade adaptation in monkey. J Neurophysiol 90:1235-44 [Journal]

Rodemann T, Joublin F, Korner E (2004) Saccade adaptation on a 2 d of camera head Self Organization and Adaptive Behaviour :94-103

Rottach KG, Riley DE, DiScenna AO, Zivotofsky AZ, Leigh RJ (1996) Dynamic properties of horizontal and vertical eye movements in parkinsonian syndromes. Ann Neurol 39:368-77

Schwartz EL (1977) Spatial mapping in the primate sensory projection: analytic structure and relevance to perception. Biol Cybern 25:181-94

Scudder CA, Batourina EY, Tunder GS (1998) Comparison of two methods of producing adaptation of saccade size and implications for the site of plasticity. J Neurophysiol 79:704-15 [Journal]

Shuler MG, Bear MF (2006) Reward timing in the primary visual cortex. Science 311:1606-9

Slater A (2001) Visual Perception Blackwell Handbook of Infant Development :5-34

Sparks DL (2002) The brainstem control of saccadic eye movements. Nat Rev Neurosci 3:952-64

Squire L, Roberts J, Spitzer N, Zigmond MSK, Bloom F (2003) Fundamental Neuroscience (2nd ed), Squire L:Roberts J:Spitzer N:Zigmond MSK:Bloom F, ed.

Stanford TR, Freedman EG, Sparks DL (1996) Site and parameters of microstimulation: evidence for independent effects on the properties of saccades evoked from the primate superior colliculus. J Neurophysiol 76:3360-81 [Journal]

Vessel EA, Biederman I, Cohen MS (2003) How opiate activity may determine spontaneous visual selection J Vision 3:6

Walker MF, Fitzgibbon EJ, Goldberg ME (1995) Neurons in the monkey superior colliculus predict the visual result of impending saccadic eye movements. J Neurophysiol 73:1988-2003 [Journal]

Wallace MT, McHaffie JG, Stein BE (1997) Visual response properties and visuotopic representation in the newborn monkey superior colliculus. J Neurophysiol 78:2732-41 [Journal]

Watanabe S, Ogino S, Nakamura T, Koizuka I (2003) Saccadic adaptation in the horizontal and vertical directions in normal subjects. Auris Nasus Larynx 30 Suppl:S41-5

Weber C, Karantzis K, Wermter S (2005) Grasping with exible viewing-direction with a learned coordinate transformation network Proc Humanoids

Wegener D, Freiwald WA, Kreiter AK (2004) The influence of sustained selective attention on stimulus selectivity in macaque visual area MT. J Neurosci 24:6106-14

(52 refs)

Weber C, Wermter S (2006) A self-organizing map of sigma-pi units Neurocomputing 70(13-15):2552-2560

References and models cited by this paper

References and models that cite this paper

Andersen CH, van_Essen DC, Olshausen B (2004) Directed visual attention and the dynamic control of information flow Encyclopedia of Visual Attention, Itti L:Rees G:Tsotsos J, ed.

Bishop C (1995) Neural Networks For Pattern Recognition

Buneo CA, Jarvis MR, Batista AP, Andersen RA (2002) Direct visuomotor transformations for reaching. Nature 416:632-6 [PubMed]

Cohen YE, Andersen RA (2002) A common reference frame for movement plans in the posterior parietal cortex. Nat Rev Neurosci 3:553-62

Crawford JD, Medendorp WP, Marotta JJ (2004) Spatial transformations for eye-hand coordination. J Neurophysiol 92:10-9 [Journal] [PubMed]

Deneve S, Latham PE, Pouget A (2001) Efficient computation and cue integration with noisy population codes. Nat Neurosci 4:826-31 [PubMed]

Duhamel JR, Bremmer F, BenHamed S, Graf W (1997) Spatial invariance of visual receptive fields in parietal cortex neurons. Nature 389:845-8 [PubMed]

Grimes DB, Rao RP (2005) Bilinear sparse coding for invariant vision. Neural Comput 17:47-73 [PubMed]

Kohonen T (2001) Self-Organizing Maps (3rd edition)

Luttrell SP, Bayesian A (1994) A Bayesian analysis of self-organizing maps Neural Comput 6:767-794

Mel BW, Koch C (1990) Sigma-pi learning: on radial basis functions and cortical associative learning Advances In Neural Information Processing Systems, Touretzsky DS, ed. pp.474

Sauser E, Billard A (2005) Three dimensional frames of references transformations using recurrent populations of neurons Neurocomputing 64:5-24

Schluppeck D, Glimcher P, Heeger DJ (2005) Topographic organization for delayed saccades in human posterior parietal cortex. J Neurophysiol 94:1372-84 [Journal] [PubMed]

Snyder LH (2000) Coordinate transformations for eye and arm movements in the brain. Curr Opin Neurobiol 10:747-54

Spratling MW, Hayes GM (2000) Learning synaptic clusters for nonlinear dendritic processing Neural Proc Letters 11:17-27

van_Rossum MCW, Renart A (2004) Computation with population codes in layered networks of integrate-and-fire neurons Neurocomputing 58:265-270

Weber C, Muse D, Elshaw W (2006) A camera-direction dependent visual-motor coordinate transformation for a visually guided neural robot Knowledge-Based Systems (in press)

Weber C, Wermter S, Zochios A (2004) Robot docking with neural vision and reinforcement. Knowledge-based Systems 17:165-172

(18 refs)