Multisensory Control of Movement

MUCOM II - 6615

Work Area: Robotics (Sensing and Control)

Keywords sensor fusion, sensorimotor coordination, movement perception, posture control, manipulators, 3-d movement control

Start Date: 1 July 92 / Duration: 36 months / Status: running

[ participants / contact ]

Abstract Goals to be addressed concern the problems of how the brain builds an internal representation of space and movement in order to allow action, orientation, and navigation, and the relation of these mechanisms with robotics of manipulators and mobile robots. The work builds on the results of MUCOM (3149).


MUCOM aims to answer the following questions: (i) What are the reference frames in the control of manipulators and hand arm coordination? (ii) What are the mechanisms of the reduction of degrees of freedom and motor coordination in arm movement control? (iii) What are the spatio-temporal transformations involved in gaze control during orienting movements? (iv) What is the contribution of optic flow in self-motion and object-motion perception? (v) What is the contribution of inertial signals to self-motion perception, and what are the mechanisms for fusion of sensors and object motion detection during navigation?

Approach and Methods

Progress and Results

Acting: Mechanisms of arm movement control have been studied. It was shown that the brain uses a flexible egocentric frame of reference. A formal model of 3-D computation of arm reaching movements was developed using populations of neurons coding movement in body-centred reference frames combined with others using gaze-centred reference frames. Same principles used by the brain to reduce the number of degrees of freedom to be controlled have been discovered which may be useful for robot arm control. Finally, results have been obtained concerning multisensory coordination of arm movement in 3-D space: sensory-motor transformations necessary to establish a congruence between target visual information and kinesthetic information on hand position use an intermediate hybrid frame.

Orienting: The mechanisms of oriented eye-head movements have been studied at neuronal and behavioral level. A mechanism for slow correction of saccades has been described. Its neuronal mechanisms have been discovered by revealing the involvement of tecto-reticulo-spinal neurons. The relation between visual and acoustic specification of target contribution has been evaluated and a gaze position feedback discovered in the acoustic system. Superior colliculus neurons have also been shown to control both eye and hand movements during reaching. Models of the colliculus have been compared. These models stimulate dynamic feedback, dynamic memory, and multisensory information.

Navigating and fusion of sensors : The postural sway induced by moving visual patterns was studied and the importance of coupling mechanisms between vision and the active postural system was revealed. The threshold of motion detection are lowered during active movement of the observer which has implication in robot vision. Various aspects of the contribution of inertial cues to navigation have been studied combining modelling and experiments in humans and animals. Finally it has been shown that extraretinal signals contribute to depth measurement and the 3-D aspects of vestibular control of eye movements have been studied.


The work is expected to lead to new basic concepts and results concerning the algorithms and computational mechanisms of the brain; new and improved methods for the study of sensorimotor coordination and fusion of sensors; possible new methods for diagnostic tools usable in biomedical exploration; new concepts for the control of mobile robots and new methods and theories of brain studies derived from robotics: and principles of sensor fusion, which could be useful in telemanipulation or telepresence applications.

Latest Publications


Laboratoire de Physiologie neurosensorielle
Rue de l'Ecole de Médecine 15
F - 75006 PARIS


Université Catholique de Louvain - B
Universitätsspital Zürich - CH
Université de Genève - CH
Ruhr-Universität Bochum - D
Université de Montpellier - F
Université Pierre et Marie Curie - F
Consiglio Nazionale delle Ricerche - I
Katholieke Universiteit van Nijmegen - NL

Associate partner

Università di Roma - I


Dr. A. Berthoz
tel +33/1-43296154
fax +33/1-43541653

LTR synopses home page LTR work area index LTR acronym index LTR number index LTR Projects index
All synopses home page all acronyms index all numbers index

MUCOM II - 6615, August 1994

please address enquiries to the ESPRIT Information Desk

html version of synopsis by Nick Cook