A Multi-Modal Interface for Human-Machine Interaction with Knowledge-Based Systems


MMI2 - 2474

Keywords multi-modality, knowledge-based systems, natural language processing, graphical interfaces, dialogue management


Start Date: 01-JAN-89 / Duration: 60 months

[ contact / participants ]


Objectives and Approach

The MMI2 project undertakes research into multi-modal interfaces to knowledge-based systems (KBS). The interface allows the user to communicate with the underlying application KBS using natural language, direct manipulation and gesture, as well as a command language. It also allows the application to communicate with the user through natural language and structured graphics. Natural language modules are developed for English, French and Spanish. The English system is based on previous results and prototypes built during an ESPRIT I project (LOKI). The ergonomic aspects of the interface receive special attention in order to maximise user friendliness and efficiency and allow for use even in real-time situations.

The interface incorporates a dialogue management and mode selection system which uses knowledge of the specificities of individual modes, knowledge of the context of previous interactions, and knowledge of the application domain to determine the content of system output and to select the most appropriate mode in which to present particular information. A user modelling module interacts with dialogue management, in order to allow the system to react appropriately to different classes of users and different types of individual users.

The interface developed is designed to be portable across a range of potential applications of Prolog-based KBS. The project focuses on realistic problems and will be demonstrated and evaluated using a specific application, an expert system in computer network design. This application is also developed inside the project. A number of demonstrator prototypes of the interface integrated with the application will be produced in the course of the project.

Progress and Results

During the first phase of the project, efforts were concentrated on the design of the architecture and on the operating principles of the global system on the one hand, and on the development of individual interface models and the application KBS on the other hand. A prototype MM12 system was produced at the end of 1990. An enchained and more complete demonstrator version was delivered at the end of 1991.

During the second phase of the project, the demonstrator system was regarded as the technological basis and a frame of reference within which further research and development was carried out by individual partners according to their organisation view of potential exploitation of the MM12 technology. New integrated demonstrators are to be produced showing advanced features in, for example, the domain of French and Spanish natural language understanding. The portability of MM12 technology, and its industrial viability, is to be demonstrated by porting the interface from the network design KBS to a commercial application in the network management domain.

Exploitation

The achievements of this project have been widely disseminated by publications in scientific journals and by presentations and demonstrations at conferences and exhibitions. Some of the research results will be commercially exploited by redefining and integrating them as added value layers on more traditional, existing interface systems.


CONTACT POINT

Ms. Lieve Debille
BIM SA
Kwikstraat 4
B - 3078 EVERBERG
tel: + 32/ 2-719 2611
fax: + 32/ 2-725 4783
email: ld@sunbim.be

Participants

BIM SA - B - C
CRISS - F - P
ECOLE DES MINES DE SAINT-ETIENNE - F - P
INTELLIGENT SYSTEMS SOLUTIONS - E - P
UNIVERSITY OF LEEDS - UK - P
RUTHERFORD APPLETON LABORATORY - UK - P
INRIA - F - A


ST synopses home page ST acronym index ST number index
All synopses home page all acronyms index all numbers index

MMI2 - 2474, December 1993


please address enquiries to the ESPRIT Information Desk

html version of synopsis by Nick Cook