Validation Methods and Tools for Knowledge-Based Systems

VALID - 2148

Keywords knowledge-based systems, software validation

Start Date: 23-DEC-88 / Duration: 36 months

[ contact / participants ]

Objectives and Approach

VALID aimed to advance the ability to validate knowledge-based systems. The objective was to develop a tool-box based on a common representation within which the validation concepts could be defined and analysed.

The approach was to develop the following methodologies:

• the definition of a KBS quality manual, with special attention to validation actions during the different phases of KBS life cycles
• a common conceptual representation (CCR) in which most KBSs can be expressed
• the definition of a validation metalanguage (VETA) to provide the abstract mechanisms necessary to build validation tools.

These have provided the basis for several software products:

• an implementation of VETA;
• a set of representative tools for the validation of KBSs;
• interfaces to a set of existing KBSs.

Progress and Results

VALID provides the KBS project leader with:

• a quality manual that describes the related actions to be performed during the whole KBS life cycle,
• a toolbox that contains a validation metalanguage (VETA) and seven validation tools.

The quality manual: The KBS quality assurance and control consists in a set of actions to be done along all the life-cycle. The quality manual describes each action with respect to its goals, the validation tools required and the people involved in the action (knowledge engineer, developer, expert, user).

VETA: VETA is a metalanguage that features around 1000 primitives dedicated to validation, working on a common conceptual knowledge representation (CCR). This CCR allows to support any KBS that deals with: knowledge representation based on objects and rules, single-inheritance, first-order logic, rule-set, metarules, etc. The validation tools have been developed with VETA.

Validation tools: The seven validation tools cover a wide area of validation problems, ranging from purely practical aspects to validation problems that are topics of research:

• conkret refines explicit control knowledge contained in metarules. It detects deficiencies and proposes repairs.
• controller helps the experts and the knowledge engineer to express a coherent input to the KBS, a coherent output from the KBS and a coherent relation between the two. This tool allows one to test cases in a batch mode.
• curriculum kb enables the developer to examine the different relationships between the KB objects at different levels of details and to make structural verifications. It is based on a query language.
• execution visualiser allows the visualisation of the execution of the KBS through a step-by-step play-back.
• in-depth checks attribute-value rule bases for inconsistency, taking into account the uncertainty and the control features.
• penic checks propositional rule bases for inconsistency (logical or semantic) through a Petri-net transformation.
• witness manages the comments made during cases execution by experts and users. It allows, for example, the retrieval of comments linked to specific components of the KB or to a specific release of the KB.

Exploitation

VALID meets the need for the development of a uniform approach to the validation of knowledge-based systems and the provision of the associated generic methods and tools. The results from the project are used by FRAMENTEC-COGNITECH in relation with its existing KBS applications and shells. Feedback from experience allows to consolidate and regularly complete these results for example by considering more complex type of KBS including for example several technologies like artificial intelligence, operation research, databases and classical software. These extended results are also used by the company in its consulting activities.

Computer Resources International applies the results of VALID in the Aerospace industry. This is primarily done via the Esprit VIVA project where the results from VALID are carried further towards a method and toolset supporting Validation and Verification as part of the development process for knowledge-based systems.

Centre d'Estudis Avancats de Blanes makes research use of the VALID results and incorporates them into its own KBS building tool.

The AI laboratory of Universidad Politecnica de Madrid has incorporated both the theoretical results to the current courses, and practical results for improving the VALID software and also developing new tools for V&V industrial needs.

CONTACT POINT

Mr Pierre Mariot
FRAMENTEC-COGNITECH SA
Tour Fiat Cedex 16
F - 92084 PARIS LA DEFENSE
tel: + 33/ 1 47967358
fax: + 33/ 1 47964600

Participants

FRAMENTEC-COGNITECH SA - F - C
CEAB - E - P
UNIVERSIDAD POLITECNICA DE MADRID - E - P
CRI A/S - DK - P

VALID - 2148, December 1993

please address enquiries to the ESPRIT Information Desk

html version of synopsis by Nick Cook