Reconstruction of Reality from Image Sequences

REALISE - 8878

Work Area: Foundations of Visualisation and Multi-Modal Interfaces

Keywords computer vision, reconstructions of scenes, reflectance analysis, virtual reality, simulation

Start Date: to be announced / Status: starting

Abstract The goal of this project is to extract from sequences of images acquired with a slowly moving camera the information necessary for determining the 3D structure of a scene and its photometric aspects. This information will be transformed according to the requirements of the database needed for virtual reality systems, then enabling the original scene to be simulated. Research will be done in the area of computer vision (structure from motion); photometry (estimation of Bidirectional Reflection Distribution Function, 3D texture) and computer graphics (adaptation of databases: noise elimination, data structuring).

Aims

Virtual reality systems offer exciting possibilities. We propose to help construct virtual reality databases automatically from sequences of images taken by moving camera(s). Such sequences contain, a priori, the necessary information for being able to determine the 3D structure of a scene and its photometric aspects.

Approach and Methods

The principal objective is to define and develop algorithms for extracting data from sequences of images acquired with a slowly moving camera. We hope to define a sequence of operations such that the construction of the database from the sequences of images is as automatic as possible.

Existing computer vision research shows that it is possible to extract the 3D structure of a scene which is observed by a stereovision setup or by a camera following an unknown path. However, the 3D structure and the motion are note very precisely estimated and the consortium will explore new ideas to improve these results. It will also address the internal parameters of the camera changing during a sequence (zoom for example).
To compute a realistic view of a scene, we must not only know the structure of the scene but also we must determine how the surfaces reflect light. The BRDF (Bidirectional Reflection Distribution Function) completely characterises surface properties at a given level of resolution. It is the most natural description of local surface properties. We will study how to estimate it from the sequences of images and from the estimated structure of the scene. We will also work on 3D texture as a means to characterise the reflectance of materials at any scale.
The information which could be obtained by analysing the sequences will probably be very different from the ones which are obtained usually by constructing databases from maps by an operator. We will explore how the data structures can be adapted to the requirements of virtual reality systems.

Potential

Our first result will be to provide new interesting insights in the three research areas that we address: structure from motion in computer vision, understanding of the reflectance properties of materials and synthesis of images from noisy and unstructured data. We aim to setup the following exciting demonstration: select an old film which includes sequences of images of a building which no longer exists and then create a database for it. Other possibilities include the construction of databases necessary for the simulation of driving cars, trains, boats, planes or for the design of furniture.

Coordinator

Thomson-CSF - F
Laboratoire Central de Recherches
Domaine de Corbeville
F- 91404 ORSAY

Partners

IGD - D
INRIA - F
Flux (University of Utrecht) - NL

CONTACT POINT

Dr. R. Vaillant
tel +33/1 60 197316
fax +33/1 60 197120

REALISE - 8878, August 1994

please address enquiries to the ESPRIT Information Desk

html version of synopsis by Nick Cook