Skip to content

Personal tools
You are here: Home » Research » Projects » Phase I » Multimodal Information: Deployment, Storage and Access

Multimodal Information: Deployment, Storage and Access

Document Actions
DS: Multimodal Information: Deployment, Storage and Access

EPFL, Database Laboratory (future IP leader): Stefano Spaccapietra,
EPFL, Peripheral Systems Laboratory (current IP leader): Roger D. Hersch,
Univ. of Lausanne, HEC-Inforge, Christine Parent

Context and Goals
IM2 has adopted the policy to use a common framework, denoted the Smart Meeting Room application (cf. IM2-AP description), to demonstrate the innovations that will be developed within its projects. The aim of the DS project is to design and implement services to support information needs of the Smart Meeting Minutes application. The targeted services include interfaces and supporting facilities to store and retrieve both the raw data produced at the meetings (namely, the video and audio records of the meetings), and the corresponding metadata produced after the meetings (namely, various annotations to describe in particular relevant segmentations of the audio and video files and, as far as possible, their semantic content). Due to the large volume of data that is expected to be produced at each meeting, issues such as performance and scalability play a significant role in the design of the information management solutions.

The research nature of the context in which the Smart Meeting Minutes application operates is expected to require higher evolvability in user requirements, to follow up with rapidly changing ideas, approaches and techniques to be demonstrated. Supporting easy and fast metadata evolution along paths that are not necessarily foreseeable in advance is a second major challenge for the DS IP that calls for investigation into innovative techniques for schema evolution.

Research Issues

  • Multimodal applications require immediate adaptability of the data management system to the unplanned evolution of their needs. Efficient schema evolution techniques are to be designed which provide a maximum flexibility for already operational databases, including major restructuring in user requirements leading to all sorts of split and merge operations and objectification. The goal is to achieve a data management technology that supports the dynamic nature of a multimodal application such as the smart meeting minutes.
  • Spatio-temporal data management techniques shall enrich data with time-based queries (essential to video data management) and space based information, e.g. where people are sitting in a meeting room (including who is next to whom) and how people are moving around during a meeting.
  • A rich conceptual data modeling approach should facilitate the user understanding of how the data structures can be used to best represent all interesting facets of the information to be stored in support of the application.
  • The efficient physical storage and retrieval of audio and video files should provide time-stamp based non sequential access to segments located in different media files in response to a single user query. We will investigate how to deploy streaming media servers in order to ensure the delivery of sequences of video and audio streams with hard real-time deadlines according to queries made to the database and to the dynamicity induced by user interaction.

Year 1: Deployment of a first version of the common database and related interfaces.
Establishment of a first conceptual schema for the smart meeting database (collaboration with CP, IIR, MDM, SA, SP). Work on the scenario of a first smart meeting elaborated with IM2.CP. Support for the acquisition of corresponding video and audio streams. Annotation of the streams (XML format) and storing them according to the proposed database schema. Library for the real-time access to selected sequences of multiple video streams. First multimodal querying capabilities.
Year 2: Adaptation of the data stores in response to the evolution of application requirements. Integration of the new annotation techniques as they become available from other work packages (e.g. IM2.IIR, SP). Consolidation of the proposals in terms of schema evolution techniques and query interfaces. Investigations into an ontology-based approach for the Smart Meeting Minutes application. Deployment of dynamic streaming services offering various degrees of quality of service.

Related projects whose results provide input knowledge and tools to IM2.DS:
Multi-representations and multiple resolutions in geographic databases:
Visible Human Web Server:


Quarterly status reports
Available on the local site (password protected).

Last modified 2006-02-03 15:53

Powered by Plone