Penta, Antonio (2008) MULTIMEDIA KNOWLEDGE REPRESENTATION AND MANAGEMENT USING ONTOLOGIES. [Tesi di dottorato] (Unpublished)

Preview

PDF (Tesi Dottorato) Penta_Antonio.pdf Download (23MB) | Preview

Collection description

Despite the great efforts in multimedia processing in both the academic and industrial contexts, nowadays representing, organizing and managing multimedia data and the related semantics by means of a formal framework still remains a challenge. In the data and knowledge engineering community, the formal representation of domain knowledge is systematically treated by using ontologies. Especially in the semantic web field, many papers, several models and language have been proposed in order to define the concept of ontology and to design suitable and efficient systems that really use the ontological framework for real problems. In the multimedia community, a great emphasis has been given to the extensional aspects of multimedia ontology: it is easy to find ontologies about images, videos and audios that contain a number of relevant information about technical aspects related to multimedia data, its format and a variety of ways used for annotating complex and rough data. Unfortunately, the same is not true as far as the intensional aspects of multimedia ontologies are concerned. Indeed, there is still great work to be done concerning this aspect: starting from the very beginning, it is still not at all clear whether a multimedia ontology is simply a taxonomy, or a semantic network, what is the role of concrete data (if any) or whether it is a simple organization of metadata. In addition, the semantics of multimedia data itself are very hard to define and to capture: for example, in the image domain, the information carried by an image is inherently both complex and uncertain. Therefore, its semantics has a fuzzy nature. To best understand the aims of the present dissertation, let us describe a sample scenario from common real life situations. Let us consider a secret service investigation of a large scale anti-terrorristic operation. In order to carry out the investigation successfully and to avoid dramatic events, the agents use a large number of electronic devices, to conduct surveillance of places, of people involved and suspected members of the terroristic organizations. In particular, they may use the following devices in order to gather data and information: • The officer may have video cameras to record activities of suspected persons at various places. • In addition, the officer may have legally (hopefully) authorized telephone wiretaps, collecting audio involving conversations that the suspects have participated in. • The agent may also have a number of photographs taken, containing faces of suspected people and or a number of illegal activities. • The officer may have a great number of textual documents, containing a description of all the previous investigations, judge-court sentences about such people, and so on. • Eventually, a relational data base may contain several structured information, i.e. bank account transactions, credit card and so on. In all the previous events, it is clear that the core aspect of the investigation is the idea of multimedia document, containing a variety of formats (textual, pictorial, video, audio) and a variety of metadata, sometime manually added to the multimedia sources and sometimes automatically extracted. Note that in real cases, the number of multimedia data is very huge and the only way to process such a large number of data is to use automatic tools that can extract information and represent them in a suitable way. In this framework, it is mandatory to provide a novel methodology for storing and accessing multimedia data, taking into consideration both the variety of data sources and the associated uncertainty of automatic analysis and recognition systems. 3 In this dissertation we will describe a novel formal framework for multimedia ontologies, and in particular for an image and text data. I propose a framework based on a constructivist vision of multimedia data processing and representation. In other words, I provide a suitable knowledge base that can be used for storing and managing different levels of multimedia data in terms of rough data, intermediate and high level concepts, as wall as some abstractions that can be observed over them. In this way, I provide a comprehensive framework that can be used for a variety of purposes: e.g. information storing and management, information extraction, information retrieval and automatic annotations. In order to make our theory understandable, we will concentrate in particular on image data and text data. In other words, I will try to answer the following questions: do we really need yet another knowledge framework for images? What is a multimedia ontology? Can this kind of ontology be suitable for representing both intensional and extensional aspects of multimedia data? What kinds of advantages do we get from image annotation? Starting from this considerations, I will provide details concerning: i) how I represent and manage the multimedia information; ii) how I derive high level concepts, considering the discovered features, objects and elementary concepts.

Downloads

Actions (login required)

View Item