MULTIMEDIA INFORMATION RETRIEVAL (MMIR)

WHAT IS MMIR?

The Viper group deals with the processing and management of multimedia. In particular, we focus onmultimedia information retrieval and mining. Our current research interests include:

• Multimedia information Retrieval: Image and video (including audio)
• Multimedia information mining with our novative concept of Collection Guiding and also our efforts for large multimedia collection management
• Content-based video indexing
• Content-based image indexing
• Automated multimedia description
• Multimedia processing as a need to design efficient features.

WHY MMIR IS NEEDED NOWADAYS?

BECAUSE With the explosive growth of digital media data, there is a huge demand for new tools and systems that enables average users to more efficiently and more effectively search, access, process, manage, author and share these digital media contents.




MMIR OVERVIEW:

• CONTENT-BASED IMAGE RETRIEVAL (CBIR)

Content-based image retrieval (CBIR), also known as query by image content (QBIC) and content-based visual information retrieval (CBVIR) is the application of computer vision techniques to the image retrieval problem, that is, the problem of searching for digital images in large databases. for a recent scientific overview of the CBIR field)

"Content-based" means that the search will analyze the actual contents of the image rather than the metadata such as keywords, tags, and/or descriptions associated with the image. The term 'content' in this context might refer to colors, shapes, textures, or any other information that can be derived from the image itself. CBIR is desirable because most web based image search engines rely purely on metadata and this produces a lot of garbage in the results. Also having humans manually enter keywords for images in a large database can be inefficient, expensive and may not capture every keyword that describes the image. Thus a system that can filter images based on their content would provide better indexing and return more accurate results.

• Image Search Systems that search images by image content

            <-> Keyword-based Image Retrieval
             (ex. Google Image Search)




Applications of CBIR


�� Consumer Digital Photo Albums
�� Digital Cameras
�� Ex. WWMX by Microsoft Research
�� Medical Images
�� Digital Museum
�� Trademarks Search
�� MPEG-7 Content Descriptors




How does CBIR work ?

�� Extract Features from Images
�� Let the user do Query
�� Query by Sketch
�� Query by Keywords
�� Query by Example
�� Refine the result by Relevance Feedback
�� Give feedback to the previous result

Query techniques

• query by examples
• semantic retrrieval

• Content comparison using image distance measures

• Color
• Texture
• Shape

MANY CBIR SYSTEM HAVE BEEN BUILT

• CONTENT-BASED VIDEO RETRIEVAL (CBVR)

  
  

  
  

  Content-Based Video Retrieval: A Database Perspective concentrates on the semantic gap

  problem, i.e., the problem of inferring semantics from raw video data, as the main problem of content based video retrieval.

  
  

  
  

  
  

  
  

  

  
  
  
  The proposed architecture for a content-based video 
  

  retrieval system is as shown above . The process of 

  
  

  database population is shown with dashed lines, while 

  querying is shown with solid lines. The raw video 

  data is stored in the file system, while the storage 

  server is used to store video content meta data and 

  indexes. In the process of the database population, the 

  features, objects, and events that are specified by 

  system administrator, are extracted. Indexes and metadata are put in the storage server and videos in the file 

  system. Most queries are resolved directly in the 

  storage server, but if the query comprises something 

  that has not been already extracted the extractors do 

  that dynamically.

  
  

• CONTENT-BASED AUDIO RETRIEVAL (CBAR)

to search sounds by their features in the waveform, statistics, OR transform domains. examples such as Speech, Music, Environment Audio, Silence.

APPLICATION:

1. Entertainment

��Film making - searching sound effects

��TV/radio studio - editing programs

��Karaoke, music stores, or online shopping

- query by humming the melody

1. 
   

1. 2. Audio/video archive management

��Segmenting and indexing of raw recordings

��Searching and browsing audio/video clips

3.     Surveillance

��Monitoring criminal or emergent events

��Film rating

WHAT IS AUGMENTED REALITY

Augmented Reality is a type of VIRTUAL REALITY that aims to duplicate the world's environment in a Computer. An augmented reality system generates a composite view for the user that is the combination of the real scene viewed by the user and a virtual scene generated by the computer that augments the scene with additional information. The virtual scene generated by the computer is designed to enhance the user's sensory perception of the virtual world they are seeing or interacting with.

    Augmented reality is a new technology that combines the use of web camera tracking (using a simple web cam), Flash software technology, 3d artwork, and a symbol to create an interactive experience for the user. The graphics and video seems to “pop” off of the screen during the simulation, offering an interesting way to interact through greeting cards, games, promotions, and other creative concepts. Interestingly, the symbol used to activate the game or application can be placed on a greeting card, a piece of paper, a business card, a hat or shirt, or any piece of marketing material.

what is the GOAL ?  

The goal of Augmented Reality is to create a system in which the user cannot tell the difference between the real world and the virtual augmentation of it. Today Augmented Reality is used in entertainment,military training , engineering design, robotics, manufacturing and other industries.

HOW TO VIEW AUGMENTED REALITY ON YOUR COMPUTER TUTORIAL

CLICK THIS LINK !

some of examples of AR

The SixthSense augmented reality system lets you project a phone pad onto your hand and phone a friend -- without removing the phone from your pocket