event thumbnail image
MUSCLE Conference joint with VITALAS Conference

Crossing textual and visual content in different application scenarios

author: Gabriela Csurka, Xerox Research Centre Europe, Xerox

Description

In this presentation, we present a method based on Trans-media Pseudo-Relevance Feedback that allows crossing visual and textual content through multimodal knowledge base. The main idea is to use one of the modalities to retrieve multimodal documents from the knowledge base and then to switch (or to fuse with) the other modality in the next step. The different potentials of the method (retrieval, image annotation, text illustration ...) are illustrated within a Travel Blog Assistance System example scenario.

You might be experiencing some problems with Your Video player.
Slides
0:00 Crossing textual and visual content in different application scenarios
0:17 The main idea - 1
1:05 An example scenario - 1
2:11 An example scenario - 2
2:53 The main idea - 2
2:59 Outline
3:15 - Image representation - 1
3:16 Image similarity
3:52 Low-level features
4:16 Visual vocabulary with a GMM
4:24 The Fisher vector
4:50 Similarity between images
5:11 - Image Representation - 2
5:13 Example of retrieved images in our TBAS
5:44 - Image Representation - 3
5:45 Image metadata examples in TBAS using GVC
6:38 - Textual Similarity
6:40 Text Representation
7:13 - Crossing textual and visual content - 1
7:21 Fusion between image and text
8:16 Intermediate level fusion
9:14 Pseudo feedback (PF)
9:46 - Crossing textual and visual content - 2
9:48 Text illustration
11:00 Image annotation
11:36 Examples of auto-annotation from the repository
12:02 - Crossing textual and visual content - 3
12:07 Information retrieval
14:00 Retrieval results of ImageClefPhoto
14:51 - Crossing textual and visual content - 4
15:01 Relating text and image through a repository
15:47 Examples of a p es o text and images linked by the TBAS
16:20 Conclusion
16:55 Thank you
17:24 - Questions
17:43 - Questions
18:12 - Questions
20:12 - Questions

Lecture rating

People found this lecture:
Worth seeing
because it is:
 Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Reviews and comments:

Comment1 Forty2, February 25, 2008 at 11:12 a.m.:

I like your work, try to find the way, you just can see what you know.

Write your own review or comment:

make sure you have javascript enabled or clear this field: