Crossing textual and visual content in different application scenarios
author:
Gabriela Csurka,
Xerox Research Centre Europe, Xerox
Description
In this presentation, we present a method based on Trans-media Pseudo-Relevance Feedback that allows crossing visual and textual content through multimodal knowledge base. The main idea is to use one of the modalities to retrieve multimodal documents from the knowledge base and then to switch (or to fuse with) the other modality in the next step. The different potentials of the method (retrieval, image annotation, text illustration ...) are illustrated within a Travel Blog Assistance System example scenario.
You might be experiencing some problems with Your Video player.
| Slides | |
| 0:00 | Crossing textual and visual content in different application scenarios |
| 0:17 | The main idea - 1 |
| 1:05 | An example scenario - 1 |
| 2:11 | An example scenario - 2 |
| 2:53 | The main idea - 2 |
| 2:59 | Outline |
| 3:15 | - Image representation - 1 |
| 3:16 | Image similarity |
| 3:52 | Low-level features |
| 4:16 | Visual vocabulary with a GMM |
| 4:24 | The Fisher vector |
| 4:50 | Similarity between images |
| 5:11 | - Image Representation - 2 |
| 5:13 | Example of retrieved images in our TBAS |
| 5:44 | - Image Representation - 3 |
| 5:45 | Image metadata examples in TBAS using GVC |
| 6:38 | - Textual Similarity |
| 6:40 | Text Representation |
| 7:13 | - Crossing textual and visual content - 1 |
| 7:21 | Fusion between image and text |
| 8:16 | Intermediate level fusion |
| 9:14 | Pseudo feedback (PF) |
| 9:46 | - Crossing textual and visual content - 2 |
| 9:48 | Text illustration |
| 11:00 | Image annotation |
| 11:36 | Examples of auto-annotation from the repository |
| 12:02 | - Crossing textual and visual content - 3 |
| 12:07 | Information retrieval |
| 14:00 | Retrieval results of ImageClefPhoto |
| 14:51 | - Crossing textual and visual content - 4 |
| 15:01 | Relating text and image through a repository |
| 15:47 | Examples of a p es o text and images linked by the TBAS |
| 16:20 | Conclusion |
| 16:55 | Thank you |
| 17:24 | - Questions |
| 17:43 | - Questions |
| 18:12 | - Questions |
| 20:12 | - Questions |
Lecture rating
| People found this lecture: | ||
| Worth seeing | ||
| because it is: | ||
| Valuable and informative | ||
| Well presented | ||
| Easily understandable | ||
| Acceptably recorded | ||
| You need to login to cast your vote. | ||
Report a problem or upload files
If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Related content
Visitors who watched this lecture also watched...
SEE ALSO:
Link this page
Would you like to put a link to this lecture on your homepage?Go ahead! Copy the HTML snippet !



I like your work, try to find the way, you just can see what you know.