Extracting Key Terms From Noisy and Multitheme Documents

author: Maria Grineva, Institute for System Programming of the Russian Academy of Sciences
author: Maxim Grinev, Institute for System Programming of the Russian Academy of Sciences
author: Dmitry Lizorkin, Institute for System Programming of the Russian Academy of Sciences
published: May 20, 2009,   recorded: April 2009,   views: 5075
Categories

Slides

Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.
  Bibliography

Description

We present a novel method for key term extraction from text documents. In our method, document is modeled as a graph of semantic relationships between terms of that document. We exploit the following remarkable feature of the graph: the terms related to the main topics of the document tend to bunch up into densely interconnected subgraphs or communities, while non- important terms fall into weakly intercon-nected communities, or even become isolated vertices. We apply graph community detection techniques to partition the graph into thematically cohesive groups of terms. We introduce a criterion function to select groups that contain key terms discarding groups with unimportant terms. To weight terms and determine semantic relatedness between them we exploit information extracted from Wikipedia. Using such an approach gives us the following two advantages. First, it allows effectively processing multi-theme documents. Second, it is good at filtering out noise information in the document, such as, for example, navigational bars or headers in web pages. Evaluations of the method show that it outperforms existing methods producing key terms with higher precision and recall. Additional experiments on web pages prove that our method appears to be substantially more e ective on noisy and multi- theme documents than existing methods.

See Also:

Download slides icon Download slides: www09_grineva_ektnmd_01.ppt (6.7┬áMB)


Help icon Streaming Video Help

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Reviews and comments:

Comment1 Maria Grineva, May 27, 2009 at 9:07 p.m.:

I have described the idea of this paper in a blog post :
http://mint.typepad.com/blog/2009/05/...

Write your own review or comment:

make sure you have javascript enabled or clear this field: