Language identification of documents and queries

author: Nikolai Grigoriev, Yandex
published: Oct. 8, 2013,   recorded: August 2012,   views: 2341

Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


Language identification is a relatively simple and well-solved task. In the talk, I will give an overview of existing standard techniques, and discuss their application to two text types: crawled Web documents and user search queries. Both present specific challenges: - for Web documents - multilinguality, genre variability; - for queries - they are just too short for reliable attribution: hence the need for extra data (user context) to resolve potential ambiguity. I will talk about Yandex endeavours to cope with all that.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: