REGO: Rank-based estimation of Renyi information using Euclidean graph optimization

author: Barnabás Póczos, Machine Learning Department, School of Computer Science, Carnegie Mellon University
published: June 14, 2010,   recorded: May 2010,   views: 3319


Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


We propose a new method for a non-parametric estimation of Renyi and Shannon information for a multivariate distribution using a corresponding copula, a multivariate distribution over normalized ranks of the data. As the information of the distribution is the same as the negative entropy of its copula, our method estimates this information by solving a Euclidean graph optimization problem on the empirical estimate of the distribution's copula. Owing to the properties of the copula, we show that the resulting estimator of Renyi information is strongly consistent and robust. Further, we demonstrate its applicability in the image registration in addition to simulated experiments.

See Also:

Download slides icon Download slides: aistats2010_poczos_rrbe_01.pdf (1.7 MB)

Help icon Streaming Video Help

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Reviews and comments:

Comment1 edith, June 27, 2013 at 4:04 p.m.:

At the beginning the problem was that we want to calculate the mutual information The problem is caused by the fact we do not know the joint probability density. At the end it is used a distribution with uniform marginals (discrete copula), but wich has for each vector realization probability 1/n. The entropy of such distribution is well known. So I do not see the point, why is better using discrete copula as using a sample from the joint probability distribution.

Write your own review or comment:

make sure you have javascript enabled or clear this field: