2024-09-19 Thursday Sign in CN

Activities
Entropy estimation via normalizing flow
Home - Activities
Reporter:
Jinglai Li, , Professor, University of Birmingham, UK
Inviter:
Tao Zhou, Professor
Subject:
Entropy estimation via normalizing flow
Time and place:
15:30-16:30 May 11(Wednesday)
Abstract:

Entropy estimation is an important problem in statistical science. Many popular entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for high-dimensional problems. In this work we propose a transform-based method for high-dimensional entropy estimation, which consists of the following two main ingredients. First by modifying the k-NN based entropy estimator developed in (Kozachenko \& Leonenko, 1987), we propose a new estimator which enjoys small estimation bias for samples that are close to a uniform distribution. Second we design a normalizing flow based mapping that pushes samples toward a uniform distribution, and the relation between the entropy of the original samples and the transformed ones is also derived. As a result the entropy of a given set of samples is estimated by first transforming them toward a uniform distribution and then applying the proposed estimator to the transformed samples. Numerical experiments demonstrate the effectiveness of the method for high-dimensional entropy estimation problems.