2024年12月22日 星期日 登录 EN

学术活动
Entropy estimation via normalizing flow
首页 - 学术活动
报告人:
Jinglai Li, , Professor, University of Birmingham, UK
邀请人:
Tao Zhou, Professor
题目:
Entropy estimation via normalizing flow
时间地点:
15:30-16:30 May 11(Wednesday)
摘要:

Entropy estimation is an important problem in statistical science. Many popular entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for high-dimensional problems. In this work we propose a transform-based method for high-dimensional entropy estimation, which consists of the following two main ingredients. First by modifying the k-NN based entropy estimator developed in (Kozachenko \& Leonenko, 1987), we propose a new estimator which enjoys small estimation bias for samples that are close to a uniform distribution. Second we design a normalizing flow based mapping that pushes samples toward a uniform distribution, and the relation between the entropy of the original samples and the transformed ones is also derived. As a result the entropy of a given set of samples is estimated by first transforming them toward a uniform distribution and then applying the proposed estimator to the transformed samples. Numerical experiments demonstrate the effectiveness of the method for high-dimensional entropy estimation problems.