Information measures and geometry of the hyperbolic exponential families of Poincaré and hyperboloid distributions

Author

Nielsen, Frank and Okamura, Kazuki

Abstract

Hyperbolic geometry has become popular in machine learning due to its capacity to embed hierarchical graph structures with low distortions for further downstream processing. It has thus become important to consider statistical models and inference methods for data sets grounded in hyperbolic spaces. In this paper, we study various information-theoretic measures and the information geometry of the Poincaré distributions and the related hyperboloid distributions, and prove that their statistical mixture models are universal density estimators of smooth densities in hyperbolic spaces. The Poincaré and the hyperboloid distributions are two types of hyperbolic probability distributions defined using different models of hyperbolic geometry. Namely, the Poincaré distributions form a triparametric bivariate exponential family whose sample space is the hyperbolic Poincaré upper-half plane and natural parameter space is the open 3D convex cone of two-by-two positive-definite matrices. The family of hyperboloid distributions form another exponential family which has sample space the forward sheet of the two-sheeted unit hyperboloid modeling hyperbolic geometry. In the first part, we prove that all Ali–Silvey–Csiszár’s f-divergences between Poincaré distributions can be expressed using three canonical terms using the framework of maximal group invariance. We also show that the f-divergences between any two Poincaré distributions are asymmetric except when those distributions belong to a same leaf of a particular foliation of the parameter space. We report a closed-form formula for the Fisher information matrix, the Shannon’s differential entropy and the Kullback–Leibler divergence between such distributions using the framework of exponential families. In the second part, we state the corresponding results for the exponential family of hyperboloid distributions by highlighting a parameter correspondence between the Poincaré and the hyperboloid distributions. Finally, we describe a random generator to draw variates and present two Monte Carlo methods to estimate numerically f-divergences between hyperbolic distributions.