Gaussian mixture-based estimation of entropy

Compute an estimate of the (differential) entropy from a Gaussian Mixture Model (GMM) fitted using the mclust package.

Usage

EntropyGMM(object, ...) 

# S3 method for densityMclust
EntropyGMM(object, ...)
# S3 method for Mclust
EntropyGMM(object, ...)
# S3 method for densityMclustBounded
EntropyGMM(object, ...)
# S3 method for matrix
EntropyGMM(object, ...)
# S3 method for data.frame
EntropyGMM(object, ...)

EntropyGauss(sigma)

nats2bits(x)
bits2nats(x)

Arguments

object: An object of class 'Mclust', 'densityMclust', or 'densityMclustBounded', obtained by fitting a Gaussian mixture via, respectively, Mclust(), densityMclust(), and densityMclustBounded(). If a 'matrix' or 'data.frame' is provided as input, a GMM using the provided data is estimated preliminary to computing the entropy. In this case further arguments can be provided to control the fitted model (e.g. number of mixture components and/or covariances decomposition).
sigma: A symmetric covariance matrix.
x: A vector of values.
...: Further arguments passed to or from other methods.

Value

EntropyGMM() returns an estimate of the entropy based on a estimated Gaussian mixture model (GMM) fitted using the mclust package. If a matrix of data values is provided, a GMM is preliminary fitted to the data and then the entropy computed.

EntropyGauss() returns the entropy for a multivariate Gaussian distribution with covariance matrix sigma.

nats2bits() and bits2nats() convert input values in nats to bits, and viceversa. Information-theoretic quantities have different units depending on the base of the logarithm used: nats are expressed in base-2 logarithms, whereas bits in natural logarithms.

Author

Luca Scrucca

References

Robin S. and Scrucca L. (2023) Mixture-based estimation of entropy. Computational Statistics & Data Analysis, 177, 107582. https://doi.org/10.1016/j.csda.2022.107582

Examples

# \donttest{
X = iris[,1:4]
mod = densityMclust(X, plot = FALSE)
h = EntropyGMM(mod)
h
#> [1] 1.438173
bits2nats(h)
#> [1] 0.9968657
EntropyGMM(X)
#> [1] 1.438173
# }