Skip to contents

Gaussian mixture-based estimation of entropy

EntropyGMM.Rd

Compute an estimate of the (differential) entropy from a Gaussian Mixture Model (GMM) fitted using the mclust package.

Usage

EntropyGMM(object, ...)

# S3 method for class 'densityMclust'
EntropyGMM(object, ...)

# S3 method for class 'densityMclustBounded'
EntropyGMM(object, ...)

# S3 method for class 'Mclust'
EntropyGMM(object, ...)

# S3 method for class 'data.frame'
EntropyGMM(object, ...)

# S3 method for class 'matrix'
EntropyGMM(object, ...)

EntropyGauss(sigma)

nats2bits(x)

bits2nats(x)

Arguments

object

An object of class 'Mclust', 'densityMclust', or 'densityMclustBounded', obtained by fitting a Gaussian mixture via, respectively, mclust::Mclust(), mclust::densityMclust(), and densityMclustBounded().

If a matrix or data.frame is provided as input, a GMM using the provided data is estimated preliminary to computing the entropy. In this case further arguments can be provided to control the fitted model (e.g. number of mixture components and/or covariances decomposition).

...

Further arguments passed to or from other methods.

sigma

A symmetric covariance matrix.

x

A vector of values.

Value

  • EntropyGMM() returns an estimate of the entropy based on a estimated Gaussian mixture model (GMM) fitted using the mclust package. If a matrix of data values is provided, a GMM is preliminary fitted to the data and then the entropy computed.

  • EntropyGauss() returns the entropy for a multivariate Gaussian distribution with covariance matrix sigma.

  • nats2bits() and bits2nats() convert input values in nats to bits, and viceversa. Information-theoretic quantities have different units depending on the base of the logarithm used: nats are expressed in base-2 logarithms, whereas bits in natural logarithms.

Details

For more details see vignette("mclustAddons")

References

Robin S. and Scrucca L. (2023) Mixture-based estimation of entropy. Computational Statistics & Data Analysis, 177, 107582. doi:10.1016/j.csda.2022.107582

See also

mclust::Mclust(), mclust::densityMclust().

Author

Luca Scrucca

Examples

# \donttest{
X = iris[,1:4]
mod = densityMclust(X, plot = FALSE)
h = EntropyGMM(mod)
h
#> [1] 1.438173
bits2nats(h)
#> [1] 0.9968657
EntropyGMM(X)
#> [1] 1.438173
# }