Softmax function

Efficient implementation (via Fortran) of the softmax (aka multinomial logistic) function converting a set of numerical values to probabilities summing to 1.

Usage

softmax(x, v = NULL)

Arguments

x

a matrix of dimension \(n \times k\) of numerical values. If a vector is provided, it is converted to a single-row matrix.

v

an optional vector of length \(k\) of numerical values to be added to each row of x matrix. If not provided, a vector of zeros is used.

Details

Given the matrix x, for each row \(x_{[i]} = [x_1, \dots, x_k]\) (with \(i=1,\dots,n\)), the softmax function calculates $$ \text{softmax}(x_{[i]})_j = \dfrac{\exp(x_j + v_j)}{\sum_{l=1}^k \exp(x_l + v_l)} \qquad \text{for } j = 1,\dots,k $$

Value

Returns a matrix of the same dimension as x with values in the range \((0,1)\) that sum to 1 along the rows.

Author

Luca Scrucca

See also

logsumexp

References

Blanchard P., Higham D. J., Higham N. J. (2021). Accurately computing the log-sum-exp and softmax functions. IMA Journal of Numerical Analysis, 41/4:2311–2330. doi:10.1093/imanum/draa038

Examples

x = matrix(rnorm(15), 5, 3)
v = log(c(0.5, 0.3, 0.2))
(z = softmax(x, v))
#>            [,1]      [,2]       [,3]
#> [1,] 0.72376965 0.1887619 0.08746842
#> [2,] 0.45678397 0.5202153 0.02300074
#> [3,] 0.34571712 0.2460857 0.40819713
#> [4,] 0.07731899 0.8617873 0.06089369
#> [5,] 0.72587924 0.2149419 0.05917887
rowSums(z)
#> [1] 1 1 1 1 1