Exponential family

In probability and statistics, the exponential family is an important class of probability distributions. This is for mathematical convenience, on account of their nice algebraic properties; as well as for generality, as they are in a sense very natural distributions to consider.

Related Topics:
Probability - Statistics - Probability distribution

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

There are both discrete and continuous members of the exponential family which are useful and important in theoretical or practical work. We use cumulative distribution functions in order to encompass both discrete and continuous distributions. A member of the exponential family has cdf

Related Topics:
Discrete - Continuous - Cumulative distribution function - Cdf

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

:dF(x|eta) = e^{-eta^{ op} T(x) - A(eta)} dH(x)

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

If F is a continuous distribution with a density, one can write dF(x) = f(x) dx. The meanings of the different symbols in the right-hand side are as follows:

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

• H(x) is a Lebesgue-Stieltjes integrator for the reference measure. When the reference measure is finite, it can be normalized and H is actually the cumulative distribution function of a probability distribution. If F is continuous with a density, then so is H, which can then be written dH(x) = h(x) dx. If F is discrete, then so is H (with the same support).
• η is the natural parameter, a column vector, so that η^T = (η₁, ..., η_n), its transpose, is a row vector. The parameter space—i.e., the set of values of η for which this function is integrable—is necessarily convex.
• T(x) is the sufficient statistic of the distribution, and it is a column vector whose number of scalar components is the same as that of η so that η^TT(x) is a scalar. (Note that the concept of sufficient statistic applies more broadly than just to members of the exponential family.)
• and A(η) is a normalization factor without which F would not be a probability distribution. The function A is important in its own right, because in cases in which the reference measure dH(x) is a probability measure, then A is is the cumulant-generating function of the probability distribution of the sufficient statistic T(X) when the distribution of X is dH(x).

The term exponential family is also frequently used to refer to any particular concrete case, i.e., any parametrized family of probability distributions of this form, determined by a choice of H and T.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

The Bernoulli, normal, gamma, Poisson and binomial distributions are all exponential families. The Weibull distributions do not comprise an exponential family, nor do the Cauchy distributions.

Related Topics:
Bernoulli - Normal - Gamma - Poisson - Binomial - Weibull distribution - Cauchy distribution

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

 
Exponential family: Maximum entropy derivation ►