A single-parameter exponential family is a set of probability distributions whose probability density function can be expressed in the form:
fX(x;θ)=h(x) exp[η(θ)T(x)−A(θ)]
or equivalently
fX(x;θ)=h(x)g(θ) exp[η(θ)T(x)]
fX(x;θ)= exp[η(θ)T(x)−A(θ)+B(x)]
The definition in terms of one real-number parameter can be extended to one real-vector parameter θ=(θ1,θ2,…,θd)T. A family of distributions is said to belong to a vector exponential family if the probability density function can be written as
fX(x;θ)=h(x) exp[s∑i=1ηi(θ)Ti(x)−A(θ)]
Interpretation
- Sufficient statistic T(x)
Heuristic definition: We say T is a sufficient statistic if the statistician who knows the value of T can do just as good a job of estimating the unknown parameter θ as the statistician who knows the entire random sample.
Mathematical definition: A statistic T=r(X1,X2,…,Xn) is a sufficient statistic if for each t, the conditional distribution of X1,X2,…,Xn given T=t and θ does not depend on θ. - Natural parameter η
- Logarithm of the normalization factor A(η)
The normalization factor is a constant by which an everywhere non-negative function must be multiplied so the area under its graph is 1, e.g., to make it a probability density function or a probability mass function.
(http://en.wikipedia.org/wiki/Normalizing_constant)
Examples
- Normal
- Exponential
- Log-normal
- Gamma
- Chi-squared
- Beta
- Dirichlet
- Bernoulli
- Categorical
- Poisson
- Geometric
- Inverse Gaussian
- Von Mises
- Von Mises-Fisher
References
- http://en.wikipedia.org/wiki/Exponential_family
- http://ko.wikipedia.org/wiki/%EC%A7%80%EC%88%98%EC%A1%B1
- http://sens.tistory.com/423
- Sufficient statistic: http://blog.naver.com/rupy400/130108738040
- Sufficient statistic:
sufficient_statistic.pdf
- Sufficient statistic: http://blog.naver.com/weather12/130156410915
- http://blog.naver.com/pcapcoms/60100791155