Processing math: 100%


A single-parameter exponential family is a set of probability distributions whose probability density function can be expressed in the form:

fX(x;θ)=h(x) exp[η(θ)T(x)A(θ)]


or equivalently

fX(x;θ)=h(x)g(θ) exp[η(θ)T(x)]

fX(x;θ)= exp[η(θ)T(x)A(θ)+B(x)]


The definition in terms of one real-number parameter can be extended to one real-vector parameter θ=(θ1,θ2,,θd)T. A family of distributions is said to belong to a vector exponential family if the probability density function can be written as

fX(x;θ)=h(x) exp[si=1ηi(θ)Ti(x)A(θ)]


Interpretation

  • Sufficient statistic T(x)
    Heuristic definition: We say T is a sufficient statistic if the statistician who knows the value of T can do just as good a job of estimating the unknown parameter θ as the statistician who knows the entire random sample.
    Mathematical definition: A statistic T=r(X1,X2,,Xn) is a sufficient statistic if for each t, the conditional distribution of X1,X2,,Xn given T=t and θ does not depend on θ.
  • Natural parameter η
  • Logarithm of the normalization factor A(η)
    The normalization factor is a constant by which an everywhere non-negative function must be multiplied so the area under its graph is 1, e.g., to make it a probability density function or a probability mass function.
    (http://en.wikipedia.org/wiki/Normalizing_constant)



Examples

  • Normal
  • Exponential
  • Log-normal
  • Gamma
  • Chi-squared
  • Beta
  • Dirichlet
  • Bernoulli
  • Categorical
  • Poisson
  • Geometric
  • Inverse Gaussian
  • Von Mises
  • Von Mises-Fisher



References