A single-parameter exponential family is a set of probability distributions whose probability density function can be expressed in the form:

$$f_X(x;\theta) = h(x) \text{ exp} \left [  \eta (\theta) T(x) - A(\theta)  \right ]$$


or equivalently

$$f_X(x;\theta) = h(x)g(\theta) \text{ exp} \left [  \eta (\theta) T(x)  \right ]$$

$$f_X(x;\theta) = \text{ exp} \left [  \eta (\theta) T(x) - A(\theta) + B(x)  \right ]$$


The definition in terms of one real-number parameter can be extended to one real-vector parameter $\mathbf{\theta} = (\theta_1, \theta_2, \dots, \theta_d)^T$. A family of distributions is said to belong to a vector exponential family if the probability density function can be written as

$$f_X(x;\mathbf{\theta}) = h(x) \text{ exp} \left [ \sum_{i=1}^s \eta_i(\mathbf{\theta})T_i(x) - A(\mathbf{\theta}) \right ]$$


Interpretation

  • Sufficient statistic $T(x)$
    Heuristic definition: We say $T$ is a sufficient statistic if the statistician who knows the value of $T$ can do just as good a job of estimating the unknown parameter $\theta$ as the statistician who knows the entire random sample.
    Mathematical definition: A statistic $T=r(X_1, X_2, \dots, X_n)$ is a sufficient statistic if for each $t$, the conditional distribution of $X_1, X_2, \dots, X_n$ given $T=t$ and $\theta$ does not depend on $\theta$.
  • Natural parameter $\eta$
  • Logarithm of the normalization factor $A(\eta)$
    The normalization factor is a constant by which an everywhere non-negative function must be multiplied so the area under its graph is 1, e.g., to make it a probability density function or a probability mass function.
    (http://en.wikipedia.org/wiki/Normalizing_constant)



Examples

  • Normal
  • Exponential
  • Log-normal
  • Gamma
  • Chi-squared
  • Beta
  • Dirichlet
  • Bernoulli
  • Categorical
  • Poisson
  • Geometric
  • Inverse Gaussian
  • Von Mises
  • Von Mises-Fisher



References



티스토리툴바