ART

In probability theory and statistics, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and precision matrix (the inverse of the covariance matrix).[1]

Definition

Suppose

\( {\displaystyle {\boldsymbol {\mu }}|{\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Lambda }}\sim {\mathcal {N}}({\boldsymbol {\mu }}_{0},(\lambda {\boldsymbol {\Lambda }})^{-1})} \)

has a multivariate normal distribution with mean \( {\boldsymbol {\mu }}_{0} \) and covariance matrix \( (\lambda {\boldsymbol \Lambda })^{{-1}}, \) where

\( {\boldsymbol \Lambda }|{\mathbf {W}},\nu \sim {\mathcal {W}}({\boldsymbol \Lambda }|{\mathbf {W}},\nu ) \)

has a Wishart distribution. Then \( ({\boldsymbol \mu },{\boldsymbol \Lambda }) \) has a normal-Wishart distribution, denoted as

\( ({\boldsymbol \mu },{\boldsymbol \Lambda })\sim {\mathrm {NW}}({\boldsymbol \mu }_{0},\lambda ,{\mathbf {W}},\nu ). \)

Characterization
Probability density function

\( f({\boldsymbol \mu },{\boldsymbol \Lambda }|{\boldsymbol \mu }_{0},\lambda ,{\mathbf {W}},\nu )={\mathcal {N}}({\boldsymbol \mu }|{\boldsymbol \mu }_{0},(\lambda {\boldsymbol \Lambda })^{{-1}})\ {\mathcal {W}}({\boldsymbol \Lambda }|{\mathbf {W}},\nu ) \)

Properties
Scaling
Marginal distributions

By construction, the marginal distribution over \( {\boldsymbol {\Lambda }} \) is a Wishart distribution, and the conditional distribution over \( \boldsymbol\mu \) given \( {\boldsymbol {\Lambda }} \) is a multivariate normal distribution. The marginal distribution over \( \boldsymbol\mu \) is a multivariate t-distribution.
Posterior distribution of the parameters

After making n observations \( {\displaystyle {\boldsymbol {x}}_{1},\dots ,{\boldsymbol {x}}_{n}} \) , the posterior distribution of the parameters is

\( {\displaystyle ({\boldsymbol {\mu }},{\boldsymbol {\Lambda }})\sim \mathrm {NW} ({\boldsymbol {\mu }}_{n},\lambda _{n},\mathbf {W} _{n},\nu _{n}),} \)

where

\( {\displaystyle \lambda _{n}=\lambda +n,} \)
\( {\displaystyle {\boldsymbol {\mu }}_{n}={\frac {\lambda {\boldsymbol {\mu }}_{0}+n{\boldsymbol {\bar {x}}}}{\lambda +n}},} \)
\( {\displaystyle \nu _{n}=\nu +n,} \)
\( {\displaystyle \mathbf {W} _{n}^{-1}=\mathbf {W} ^{-1}+\sum _{i=1}^{n}({\boldsymbol {x}}_{i}-{\boldsymbol {\bar {x}}})({\boldsymbol {x}}_{i}-{\boldsymbol {\bar {x}}})^{T}+{\frac {n\lambda }{n+\lambda }}({\boldsymbol {\bar {x}}}-{\boldsymbol {\mu }}_{0})({\boldsymbol {\bar {x}}}-{\boldsymbol {\mu }}_{0})^{T}.} \) [2]

Generating normal-Wishart random variates

Generation of random variates is straightforward:

Sample \( {\boldsymbol {\Lambda }} \) from a Wishart distribution with parameters \( \mathbf {W} \) and \( \nu \)
Sample \( \boldsymbol\mu \) from a multivariate normal distribution with mean \( {\boldsymbol {\mu }}_{0} \) and variance \( (\lambda {\boldsymbol \Lambda })^{{-1}} \)

Related distributions

The normal-inverse Wishart distribution is essentially the same distribution parameterized by variance rather than precision.
The normal-gamma distribution is the one-dimensional equivalent.
The multivariate normal distribution and Wishart distribution are the component distributions out of which this distribution is made.

Notes

Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media. Page 690.

Cross Validated, https://stats.stackexchange.com/q/324925

References

Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media.

Undergraduate Texts in Mathematics

Graduate Texts in Mathematics

Graduate Studies in Mathematics

Mathematics Encyclopedia

World

Index

Hellenica World - Scientific Library

Retrieved from "http://en.wikipedia.org/"
All text is available under the terms of the GNU Free Documentation License