Linear model

In statistics the linear model can be expressed by saying

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

:Y = X eta + epsilon

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

where Y is an n×1 column vector of random variables, X is an n×p matrix of "known" (i.e. observable and non-random) quantities, whose rows correspond to statistical units, β is a p×1 vector of (unobservable) parameters, and ε is an n×1 vector of "errors", which are uncorrelated random variables each with expected value 0 and variance σ2. Often one takes the components of the vector of errors to be independent and normally distributed. Having observed the values of X and Y, the statistician must estimate β and σ2. Typically the parameters β are estimated by the method of maximum likelihood, which in the case of normal errors is equivalent (by the Gauss-Markov theorem) to the method of least squares.

Related Topics:
Statistical unit - Independent - Normally distributed - Maximum likelihood - Gauss-Markov theorem - Least squares

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

If, rather than taking the variance of ε to be σ2I, where I is the n×n identity matrix, one assumes the variance is σ2M,

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

where M is a known matrix other than the identity matrix, then one estimates β by the method of "generalized least squares", in which, instead of minimizing the sum of squares of the residuals, one minimizes a different quadratic form in the residuals — the quadratic form being the one given by the matrix M-1.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

If all of the off-diagonal entries in the matrix M are 0, then one normally estimates β by the method of "weighted least squares", with weights proportional to the reciprocals of the diagonal entries.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

Ordinary Linear regression is a very closely related topic.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~