What is a residual? A residual of an observed value is the difference between the observed value and the estimated function value. This is distinguished from the error, which is the difference between the observed value and the true (unobservable) function value. This is not particularly enlightening. An example would be if we have a population, apparently the mean of the population cannot be known (unless we take the whole population as a sample?) but the mean of a sample of that population can be known. Therefore the mean of the sample is an estimate of the mean of the population. Now the difference between an error and a residual is error means how far a sample is from the population mean and a residual is how far a sample is from the sample mean. Also, the sum of the residuals must be zero within a random sample, I think this implies that if the sum of the residuals is not zero then the sample was not random. That I can stomach. This actually does make sense when you think about taking a repeated measurement. You are only ever taking a sample, you can never know the whole 'population'. The mean of your samples is what you will take as your measurement, and it will most likely be very close to the 'true' values, but that true value can never be known oooooo~~~~~~ What is a uni-variate distribution? A uni-variate distribution is a probability distribution of only one random variable. This is in contrast to a multivariate distribution, the probability distribution of a random vector. So the normal distribution curve that you are used to seeing is an example of a uni-variate distribution. A multivariate distribution is a generalization to higher dimensions, like how a scalar could be thought of as a special case of a vector. A normal distribution in 2-D looks something like this: What is a statistical model? A statistical model is a formalization of relationships beteen variables in the form of mathematical equations. It describes how one or more random variables are related to one or more other variables. The model is statistical as the variables are not deterministically but stochastically related. An example would be nice here. (A stochastic system is one whose subsequent state cannot be determined deterministically. Stochastic means aim. A good image to have in mind is a pattern of arrows around a target stick. Factors such as weather, and kill of the archer to name a few render this process non-deterministic. Perhaps philosophically it is deterministic.) Height and age make for a very good example. They are stochastically related; when you know a person is of age 7, this influences their chances of being 6 feet tall, non-deterministically i.e. age is not enough to say for certain that that person can not be 6 feet tall. We could (and will) formalize this relationship in a linear regression model.
