#generalized linear model

A generalized additive model (GAM) is much more flexible than a general linear model  A GAM is of the form

g(E(Y)) = β0 + f1(x1) + f2(x2) +...+ fn(xn)

  Where the functions fi’s  and g functions can be parametric , nonparametric or splines.

A generalized linear model (GLM), there is a g function but no f functions. So, the model is of the form:

g(E(Y)) = β0 + f1(x1) + f2(x2) +...+ fn(xn)

 A general linear model (also called GLM, hence create confusion), there is no g function and f functions are scalar multiplication by numbers. So, the model is of the form:

Y = β0 + β0 x1 + β0 x2 +...+ β0 xn

 Generalized linear model Vs general linear models:

For general linear models the distribution of residuals is assumed to be Gaussian. If it is not the case, it turns out that the relationship between Y and the model parameters is no longer linear. But if the distribution of residuals is one from the exponential family such as binomial, Poisson, negative binomial, or gamma distributions, there exists some functions of mean of Y, which has linear relationship with model parameters. This function is called link function.

For example, a binomial residual can use a logit or a probit link function. A Poisson residual uses a log link function.

Commonly used link functions for different types of response variables:

Where μi denotes the mean response of the ith row and Φ−1(·) denotes the inverse cumulative distribution function of the normal distribution

Read more about link functions: https://en.wikipedia.org/wiki/Generalized_linear_model

 The basic difference between generalized linear model & general linear model can be summarized as follows:

  Examples:

Multiple linear regressions (general linear model):

Example_1:

House price = β0 + β1 * number of rooms + β2  * size of house + β3  * covered parking available (yes/ no) + ... + βn * average income in the respective neighborhood according to census data + white noise

Logistic Regression     (Generalized linear model):

Example_1:

Log (probability the Donald Trump will win US presidential election/ He will lose)

= β0 + β1 * amount of money spent on his campaign+ β2  * amount of time spent campaigning negatively+ β3  * his popularity index + ... + βn * other factors + white noise

Here, Log (probability the Donald Trump will win US presidential election/ He will lose) is called logit, which can be interpreted as log(odd ratio). Interpret regression coefficients is different from that of in case of linear regression.

Example_2:

Log (student will get admission in graduate school/ He/she will not get admission)          

= β0 + β1 * Graduate Record Exam scores + β2 * grade point average+ β3 * prestige of the undergraduate institution+ ... + βn * other factors + white noise

Generalized additive model:

Log (customer will make a purchase on festive sale/ He or she will not make a purchase)

= β0 + f1(number of accounts) + f2(active account types) + f3(credit limits) + ... + βn * (age of the individual) + white noise

 Software implementation:

Reference:

Generalized Additive Models were developed by Hastie and Tibshirani (1986)1.

1http://web.stanford.edu/~hastie/Papers/gam.pdf