#linear model

WOE (Weight of Evidence) & Information Value (IV) Methods for Variable Selection in Machine Learning

Evolved from the logistic regression technique, these two concepts – WOE (Weight of Evidence) and Information value (IV) are being used as a benchmarking technique to screen variables in credit risk modeling / customer churn models to predict the probability of fraud or customer attrition. The Weight of Evidence (WOE) tells the predictive power of an independent variable relative to the dependent…

Revisiting “The Multicollinearity” Problem

What is Multicollinearity?

As we already know that multicollinearity exists whenever two or more than two predictors in a model are moderately or highly correlated. One can think of why cant the researchers collect the data in a way that the predictor variables are not highly or moderately correlated. However, no researcher can control the predictors and how they relate with other predictor…

10 R Powered Visualizations for Power BI Dashboards

While Power BI continues to introduce features and plugins to make this application one stop shop for performing descriptive, predictive and prescriptive analysis. As we all know that Power BI has the capability to embed R Scripts and Visualizations without any fuss. To make things more interesting Microsoft Power BI team has introduced 10 most amazing R based visualizations plugins which are…

The general linear model assumes various predictors affects the response variable additively.

For example, in the model 

Ŷ   =   β0  +  β1 X1  +  β2 X2                                                          ...(1)

the predictors  X1 and X2 are assumed to contribute additively to the response variable Y.

The coefficient β1 can be interpreted as slope of the line or constant of proportionality. It represent the absolute change in Y for one unit absolute change in predictor X1 keeping other predictor x2 fixed. Similarly, if X2 increases by one unit, X1 kept constant, Y is expected to increase by β2 units.

And if both X1 and X2 increase by one unit, then Y is expected to change by (β1 + β2) units. In other words, the total expected change in Y is determined by adding the effects of the separate changes in X1 and X2.

However, in some cases, predictors contribute multiplicatively to the response variable. Then, the model can be expressed as: 

Ŷ   =   β0 *( X1 β1 )*( X2 β2)                                                         ...(2)

Unlike additive models, here the expected percentage change in the response variable Y is proportional to percentage change in the predictor x1 (and similarly for X2).

And if X1 and X2 both change, then the expected total percentage change in Y should be the sum of the percentage changes that would have resulted separately.

The multiplicative model can be converted into linear model by using logarithm transformation.

Applying log transformation to equation (2), we get

Log(Ŷ) = Log(β0) + β1*Log(X1) + β2*Log(x2) 

Biosensors: Trends & Forecasts

C O N T E N T S:

KEY TOPICS

Conclusions and Future Perspectives This paper has examined R&D on nanoparticle-based biosensors and employed bibliometric analyses as a means to help forecast R&D trends and identify the emerging nanoparticle roles in biosensors.(More…)

Although it?s not yet on the market, another trend in biosensor development is non-contact-based vital sign monitoring during surgery.

Here I am using the GapMinder dataset and to examine my research question: whether breast cancer rate can be predicted by urbanization rate and alcohol consumption.

Here, I am considering the below variables:

Predictor Variable: urbanrate, alcconsumption Response Variable: breastcancerper100th

Since both my independent predictors are quantitative, I will need to center them before using them in the regression analysis.

Here is the SAS code to find out the means for my variables:

This snippet generates the means as follows:

Next, I would center the quantitative predictors with the below code snippet and plot regression lines individually to get an idea of the relation of each predictor with the response variable: 

The snippet produces the below scatter plots with regression lines (the response variable against one predictor variable at a time):

Since both individual scatterplots indicate a positive linear trend, I will go ahead with fitting both the variables in my regression model one at a time.

Adding only urbanrate to begin with:

This produces:

It is statistically significant as can be seen from the p-value < 0.0001. However, R-square is only 0.32 meaning we can account for only 32% of the variability in breast cancer using this model.

Next, I’ll try to add another predictor, alcconsumption to see if that improves the model. This can be done with the below snippet: 

This time it produces:

As can be clearly seen, not only are both statistically significant (p-value < 0.0001), together they increase R-square to 0.45 meaning with this modified model we can account for 45% of the variability in breast cancer rate.

Going further, I added a quadratic term for urbanrate to my model:

This time, I have also added code to request residual statistics. This gives me the below:

All three predictors are still statistically significant ((p-value < 0.05), the R-square has risen to 0.48.

As I requested residual stats as well, we get the below from the same code:

Q-Q Plot

Outlier and Leverage Diagnostic:

Studentized Residuals

Summary of Observations:

1.     Each individual predictor (urbanrate and alcoholconsumption) is separately showing positive linear correlation with the response variable breast cancer rate.

 2.     Urbanrate, on its own, can be used as a predictor for breast cancer rate (p-value < 0.0001). This model allows us to account for about 32% of the variability in breast cancer rate. The regression equation we get from this model is:

breastcancerper100th = 37.90639200 + 0.57169795 * urbanrate_c

(where urbanrate_c is the centered urbanrate)

 3.     Adding alcohol consumption rate to the model (along with urbanrate) improves the model. Both predictors still remain statistically significant (p-value < 0.0001). This model gets R-square up to 0.446, in other words, we can account for around 45% of the variability in breast cancer using this model. The regression equation for this model is:

breastcancerper100th = 37.90973617 + 0.46996409 * urbanrate_c + 1.64345926 * alcconsumption_c

(where urbanrate_c is the centered urbanrate and alcconsumption_c is centered alcconsumption)

 4.     Adding a quadratic term for urbanrate to the model, still keeps all the predictors statistically significant (p-values < 0.05) but pushes the R-square up to 0.484. In other words, using this model, we can account for around 48% of the variability of breast cancer rate. The regression equation from this model is:

breastcancerper100th = 33.47240804 + 0.49935408 * urbanrate_c + 0.00856208 * urbanrate_c * urbanrate_c + 1.77696015 * alcconsumption_c

(where urbanrate_c is the centered urbanrate and alcconsumption_c is centered alcconsumption)

 5.     While adding the variables one at a time, I did not see any evidence of confounding.

 6.     From the studentized residual plot, I see there are only 8 residual points that fall outside the -2 to 2 range out of the possible 165 data points. This amounts to less than 5% which is in line with the expectations of standard normal distribution.

 7.     From Outlier and leverage plot, I see there are 6 outliers. None of the outliers fall in the high leverage category. There are 11 high leverage points but none of them are outliers.

Data preparation is essential activity in data analysis. There are several statistical methods that are used in data analysis which include: Linear model, logistic regression, k-means and decision trees. In this blog we will explain these statistical methods.

Data preparation

Data preparation, also known as data pre-processing is manipulation of the data in a form suitable for additional analysis and processing. Many different tasks are involved in this process and these tasks cannot be fully automated. More of the data preparation activities are tedious, routine and time consuming. According some estimations around 60% to 80% of the time spent in data mining projects belongs to the process of data preparation. Data preparation is essential activity for success in data mining projects. The process of data preparation improves the data quality and improves the quality of the data mining results. The process of data preparation involves several steps like checking the data for accuracy, checking or logging the data in, entering the data into the computer and transforming the data.

 Linear model

Linear model or linear regression is a statistical approach for modelling of the relationship between a scalar dependent variable and one or more explanatory variables. There are two cases of linear regression:

simple linear regression where we have one explanatory variable

multiple linear regression where we have two or more explanatory variables.

In linear regression, data is modelled with using of linear predictor functions and unknown model parameters are estimated from the data. These models are called linear models.

Predictions

For generating predictions and residuals from the model, rxPredict() is the function which is used on various types of models. Some of the key arguments used in the formation of the syntax are as follows:

Logistic regression

Next we move on to logistic regression or a logit model. It is used to represent binary outcome variables. The log odds of the output result are represented as a linear arrangement of the predictor variables. To extract the output from R, we can use the summary command for the logit model:

Logit Models need a greater number of scenarios (greater sample size) as they utilise highest likelihood estimation techniques. In few situations, we can evaluate models for dichotomous outcomes where there are just a very few cases using exact regression analysis. It is hard to assess a logit model when the outcome is unusual. This does not depend on the size of the dataset.

k-means

Now, we move to k-means clustering. The k-mean clustering is a popular cluster analysis method in data analysis.  By using rxKmeans() function, we can easily create good visualization for k-means clustering on a certain database.

First of all about the rxKmeans function we should know is the syntax. The rxKmeans function syntax includes formula, data, outFile, numClusters…etc like below. Here we provide the explaination for the core syntax.

formula - Specifies formulas with variables are provided in the clustering algorithm.

data - Variables can be search and found in the formula dataset.

outFile - Cluster IDs can be note and written down in the outFile dataset.

numClusters - Estimation of the clusters’ number (k)

Algorithm means additional arguments which helps  to control the k

Decision tree

The other function which is popularly used in data mining and we call it decision tree. The critical tool in data mining helps analytics to analyse large datasets, whilst by using the decision tree, it is easier to build better visualization. The visualization mainly explores the decision rules for predicting the categorical or continuous outcome. Here we present a bit brief about the syntax we use to grow the tree:

We would like to thank our mentor, Dr Shah Jahan Miah, for providing guidance and support during the activities, which has motivated us to compile our ideas and experience in this blog.

 Authors:

Aleksandar Jankulov   s4518571

Komal Bhalla               s4531062