#logistic regression

"Talented people will find a way to make a living.

How can you be certain it's with you and your company?"

 Attrition of employees has become a critical concern for a company's competitive edge. Finding, hiring, and training new employees is quite costly. It is more cost-effective for a corporation to maintain its current staff. To keep its staff for a longer amount of time, a corporation must maintain a nice working environment.

Whether an employee will stay or quit a firm, his or her response is just binary, i.e., "YES" or "NO." A link between predictor variables and a categorical response variable is analyzed using logistic regression. It's a technique for analyzing a data set with a dependent variable and one or more independent variables in order to predict the result of a binary variable, which means there are only two possibilities. The sigmoid function, also known as the logistic function, produces a 'S' shaped curve that may be used to transfer any real-valued integer to a value between 0 and 1.

p = 1 / 1 + e-y

e - y = (p / p – 1)

y = log (p / p – 1)

log (p / p – 1) = β0 + β1X1 + β2X2 + … + βnXn

Here employee attrition will be the dependent categorical variable so we are using logistic regression to predict and analyze the employee attrition.

I'll show you how to utilize R software to assess employee attrition in five simple stages.

·         Exploration of data set

·         Data pre-processing

·         Dividing the data into two parts "training" and "testing"

·         Use the "training data set" to build the model.

·         Use the "testing data set" to conduce the accuracy test.

Exploration of Data set

This data set was collected using Kaggle. There are 14999 observations and 10 variables in this dataset. "Attrition(left)" is the dependent variable among ten variables.

Data pre processing

1.      Detecting the variable names in the data set

#Variable names in the data set

variable.names(HR)

2.      Detecting the summary of all variables in the data set

#Summary of all the variables in the data set (5point summary)

summary(HR)

3.      Detect the missing values

Let check whether there is any missing values in the dataset.

missmap(HR,main ="Missing values vs observed")

Result: No missing value was observed

4.      Changing data types

To build a good model, we must first change the data type of the character variables to factor.

Department and pay are the two-character variables. 8 and 9 are the column numbers, respectively.

#Converting the data of department and salary into factor to analyse the data

HR [, c (8,9)] =lapply (HR [, c (8,9)], as. factor)

Dividing the data into "training" and "testing"

In every regression analysis, the dataset must be divided into two parts:

·         Data set for Training

·         Data set for Testing

First, we’ll develop our model with the Training Data set and then verify its accuracy with the Testing Data set.

# Splitting of data set as training data and testing data to analyse further

ran=sample(x=c("Training","Testing"),size=nrow(HR),replace=T,prob=c(0.8,0.2))

TrainingData1=HR[ran=="Training",] #training data set

TestingData1=HR[ran=="Testing",] #testing data set

#no. of rows presents in the training data set

nrow(TrainingData1)

#no. of rows presents in the testing data set

nrow (TestingData1)

Building up the model

We'll next build the model together in a few simple steps, as follows:

·         Identify the variables that are independent.

·         Build the model by incorporating TRAINING data into the equation.

#Dependent variable is attrition (left)

independent variables=colnames(HR[,1:9])

#Display of all the independent variables in the data set

independent variables

Then, using the "glm" function, we'll add "Training Data" to the equation and create a logistic regression model.

#Incorporation of training data to build the logistic regression model

#Using glm function

model1=glm(HR$left~HR$satisfaction_level+HR$last_evaluation+HR$number_project+HR$average_montly_hours+HR$time_spend_company+HR$Work_accident+HR$promotion_last_5years+HR$Department+HR$salary,data=TrainingData1,family=binomial(link='logit'))

#Summary of the logistic regression model

summary(model1)

As we can see from the table above, employee satisfaction, last evaluation, number of projects, average monthly hours, time spent in the company, work accident, promotion in the last five years, and salary are all crucial factors in influencing employee attrition. There will be less chances of losing an employee if the organisation focuses on these areas.

Confusion Matrix

# Run the test data through the model

res1<-predict (model1,TestingData1, type = "response")

res1

#validate the model - confusion matrix

confusionMatrix<-table(Actual_value=HR$left,predicted_value = res1 >0.5)

confusionMatrix

Interpretation

Our logistic regression model works quite well. This method can be used to analyse any dataset of employee attrition facts, helping the company to gain a competitive edge and effectively manage its resources.

Course: Master of Business Administration

Amrita School of Business, Coimbatore

Amrita Vishwa Vidyapeetham.

... according to an algorithm trained to recognize cats.

Artists of ATLA/LOK ... I want to use your owl-cats, cat-bats, cat-erpillars, orangu-cats, and other cat hybrid art!

I’ll classify them as ‘cat’ or ‘not cat’ using my algorithm, and make a fun post where you’ll of course be credited for your contribution.

Three Regression Types

Generalized Linear Models (GLM) extend the ordinary linear regression and allow the response variable y to have an error distribution other than the normal distribution.

8 1 Generative vs Discriminative Models https://youtu.be/YQClUDd9ff4

8 2 Making features from text for discriminative NLP models https://youtu.be/MemiaOYSB0k

empirical E() 는 실제 data를 이용해서 계산하고 model expectation은 P() 즉 확률을 이용하는 것 같다. 기대값 구하는 공식중에 확률에 확률값을 곱하면 기대값이 되었던 것을 기억하면 이해가 쉽다.

8 3 Feature Based Linear Classifiers https://youtu.be/7-7MlBdy3EE

위 그림에서 location인지 아닌지를 평가하는 feature1의 결과는 1.8 feature2의 결과는 -0.6이다. 그래서 이를 합산하면 1.2가 된다. 

exp 를 함으로써 합산된 features계산 값들이 양수가 되게한다. 

참고) exp 그래프

In my analysis, "Predicting Political Party Membership: A Validated Decision Tree Approach," I set out to predict a rare event: Tea Party membership (only 2% of the unweighted sample). A decision tree was built that achieved validation AUC = 0.862 and caught 79% of actual Tea Party members (sensitivity).

But before settling on that tree, I tested three other widely used algorithms. All of them failed – some, in spectacular fashion – for reasons that teach an important lesson about small, imbalanced data.

1. Logistic Regression (SAS, stepwise, validated on 30% holdout)

A stepwise logistic regression with the same weighting (weight=34 for Tea Party members) and the same validation split. I even added the MISSING option to handle missing values like the decision tree.

Results (validation):

AUC = 0.653 (barely better than a coin flip)

Sensitivity = 0.31 (caught only 31% of true Tea Party members)

Specificity = 0.88

Why it failed: Logistic regression assumes linear, additive relationships – no interactions. The true patterns in the data are non‑linear and interactive (e.g., “strongly agree with equal opportunity” and “rate undocumented immigrants very low” → high Tea Party probability). A linear model cannot capture that, no matter how many variables you throw at it.

2. XGBoost (Python, single tree mode, tuned)

I used the same features, one‑hot encoding, missing indicators, and class weight (scale_pos_weight=34) with depths from 4 to 10 and weights from 20 to 47.

Best result (validation):

AUC = 0.718

Sensitivity = 0.071 (only 1 out of 14 true positives caught)

Specificity = 0.99 (almost never predicted Tea Party)

Why it failed: XGBoost’s gradient‑based splitting is dominated by the massive majority class, even with scale_pos_weight. It became extremely‑conservative, preferring to predict “non‑member” almost always. The rare class signal was too weak for its boosting mechanism.

3. Random Forest (Python, 100 trees, max depth=5, sample weights)

A random forest with the same depth and leaf size as the decision tree, using sample_weight to replicate SAS’s WEIGHT statement.

Results (validation):

AUC = 0.806 (slightly higher than the single tree)

Sensitivity = 0.214 (only 3 of 14 true positives caught)

Specificity = 0.973

Why it failed (for my purpose): Although the AUC was respectable, the forest missed 79% of actual Tea Party members. It was too conservative – it only predicted “Tea Party” when the signal was overwhelming. For my goal (identifying who is likely to be a Tea Party member), that low sensitivity makes the model useless. The single tree was far better at actually finding the rare cases.

The Lesson: Fancy is not always Better

With only 2,294 observations and a 2% target rate, complex algorithms often:

Over‑fit to the majority class

Become overly conservative

Fail to learn rare patterns

A well‑pruned, weighted decision tree proved to be the best – because it:

Captures non‑linear interactions naturally

Handles missing values by treating them as a separate category

Gives interpretable rules (e.g., “If BWEqulOppty = 1 or 2 and RateUnDoc_100 < 15.16 → 75% Tea Party probability”)

Achieves high sensitivity: it actually finds the people we care about.

So when someone tells you “you should use XGBoost or random forest, (or some other, more powerful algorithm)” remember: the simplest model that fits your data and your goal is often the right one.

*** Final Note: Why SVM and Neural Networks Were Not Attempted ***

Again, with only 2,294 observations and a 2% target rate, two other popular algorithms – Support Vector Machines (SVM) and Neural Networks – were not even attempted. Here’s why:

SVM – Relies on finding support vectors from both classes. With only ~45 positive cases, the minority‑class support vectors would be too few to define a stable decision boundary. SVMs also cannot handle missing values natively and do not produce interpretable rules.

Neural Networks – Require large amounts of data to learn meaningful weights. With 2,294 rows and a rare target, any neural network would either memorize the training set or never converge to a useful pattern. Moreover, they lack the interpretability of the simple decision tree which is the antithesis of my goal to understand why people are Tea Party members.

Introduction

Logistics teams deal with numbers all day, delivery times, failed attempts, delays, customer availability, traffic conditions, and more. The challenge is not the lack of data, but understanding what actually matters. This is where logistic regression software becomes useful. Instead of relying on assumptions or past habits, teams can study patterns and probabilities to make better decisions. Platforms like LogiNext have shown how structured data analysis supports clearer planning and fewer surprises in daily operations.

What Is Logistic Regression Software?

Logistic regression software is used to study how different factors influence a specific outcome. In logistics, this outcome could be something like whether a delivery is completed on time, whether a delivery attempt fails, or whether a route is likely to face delays.

The software looks at historical data and identifies relationships between inputs and results. For example, it can help determine whether delivery time windows, distance, traffic patterns, or order volume increase the risk of delay. The goal is not to predict exact outcomes, but to understand the likelihood of different scenarios.

How It Fits Into Logistics Operations

In real-world logistics, decisions are often made under pressure. Logistic regression software helps teams slow things down and look at facts instead of instincts.

Operations teams can use it to identify recurring issues, such as why certain areas see more failed deliveries or why delays happen at specific times of day. Planners can test “what-if” scenarios before changing routes or schedules. Over time, this builds confidence in planning decisions.

The software does not replace human judgment. Instead, it supports teams by highlighting patterns that may not be obvious at first glance.

Practical Benefits for Logistics Teams

Using logistic regression software brings clarity to complex operations.

Teams gain a better understanding of delivery risks Planning decisions become more consistent Operational issues are identified earlier Performance reviews are based on data, not opinions

By learning from past outcomes, teams can avoid repeating the same mistakes and improve overall efficiency.

Why It Matters in a Data-Driven Environment

As logistics operations grow, relying only on experience becomes risky. Volumes increase, routes change, and customer expectations rise. Logistic regression software helps teams adapt by turning raw data into meaningful insights.

It allows businesses to move from reactive problem-solving to proactive planning. Instead of asking why something went wrong, teams start asking how to prevent it next time.

Conclusion

To further investigate the factors associated with life expectancy, we fitted a logistic regression model using a binary indicator of life expectancy (high vs. low) as the response variable. Income per person was specified as the primary explanatory variable, with urban rate and internet use rate included to assess their independent effects and potential confounding. The Python syntax below shows how the model was estimated.

The model output were as follows:

Interpretation of the results: The overall model fit was strong (Pseudo R² = 0.53; likelihood ratio test p < 0.001), indicating that the explanatory variables jointly explain a substantial proportion of the variation in life expectancy.

Among the explanatory variables, internet use rate was significantly and positively associated with the odds of having a high life expectancy. Specifically, each one-unit increase in internet use rate was associated with a 7.2% increase in the odds of high life expectancy (OR = 1.07, 95% CI [1.04, 1.11], p < 0.001). In contrast, income per person showed a marginal, non-significant positive association with life expectancy (OR = 1.00, 95% CI [1.00, 1.00], p = 0.079), while urban rate was not significantly associated with life expectancy after adjustment (OR = 1.02, 95% CI [0.99, 1.05], p = 0.163).

Our primary hypothesis was that higher income per person would be associated with higher life expectancy. Although the estimated odds ratio for income per person was greater than 1, the association did not reach conventional levels of statistical significance. Therefore, the results did not fully support our hypothesis in the multivariable logistic regression model.