#decision tree

Norman A. Handy, Jr. - Independent Researcher

May 2026

Abstract

Using a decision tree on the 2012 Outlook‑on‑Life (OOL) Survey (where other algorithms failed due to extreme class imbalance), this study identifies the top survey questions predicting Tea Party membership. The current scholarship extends the analysis to 2016‑2024 ANES data using Decision Tree, Random Forest, and XGBoost to predict strong Trump support (MAGA). The results show that the Black Lives Matter thermometer (therm_blm) is the direct successor to BWEqulJust_7 (criminal justice fairness), while work_way_up ("Blacks should work their way up without special favors") is the direct successor to BWEqulOppty (racial equality of achievement opportunity). This one-to-one mapping provides a precise empirical test of Dr. Ronald W. Walters' theory that racial attitudes adapt to new political symbols.

Introduction

How do researchers measure racial attitudes when explicit racism is socially unacceptable? Political strategists have long used coded language. Lee Atwater famously explained that by the 1980s, overt racial slurs were replaced by abstract economic policies, with the understanding that “Blacks get hurt worse than Whites” as a byproduct. Dr. Ronald Walters systematized this insight, coining the term proxy issues – seemingly neutral survey questions or policy positions that serve as vehicles for underlying racial resentment.

This study applies machine learning to two distinct datasets:

2012 Outlook‑on‑Life (OOL) Survey, targeting Tea Party membership. Only a decision tree produced stable results due to the very small minority class (≈2% Tea Party members).

2016‑2024 ANES Surveys, targeting strong Trump support (feeling thermometer ≥ 70). Here, Decision Tree, Random Forest, and XGBoost all perform well.

The goal is to identify which survey questions are the strongest predictors and to track how those proxies change from the Tea Party era to the MAGA era.

Results

The feature importance tables below show the top predictors for 2016, 2020, and 2024 after dropping the low‑importance features (police_treat, police_howmuch, discrimination_blacks). The BLM thermometer (therm_blm) is the dominant variable in every year and model, while work_way_up is consistently second or third.

The table shows Decision Tree results (the most interpretable model). Full tables for all three models are provided in the appendix.

 Footnote: Feature importances are normalized to sum to 1.0. A value of 0.46 means that variable accounted for 46% of the total predictive power (impurity reduction) of the decision tree; 0.75 means 75%, etc. The higher the number, the more the model relies on that survey question to separate Trump supporters from non‑supporters.

Interpretation: Walters & Atwater

Atwater’s Blueprint → Walters’s Theory → Empirical Test

This represents a key evolution from the 2012 model. In 2012, BWEqulOppty (racial equality of achievement opportunity) was the top predictor and BWEqulJust_7 (criminal justice fairness) was second. By 2016-2024, the focus shifted. The therm_blm (BLM) thermometer is now the dominant variable in every year, while work_way_up, the successor to BWEqulOppty, is consistently second or third.

Explanation for the evolution of the 2012 model

The emergence of therm_blm variable to the top of the feature importance rankings is not accidental.

2014‑2016 - Police body‑worn cameras expand rapidly. In August 2014, President Obama proposes federal funding to reimburse half the cost of body‑camera programs. By 2016, major cities (Washington, D.C., New York, Los Angeles) have launched pilot programs.

2016 – therm_blm already dominates the Decision Tree (0.46 importance). By this point, the public had seen videos of the deaths of Eric Garner, Michael Brown, Tamir Rice, Walter Scott, Freddie Gray, Alton Sterling, and Philando Castile. Body‑worn cameras were becoming standard.

2018‑2020 - Body‑worn cameras become nearly universal in large urban police departments. By 2020, footage is routinely released within days of a shooting, shaping public perception almost immediately.

2020 – therm_blm reaches 0.75 importance. This is the year of Breonna Taylor’s killing, George Floyd’s murder (viral video), and the largest racial justice protests in a generation. The BLM thermometer captured the raw, visceral reaction to what people had watched on their phones, televisions, and computers.

2024 – therm_blm remains the top predictor (0.48 importance). The baseline awareness of police violence had permanently shifted. BLM was no longer a hashtag. It is a firmly established political symbol.

Conclusion

The one-to-one mapping provides a clear empirical test of Walters’ theory: proxy issues evolve in lockstep with the political discourse they reflect. This study demonstrates that machine learning can uncover the evolving proxy issues that express racial and social resentment in American politics. From the Tea Party to MAGA, the undocumented immigrant thermometer remains a consistent strong predictor, while the BLM thermometer rises to dominance as the direct successor to the criminal justice fairness question. The work_way_up variable persists as the successor to the racial equality of achievement opportunity question.

Do Proxies Add Value Beyond Demographics?

To test whether the proxy questions capture attitudes independent of standard demographic and partisan controls, I re‑trained the Decision Tree models on an expanded set of features that included age, education, income (where available), gender, race, rural/urban, and party identification (7‑point scale). All models were evaluated on a 20% test holdout (stratified).

The results show that adding demographics – especially party ID – improves predictive performance, but the original proxies retain meaningful importance.

In 2016, the demographic‑augmented model improved AUC from 0.754 to 0.822. The decision tree feature importances show that party_id (party identification) accounted for 60% of the predictive power, yet therm_undoc (undocumented immigrants) still contributed 14%, indicating that xenophobia adds information beyond partisanship.

In 2020, the improvement was even clearer: AUC rose from 0.895 to 0.935. party_id dominated with 86% importance, but therm_blm (4%) and blacks_gotten_less (2.6%) remained non‑trivial.

In 2024, only limited demographics were available (age, gender, race, rural – party ID was missing for most respondents). Adding these did not improve performance (AUC dropped slightly from 0.839 to 0.806). In this year, the proxies themselves dominated: work_way_up (44%), therm_undoc (17%), and therm_blm (18%) were the top three features.

SHAP Interaction Finding

Across all years, the SHAP dependence plots for therm_blm (BLM thermometer) revealed that the feature with the strongest interaction was blacks_gotten_less (“Blacks have gotten less than they deserve”). The correlation between blacks_gotten_less and the SHAP values of therm_blm was consistently moderate to strong:

This means that the effect of BLM thermometer on Trump support is not uniform – it is amplified among respondents who believe Blacks have gotten less than they deserve, pointing to an interaction between anti‑Black grievance and attitudes toward the BLM movement

SHAP Bar Plots - Global Feature Importance

Figure 1: SHAP bar plots for 2016, 2020, and 2024. The bars show the mean absolute SHAP value for each feature – the average impact of that feature on the model’s prediction (positive or negative). Higher bars indicate greater overall importance.

In 2016, therm_blm is the most important feature, followed by work_way_up and therm_undoc. In 2020, therm_blm dominates with a large margin. In 2024, the most important feature is work_way_up, with therm_blm as a close second. This shift suggests that while BLM remains a powerful proxy, other attitudes (e.g., toward undocumented immigrants) may have gained relative importance by 2024.

Figure 2: SHAP summary dot plots. Each dot represents one survey respondent. The x‑axis shows the SHAP value (positive → pushes prediction toward Trump support; negative → pushes away). Color indicates the feature value (red = high, blue = low).

For therm_blm, red dots (warm toward BLM) consistently appear on the left (negative SHAP) across all years, meaning warmer BLM feelings decrease Trump support probability. Blue dots (cold toward BLM) appear on the right (positive SHAP). This pattern is strongest in 2020, where most therm_blm dots are concentrated on the far left. Other top features, such as work_way_up and therm_undoc, show the opposite direction (higher values → positive SHAP).

Figure 3: Dependence plots for therm_blm. The x‑axis shows the BLM thermometer values as coded in the ANES data (range 0–1000, where 1000 corresponds to the warmest possible feeling). The y‑axis is the SHAP value for that feature (positive → pushes toward Trump support; negative → pushes away). Points are colored by the interacting feature with the highest correlation – here, blacks_gotten_less (“Blacks have gotten less than they deserve”). In 2020, the downward slope is steepest: higher BLM warmth (toward 1000) sharply lowers Trump support probability. The color reveals that respondents who also agree with blacks_gotten_less (high values – i.e., believe Blacks got less than they deserve) show even more negative SHAP, indicating an interaction between anti‑Black grievance and BLM attitudes. Note: SHAP (SHapley Additive exPlanations) values measure how much each feature pushes a prediction away from the baseline (average Trump support rate). Positive SHAP → increases Trump support probability; negative → decreases. The bar plot shows mean absolute SHAP (overall importance). The summary dot plot shows direction and distribution. The dependence plot shows how the effect of therm_blm changes with its value and interacts with other features

Appendix: Full Feature Importance Tables 

Figure 4: Full feature importances by year and model (Decision Tree, Random Forest, XGBoost).

Ethical note

Identifying proxies is not about labeling individuals. It is about understanding how systems of racial resentment operate beneath the surface – a necessary step for those who wish to counteract the politics of division.

Interactive Dashboard

Explore the decision tree models live: Proxy Politics Dashboard

References 

Atwater, L. (1981). Interview (excerpts).   

Walters, R. W. (2003). White Nationalism, Black Interests. Wayne State University Press.    

ANES (2016, 2020, 2024). American National Election Studies.   

In my analysis, "Predicting Political Party Membership: A Validated Decision Tree Approach," I set out to predict a rare event: Tea Party membership (only 2% of the unweighted sample). A decision tree was built that achieved validation AUC = 0.862 and caught 79% of actual Tea Party members (sensitivity).

But before settling on that tree, I tested three other widely used algorithms. All of them failed – some, in spectacular fashion – for reasons that teach an important lesson about small, imbalanced data.

1. Logistic Regression (SAS, stepwise, validated on 30% holdout)

A stepwise logistic regression with the same weighting (weight=34 for Tea Party members) and the same validation split. I even added the MISSING option to handle missing values like the decision tree.

Results (validation):

AUC = 0.653 (barely better than a coin flip)

Sensitivity = 0.31 (caught only 31% of true Tea Party members)

Specificity = 0.88

Why it failed: Logistic regression assumes linear, additive relationships – no interactions. The true patterns in the data are non‑linear and interactive (e.g., “strongly agree with equal opportunity” and “rate undocumented immigrants very low” → high Tea Party probability). A linear model cannot capture that, no matter how many variables you throw at it.

2. XGBoost (Python, single tree mode, tuned)

I used the same features, one‑hot encoding, missing indicators, and class weight (scale_pos_weight=34) with depths from 4 to 10 and weights from 20 to 47.

Best result (validation):

AUC = 0.718

Sensitivity = 0.071 (only 1 out of 14 true positives caught)

Specificity = 0.99 (almost never predicted Tea Party)

Why it failed: XGBoost’s gradient‑based splitting is dominated by the massive majority class, even with scale_pos_weight. It became extremely‑conservative, preferring to predict “non‑member” almost always. The rare class signal was too weak for its boosting mechanism.

3. Random Forest (Python, 100 trees, max depth=5, sample weights)

A random forest with the same depth and leaf size as the decision tree, using sample_weight to replicate SAS’s WEIGHT statement.

Results (validation):

AUC = 0.806 (slightly higher than the single tree)

Sensitivity = 0.214 (only 3 of 14 true positives caught)

Specificity = 0.973

Why it failed (for my purpose): Although the AUC was respectable, the forest missed 79% of actual Tea Party members. It was too conservative – it only predicted “Tea Party” when the signal was overwhelming. For my goal (identifying who is likely to be a Tea Party member), that low sensitivity makes the model useless. The single tree was far better at actually finding the rare cases.

The Lesson: Fancy is not always Better

With only 2,294 observations and a 2% target rate, complex algorithms often:

Over‑fit to the majority class

Become overly conservative

Fail to learn rare patterns

A well‑pruned, weighted decision tree proved to be the best – because it:

Captures non‑linear interactions naturally

Handles missing values by treating them as a separate category

Gives interpretable rules (e.g., “If BWEqulOppty = 1 or 2 and RateUnDoc_100 < 15.16 → 75% Tea Party probability”)

Achieves high sensitivity: it actually finds the people we care about.

So when someone tells you “you should use XGBoost or random forest, (or some other, more powerful algorithm)” remember: the simplest model that fits your data and your goal is often the right one.

*** Final Note: Why SVM and Neural Networks Were Not Attempted ***

Again, with only 2,294 observations and a 2% target rate, two other popular algorithms – Support Vector Machines (SVM) and Neural Networks – were not even attempted. Here’s why:

SVM – Relies on finding support vectors from both classes. With only ~45 positive cases, the minority‑class support vectors would be too few to define a stable decision boundary. SVMs also cannot handle missing values natively and do not produce interpretable rules.

Neural Networks – Require large amounts of data to learn meaningful weights. With 2,294 rows and a rare target, any neural network would either memorize the training set or never converge to a useful pattern. Moreover, they lack the interpretability of the simple decision tree which is the antithesis of my goal to understand why people are Tea Party members.

Each decision leads to a number, look at the list below to find the link to the right number. Please check the tags of the fanfic before reading.

List

false criminal suspect

Queening the Pawn

Endless Robins

Lost Hood

Trapped in Truth

Light of Life

The city we rule in the dark

Prophecy

Family Is More Important Than Business

Every day a bat dies

King of the Underworld

H.R. is standing for Hood, Red

Soul

Emotion in cars

Where sorrows live and raptures die

I heard a rumor

Remember me

Under the Gray Hood

Pizza does not come into my house

Know your body

One of the famous ensemble algorithms is random forest which trains small decision tree models, or week model, in parallel. Then, we combine their output using voting for a classification problem. Here, we train different week models using different sampling of all data. Because of sampling, around 70% data are extracted from the original box and other 30% are taken as repeated samples. Because of this randomized dataset, each week decision tree operates differently once their trains are completed.

Decision Tree Images: Create Effective Visuals Introduction: Beyond Pixels – Representing Images for Machine Learning Traditional machine learning algorithms, particularly deep neural networks, thrive on structured data. However, images – a ubiquitous form of data in our world – are inherently unstructured. They consist of vast arrays of pixel values, often representing raw color information.…