#incrementality

Everyone wants the one attribution model that will finally settle every budget argument. In practice, anything that actually works in 2025 looks less like a magic formula and more like a small stack of imperfect signals that each answer a different question.

Here is how I’ve started structuring attribution work for media and analytics teams, in a way that stays realistic about privacy limits, GA4 weirdness, and messy channel mixes.

Layer 1: Fast, directional attribution

This is the “what is probably working right now” layer.

Tools are usually:

GA4 or similar analytics platform

Channel level cost data

A few basic models, like last click, data driven, time decay or position based

The point is not perfection. The point is to have a quick view that can answer questions like:

Did that new branded search campaign tank non brand performance

Are we spending on channels that never show up in any path

Is direct somehow “winning” everything because tracking is broken

Things that actually help here:

Trends over time, not one report Look at how channels move together or in opposite directions week to week. If CTV flights line up with spikes in branded search, that is already a useful hint.

Path patterns instead of precise credit shares Seeing search show up mainly in the middle of the path, or email frequently closing journeys, changes how you talk about those channels, even if the exact percentage credit is fuzzy.

Cleaning up obvious nonsense If “direct” is half your conversions, or display somehow has a better CPA than branded search, that is usually a tracking issue, not a miracle.

Layer 1 is good for weekly and monthly discussions, on off decisions, and quick “move a little budget” actions. It is not strong enough to defend large shifts by itself.

Layer 2: Incrementality and geo testing

This is where things start getting closer to causality.

Questions at this layer sound like:

What actually happens if we pause this channel in a few regions

How much lift does retargeting really provide when you hold out a group

Does that “upper funnel” campaign move branded search or direct in a measurable way

Simple approaches that work:

Geo split tests, where some cities or regions get media and others do not

Holdout groups for retargeting or loyalty campaigns

Treated vs control store sets when you mix offline and online

Two rules keep these tests from turning into endless science projects:

Test one thing at a time Pick one channel, tactic, or audience and isolate it as much as you reasonably can. You are looking for “directionally clear” results, not something that will go in a journal.

Pick one primary KPI before you start Decide if the test is about revenue per user, new customer rate, lead quality, or something else. If you measure ten KPIs and chase the one that looks nice, the result will never feel trustworthy.

The goal is to end up with multipliers that you can remember and use in conversation. For example, “when we run this prospecting campaign at X level, we usually see about a 10 to 15 percent lift in branded search conversions in treated regions.”

Those small rules of thumb are gold later when people fight about budgets.

Layer 3: Model based planning, like MMM

The slowest layer, but also the one leadership tends to respect once it is framed correctly.

This is where you might use:

Marketing mix modeling

Advanced data driven models across channels

Long term historical data with seasonality and promo variables

You are mostly looking for three things:

Elasticities How outcomes change when spend moves up or down, instead of just average ROAS at one spend level. That is what helps answer “what happens if we add 20 percent here and remove 15 percent there.”

Cross channel effects Maybe TV or CTV lifts search and direct. Maybe display does not close many sales but helps feeds retargeting pools. Good models quantify those relationships instead of leaving them as vague hand waving.

Diminishing returns curves Every channel breaks at some point. Being able to show “this is where paid search goes from strong to very flat” is a huge help when people want to dump extra budget into the same comfortable channel forever.

I treat these models as decision calculators during planning seasons, not as permanent truth. They serve best when they guide discussions like “how should we shift the next 10 to 20 percent of spend” rather than “who gets 100 percent of all credit forever.”

How the three layers work together

The most useful pattern has been:

Layer 1 to keep a live narrative of what is working and what looks broken

Layer 2 to validate or challenge that narrative with real tests

Layer 3 a few times per year to guide bigger mix decisions and answer “what if” questions for leadership

The result is not a single model, it is a small set of views and rules that you can actually explain in plain language.

Helpful resources if you live in this world

A few places that post solid material in this space:

Adswerve has a lot of practical content on GA4, Google Marketing Platform, and BigQuery, especially around measurement setups and reporting patterns for real media teams: https://www.adswerve.com

Bounteous publishes good pieces on analytics architecture, experimentation, and how to connect product analytics with marketing data without turning everything into a dashboard beauty contest: https://www.bounteous.com

I treat both as idea banks, not rulebooks, but they are handy when you need to sanity check your own approach or grab a new angle for explaining attribution tradeoffs to non technical folks.

If you are building your own attribution setup, I am curious how close or far this is from what you are doing.

Have you ever wondered how much impact your retargeting campaign truly has on your conversions? The answer lies within the realm of incrementality testing, a powerful tool that allows you to unravel the effectiveness of your retargeting providers. By delving into the depths of testing, you can unlock valuable insights that will help you optimize your marketing efforts and maximize your ROI. 

The Essence of Incrementality Testing

The purpose of this testing is to shed light on the portion of conversions that can be directly attributed to your retargeting campaign. By comparing the conversion rates of a control group (those not exposed to retargeting ads) and a test group (those exposed to retargeting ads), you can discern the incremental lift generated by your retargeting efforts. This insight enables you to refine your retargeting strategies, ensuring that you invest your resources where they will yield the greatest returns.

The Role of a Dedicated Analytics Team

People say high bounce rates are bad. At first glance, this makes sense. After all, who’d want customers to visit a (web or app) page and leave without seeing any more pages? We might decide, therefore, that digital advertising may be made more efficient by eliminating channels with high bounce rates.

This is a mistake. Let me explain why.

Consider the following:

1. Salesperson A stands just outside a retail store handing out pamphlets. 90% of the people who get pamphlets visit the store. Of the people who visit the store, all of them end up shopping at the store. 10% Bounce rate, 100% Conversion rate (=customers/visitors).

2. Salesperson B expends shoe-leather, visits customers’s homes and hands out pamphlets, even speaks to a few customers. 10% of the people who’ve seen the pamphlets end up visiting the store; of these, 50% end up shopping. 90% of people “bounce” i.e. they see the pamphlet but don’t visit the store. 90% Bounce rate, 50% Conversion rate.

3. Salesperson A and B have the same fixed salary.

Would you, on the basis of the above data, conclude that Salesperson A is the better salesperson? Or, that Salesperson A is attracting “higher-quality traffic” than Salesperson B?

Consider the following question instead:

Suppose Salespersons A and B hadn’t handed out pamphlets. What percentage of people would have become customers anyway?

The conclusion is the very opposite of what we’d get if we went by bounce rates or conversion rates. 

Granted, it’s a lot easier to measure things based on last-click based attribution, and a lot harder to measure Incrementality. This would be alright if a sole emphasis on last-click attribution and bounce rates resulted merely in a suboptimal solution. The trouble is, it can lead you in exactly the wrong direction.

Notes

1. The title of this essay is inspired by the computer scientist Edsger Dijkstra’s essay “Go to statement considered Harmful”, see http://www.u.arizona.edu/~rubinson/copyright_violations/Go_To_Considered_Harmful.html

2. I’ve taken some liberties in extending the concept of bounce rates to the offline world. Specifically, bounce rate is defined as the % of total sessions that begin and end at the landing page. In the Salesperson analogy, I’m considering a “view” of the pamphlet to be a visit to the landing page, and a visit to the retail store as the negation of bounce. Imperfect though the analogy is, I think it captures the essence of the problem well enough.

3. For an example of how an emphasis on different metrics, each seemingly sensible, leads to entirely opposite conclusions, see Jeff Bezos’s 2004 letter to shareholders, where he compares the effect of optimizing for free cash flows versus net profits. https://www.sec.gov/Archives/edgar/data/1018724/000119312505070440/dex991.htm. 

In this letter, Bezos says: “Though some may find it counterintuitive, a company can actually impair shareholder value in certain circumstances by growing earnings.” 

Analogously, marketing teams can actually impair their effectiveness in certain circumstances by trying to reduce bounce rates. 

There’s another great quote from the same letter: “Cash flow statements often don’t receive as much attention as they deserve. Discerning investors don’t stop with the income statement.” 

Of marketing, I’d say the following: Incrementality doesn’t receive as much attention as it deserves. Discerning marketers don’t stop at bounce rates and conversion rates, they also consider incrementality.

All retail marketers try to measure the impact of their strategies and tactics. They have been asked to prove return on investment for too long to have survived without an answer. Yet many marketing teams use the wrong approach. In fact, the most common approaches to knowing what marketing is working don’t actually measure ROI at all.

Forrester found that marketers spent approximately $1.09 billion in 2017 to measure their impact. In many large companies, that investment goes to multitouch attribution. Smaller companies rely on attribution rules, such as last-click attribution or weighted attribution. Attribution models have different reasons for their allure, but they have one thing in common: they don’t actually measure the incremental sales driven by marketing.