Understanding incrementality for marketing success (2024)

What is Incrementality in Marketing?

Key Terms and Concepts

Incrementality

“Incrementality” in marketing refers to the causal lift in sales driven by a given marketing activity for a given period of time. Not all sales that occur after a marketing exposure are deemed to be incremental, as often these sales “would have happened anyway” with or without that marketing exposure.

For example, let’s say a customer is already intending to purchase a certain product after visiting a brand’s website. The brand may subsequently serve a retargeting advertisem*nt in the days following, but this ad did not incrementally drive the purchase as the customer was already planning on purchasing anyway.

Incrementality Measurement

Incrementality measurement is the systematic use of in-market experimentation to measure media’s impact on sales and the application of experiment results to an attribution and optimization framework.

Experimentation (in-market Test vs. Control, or Exposed vs. Non-Exposed) is the only way to infer the causal or “Incremental” Impact of media on sales, in the same way that randomized control trials are the only way to measure the efficacy of a new medicine.

Methodologies that rely purely on observational models (such as MMM, MTA, etc.) are useful in many instances, but they measure the correlative impact of media on sales, and as we know, correlation is not necessarily indicative of causation.

Attribution vs. Incrementality

In general, “attribution” is the process of crediting each marketing channel with the number of sales they were responsible for driving for any given period of time.

An “Attribution System” is a continuous, granular, and comprehensive reporting framework that tracks where and when a brand deployed its marketing budget and the attribution of sales subsequently driven by the various portions of that budget.

Attribution systems allow brands to calculate the ROI (Return on Investment, aka ROAS or Return on Ad Spend) of a given marketing channel or campaign and enable ongoing optimization of budget allocation to maximize overall returns to the business.

Incrementality experimentation is a vital input into any accurate attribution system; however, due to its episodic nature, incrementality experimentation is not an attribution system in and of itself.

Incrementality results need to be applied to an attribution framework in a systematic, consistent, and intelligent manner to be useful.

How Does Incrementality Differ From Traditional Attribution Methodologies?

Incrementality is one of several methodologies that can be deployed to solve the attribution problem. For an overview of the most common approaches, see here.

How Do You Measure Incrementality?

There are several variations of incrementality experimentation, but they all rely on the same principle mechanic: splitting an audience into multiple representative groups and serving each group different media treatments to observe the impact on conversion behavior.

Core Principles of Incrementality Experimentation

Based on 1st-Party Transaction Data

One key feature of incrementality experimentation (as opposed to on-platform Lift Studies or Brand Lift Studies) is that results are based solely on a brand's 1st-party transaction data.

This is critical because the core priority in experimentation is to measure a media channel’s net impact on a brand's sales, including all the interactive effects the media may have on other media channels, organic channels, etc.

On-platform lift tests that measure differences in platform-tracked conversions, for example, are blind to the impact of one media channel on another Media channel’s performance, which is a key element in evaluating the incremental ROI.

Types of Experiments

There are many possible configurations of Incrementality experiments, but most fall under three main categories:

Holdout

A holdout experiment removes (or “holds out”) a media channel from a selected portion of the audience, and observes their conversion behavior vs. a control group who received that media as per usual.

(Note: the exposed group is referred to as a “control” in this case because they represent the typical Business as Usual scenario of the Media being active).

Scale

A scale experiment is the inverse of a holdout, where a given Media Channel is “scaled-up” in investment, usually 2x-4x, for a selected audience, and conversion behavior is compared to an audience receiving the normal level of spend.

Multi-Treatment

The most complex configuration is multi-treatment, which entails splitting an audience into more than two “cells” and testing different combinations of media to observe their relative lifts vs. a holdout group.

A common use case for multi-treatment is to measure the overlap between channels. For example, an experiment with three test cells: A Google Holdout, a Facebook Holdout, and a third cell with both Facebook and Google held-out, all compared to a control group (business as usual).

An experiment configured this way will reveal both the individual contributions of Google and Facebook, and also their impact when paired together. This will inform whether they have a synergistic or cannibalistic relationship (in other words, if they work better together or step on each other’s toes).

Types of Audience Splits

All the above examples refer to “splitting” an audience into multiple representative groups to which we can serve different media treatments. In general, there are two main ways to split an audience:

Known-Audience Split:

A Known-Audience Split groups individual users from an existing user list into different treatment cells. This is only possible for media channels where user-based targeting is available, typically CRM-based channels such as E-Mail, Catalog, SMS, etc.

When designing a Known-Audience split, the main factors to account for and control for are recency, frequency, and monetary value of recent purchases and a user's eligibility (e.g., opt-in vs. opt-out) to receive the media in question or any other related media.

Geographical Split:

A Geo-split is used when an audience is unaddressable, meaning that individual users cannot be targeted directly from a pre-existing list, and therefore, a Known-Audience Split isn’t feasible. This applies to any channel that employs broad targeting like social prospecting, CTV prospecting, and paid search.

The Geo-split method identifies specific markets within a broader region (i.e., country) that are statistically representative of that broader region and groups these markets into a test cell for experimentation. A treatment (e.g., Google Holdout) is then applied to this test cell, and conversion behavior in these markets is compared to a group of control markets (business-as-usual).

When designing a Geo-split, the main factors to account for and control for are a market’s sales trend and seasonality, population conversion rate (i.e., market penetration), and media relevancy (historical execution of the media in question being representative of the broader region).

Why is it Important to Measure Incrementality?

Incrementality experimentation is vital to accurate attribution for a number of reasons, the most significant being:

Collinearity between Marketing Vehicles

Brands tend to scale (and lower) spend across all media channels at similar times (for example, around a product launch or around their most seasonal period). This creates an inherent inter-correlation between various media channels that makes it impossible to isolate the impact of one specific media channel simply by observing the data (a common problem with MMM and MTA models).

Incrementality solves this problem by physically isolating the impact of one media channel in a Test vs. Control experiment: if all else is equal between two audiences, except for the removal of ONE media channel, we know the difference in conversion behavior between those two audiences is due to the impact of that channel alone.

Collinearity between Sales and Marketing

Many lower-funnel channels (for example, branded search) are not only potential drivers of demand but are also driven by demand (for example, when more people are interested in your product, your search click volume will increase). This creates an inherent inter-correlation between these channels and sales, which makes it difficult to determine how many sales coming after a brand search click were incrementally driven by that paid link (versus sales that would have happened anyway via organic links had the paid link not been served, for example).

The solution is to run an incrementality experiment where brand search is removed from a set of test markets, and sales trends in these markets are compared to a set of control markets where brand search remains active.

The difference in conversion behavior between these two groups determines the incremental impact of Brand Search on the business.

The Impact of Incrementality Measurement in Marketing

Incrementality Measurement allows businesses to optimize media investment by identifying which platforms, channels, and campaigns have the largest causal impact on their business, agnostic of what the platforms may be reporting themselves or what a Brand may see in site-side analytics (Google Analytics, Adobe Omniture).

This leads to better investment decisions by revealing the true Incremental Return on Investment of a given channel and enables brands to optimize their budget allocation based on this key metric.

How Do You Optimize Advertising ROI Through Incrementality Measurement?

The ultimate goal of Incrementality is to inform media budget allocation with the goal of either:

1. A) Maximizing sales for a fixed budget
2. B) Minimizing spend required to reach a specific Sales goal
3. C) Maximizing spend while maintaining a profitable Incremental ROI.

Though strategies A and B satisfy certain use cases, the best optimization strategy, generally speaking, is C, as it maximizes profitable Sales that can be driven by marketing, all else being equal.

Strategy A runs the risk of leaving profit on the table if the total ROI is well above “break-even” after every dollar in the fixed budget is invested. Strategy B could similarly leave profit on the table if the given sales goal is too low (a higher goal could be achieved profitably with additional investment).

In either case, the optimization principles work the same: reallocate investment from channel with lower mROI to higher mROI until total ROI is maximized across the entire portfolio.

Note: (mROI = Marginal ROI, or the Return on Investment of the next incremental dollar invested in a given channel)

As money flows into a channel, the mROI (return on the next dollar invested) will decrease due to diminishing returns. As such, we generally want to keep spending money on a given tactic until its mROI is no longer profitable.

This allocation is generally performed using an optimizer or a tool/program that can automatically calculate the best channel to allocate the theoretical “next-dollar” until all the dollars within a budget are allocated or until “break-even” mROI is achieved with a fluid budget.

In theory, an optimal allocation would have every channel showing equal mROI, right at or just above “break-even.” This rarely happens in actuality, as many channels are constrained by real-world factors that inhibit investing or divesting freely (e.g., you’ve already committed and paid for next year's sponsorship deal with X sports league and cannot reallocate that budget).

Incrementality Measurement Case Study: Soft Surroundings

Let's consider a women's clothing retailer, Soft Surroundings, that sought to improve its Retargeting strategies. They partnered with Measured to conduct a Retargeting experiment using Measured's incrementality platform. Surprisingly, they learned that their incremental cost per acquisition (CPA(i)) significantly exceeded both their CPA targets and vendor-reported figures. The primary retargeting vendor, responsible for most of their spend, was over-indexed and often exceeded recommended ad-serving frequencies.

Based on these insights, Soft Surroundings decided to cut its Retargeting budget by 52% in the following months. They reallocated the saved budget to more effective prospecting tactics, such as Facebook advertising. As a result, the company's top-line revenue increased by 17% month-over-month, while their yearly sales comparisons rose by 12%.

Video: Leveraging Incrementality to Value Retargeting vs. Prospecting