Why Google Ads Brand Holdout Tests Answer the Wrong Question

A Google Ads brand holdout can accurately measure the overall incremental impact of branded search advertising while concealing important differences between navigational and consideration-oriented searches. To make financially sound budget decisions, advertisers must design experiments around specific, isolatable advertising treatments, account for automation and statistical uncertainty, and evaluate incremental revenue against contribution profit rather than relying on a single blended lift estimate.

Why Can a Single Brand Holdout Hide Important Differences?

A standard brand holdout estimates the overall incremental impact of branded search advertising. But that average can hide important differences between types of brand searches. An advertiser may need to measure those differences separately to understand what the overall result actually means.

Consider a retailer testing whether its Google Ads brand campaigns generate incremental sales. The team selects matched geographic markets, pauses brand ads in the test markets, and compares sales against markets where advertising continues.

The experiment produces one overall result. But that result combines searches with very different customer intentions.

Someone searching for "BrandName returns" probably wants the retailer's returns page. If the paid ad disappears, that person may simply click the organic listing. Someone searching for "BrandName vs Competitor" is still evaluating options. If a competing advertiser appears above the retailer, removing the brand ad could change where that customer goes.

Consider these hypothetical results from separate experiments that successfully isolate two search conditions:

The combined result shows 210 fewer sales when brand ads are removed. But the separate results tell different stories. Navigational searches show little change, while consideration-oriented searches show a much larger decline.

The 10.5% average is not necessarily wrong. It's incomplete if the advertiser needs to understand how the effect differs across search conditions.

A standard geographic holdout would not automatically produce the two separate estimates. Measuring them requires an experiment that can isolate the relevant advertising changes and observe their effects on sales.

I saw this play out with a B2B SaaS client spending roughly $25,000 a month on Google Ads. Their finance team assumed the brand campaign was cannibalizing organic traffic and wanted to turn it off. We ran a segmented geographic holdout instead of a blended one, separating navigational queries such as login and customer service from consideration queries such as pricing, reviews, and comparisons against competitors.

The two segments behaved differently. On navigational queries, organic results absorbed about 95% of the traffic when the ads disappeared, so incrementality was close to zero. On consideration queries, competitors took the top sponsored positions once we paused our ads, and demo requests in the holdout markets dropped 32%. A single blended test would have averaged those two results together. We stopped bidding on navigational terms, kept high impression share targets on consideration terms, and moved the roughly $5,000 a month we stopped spending on navigational terms into non-brand campaigns. We did not see a measurable loss in leads, and the account's overall cost per acquisition was 12% lower the following quarter. It is one account, so read it as an example rather than a benchmark.

What the Research Tells Us About Organic Substitution

When search ads disappear, some paid clicks move to organic results. Others disappear entirely. Research shows that the amount of organic substitution can differ substantially across search advertising conditions.

Haus, an incrementality measurement vendor, published an analysis of its own geographic experiments in September 2025. Every experiment ran on Google text-ad campaigns that targeted only brand-name keywords. Haus defined high competition as three or more competitors with more than 50% combined impression share. Among brands facing high competition, 82% measured significant lift, compared with 35% of brands facing low competition.

Haus also reported incrementality factors, which compare the sales an ad drives with the sales the platform reports. The average was 0.37 for high-competition brands and 0.09 for low-competition brands. Despite higher cost per click, high-competition brands saw 47% higher incremental return on ad spend on average.

Haus counted lift as significant when the probability of an effect above zero was greater than 80%. That is a lower threshold than the 95% level commonly used in statistical testing, and the analysis does not say how many experiments or brands it covers. It is vendor analysis, and it appears to cover ecommerce brands rather than B2B advertisers.

Measured, another incrementality measurement vendor, shared a related figure in a November 2025 webinar it held with Revvim. Jarah Burke of Measured said that on average, 19 of every 100 conversions reported for Google brand search were incremental, and that she most often sees 4% or 5% in enterprise fashion accounts. The webinar did not explain how the 19% average was calculated, so it is a vendor average and not a benchmark.

Click incrementality is not the same as sales incrementality. A customer switching from a paid result to an organic result may still make the same purchase. That's why the holdout needs to measure sales, not just clicks.

The studies measured different advertiser populations and advertising conditions. They should not be treated as directly comparable estimates of brand-search performance. Neither result establishes how much incremental revenue a particular brand campaign generates.

What they demonstrate is that paid-to-organic substitution cannot be assumed to work the same way everywhere.

For a brand holdout, that creates an important measurement problem. A search for "BrandName returns" and a search for "BrandName vs Competitor" may respond differently when ads disappear, especially if another advertiser is competing for the second search.

How to Identify Brand Search Conditions Before Testing

Review branded search queries by customer intent and assess competitor presence separately. This helps identify which conditions may need distinct measurement, even when Google Ads cannot isolate them perfectly.

Start with the Search terms report in Google Ads. Look for queries containing words like "login," "returns," "reviews," or "vs," and group them by intent. Then examine the organic results and whether competitors or resellers appear for representative searches.

This review has limitations. Google Ads does not report every search term, keyword matching can include queries outside the intended groups, and competitor presence changes over time. Auction Insights can identify advertisers that overlap with your campaigns, but it cannot show exactly which individual searches triggered their ads.

More importantly, identifying a search condition does not mean it can be measured separately in a holdout. A query category must be sufficiently isolatable through the campaign setup, and the experiment still needs enough sales volume to estimate its effect. Competitor presence is especially difficult to isolate because it is not a stable keyword-targeting category.

Before designing the holdout, the advertiser needs to decide whether an overall brand incrementality estimate will answer the business question or whether specific search conditions need separate measurement.

For a closer look at how changing competitive conditions affect branded search bidding decisions, see How to Unlock the Trapped Capital in Your Branded Search Budget.

How Do You Design a Holdout That Can Detect an Effect Worth Acting On?

A brand holdout must be designed around the specific advertising decision it needs to support. If the goal is to measure differences between branded search conditions, the advertiser must first determine whether those conditions can be isolated, then establish the financial effect the experiment needs to detect and whether the available markets can support it.

A retailer trying to determine whether its entire brand campaign generates incremental sales needs an overall brand holdout. A retailer trying to determine whether navigational searches justify their advertising cost needs an experiment that isolates the advertising associated with those searches.

The first experiment cannot answer the second question, regardless of how precisely its overall effect is measured.

Determine Whether the Search Condition Can Be Isolated

A segment-level holdout requires a defined advertising treatment that can be applied without materially changing other search conditions. Identifying a query group in Google Ads reporting is not enough to establish that the group can be tested independently.

Start with the navigational and consideration-oriented query groups from your Search terms review.

The paid search team needs to determine whether navigational or consideration-oriented searches can be separated through campaign structure, keyword coverage, and exclusions.

That requires checking more than the keywords assigned to each campaign. Broad match and close variants can cause queries to appear outside their intended groups. Other Search campaigns and Performance Max campaigns may also serve branded searches that the experiment is supposed to suppress.

Google gives an exact match keyword in a Search campaign priority over Performance Max for the same query, but if that Search campaign is budget-limited, Performance Max can occasionally serve on exact match terms. Brand exclusions are the control that stops a Performance Max campaign from serving on brand queries, and they also block common misspellings and related subsidiary brands, which negative keywords do not do automatically.

Before committing to a segmented holdout, the team should review Search terms reporting, campaign overlap, negative keyword coverage, and the actual ad delivery associated with the proposed treatment.

Competitor presence creates an additional limitation. A competitor appearing in a branded auction is not a stable keyword category that can be reliably isolated through campaign targeting. Auction Insights can help identify overlapping advertisers, but it cannot turn competitor-present searches into a clean experimental treatment.

A holdout that removes brand ads measures the effect of removing that advertising. It does not establish what would happen if the advertiser maintained coverage at a lower bid; that would require testing a different treatment.

If the advertiser cannot isolate the intended advertising exposure, the experiment cannot credibly estimate that segment's independent effect.

The business can still run an overall brand holdout. It simply needs to recognize that the result will apply to the combined advertising conditions tested.

Set the Minimum Detectable Effect Around the Segment Decision

The minimum detectable effect (MDE) is the smallest advertising impact an experiment is designed to detect reliably. For a segmented brand holdout, the MDE should reflect the financial consequences of changing spend on that specific search condition, not the entire brand campaign.

A hypothetical retailer is evaluating a $20,000 monthly reduction in advertising on a separately isolated group of navigational brand searches.

At a 40% contribution margin before advertising costs, the retailer could lose $50,000 in monthly revenue before the lost contribution profit offsets the $20,000 in reduced ad spend.

$20,000 in reduced ad spend ÷ 40% contribution margin = $50,000 in break-even revenue loss.

That gives the experiment a financial decision threshold.

The retailer might choose a hypothetical MDE of $25,000 in monthly revenue to target sensitivity below the break-even point.

The experiment must measure the effect of reducing the navigational advertising under consideration. Pausing every brand campaign would introduce consideration-oriented and other branded searches into the treatment, producing an overall result that cannot isolate the navigational effect.

The analytics team should also calculate statistical power using historical sales data. Statistical power measures how reliably the proposed experiment can detect an effect of a specified size against normal fluctuations in revenue.

For a geographic brand holdout, analytics can use historical market-level sales patterns to estimate variability, then simulate the planned treatment and control assignments with an assumed revenue impact equal to the MDE. The proportion of simulated experiments that detect that impact at the chosen significance level provides an estimate of statistical power.

A common planning target is 80% power. For example, if a proposed design detects the assumed $25,000 revenue effect in 800 out of 1,000 simulations, its estimated power is 80%.

If the proposed design falls short, the team may need more eligible markets, a longer test, or a larger detectable effect. The calculation determines whether the experiment is sensitive enough to answer the financial question before advertising is withheld.

Match Markets and Assess Whether the Segment Has Enough Volume

A segmented geographic holdout needs suitable comparison markets and enough sales volume to detect the effect that matters. Narrowing the advertising treatment may make the question more specific, but it does not automatically make the experiment easier to power.

The analytics team should examine historical geographic sales patterns to identify markets that move similarly over time, rather than matching markets on revenue size alone.

Using historical data, the team can evaluate matched market assignments, expected sales variation, and the number of markets and weeks required to detect the chosen MDE.

For a segmented experiment, there is another constraint: the proposed advertising treatment may account for only a fraction of total brand activity.

Navigational searches may have more consistent customer behavior, but isolating them also reduces the advertising exposure being changed. A smaller expected effect can be harder to distinguish from normal variation in total sales.

The team therefore needs to assess power for the specific segment-level treatment, not reuse the feasibility assumptions from an overall brand holdout.

Where available, Google's geo-based Conversion Lift feasibility assessment can provide an additional planning check, including its minimum detectable incremental return on ad spend (iROAS), defined as incremental revenue divided by incremental advertising spend.

If the segment cannot support the required sensitivity, the advertiser may need more eligible markets, a longer test, or a broader treatment. But broadening the treatment changes the question the experiment answers.

Brand Holdout Design Worksheet

Before launch, marketing, analytics, and finance should confirm that the advertising treatment matches the business question and that the experiment can detect a financially meaningful effect.

The paid search team owns treatment feasibility and campaign coverage. Analytics owns market selection, power calculations, and experimental validity. Finance should approve the financial threshold and acceptable exposure before advertising is withheld.

If the intended search condition cannot be isolated or the experiment cannot detect a meaningful effect, the team should acknowledge that limitation before launch rather than interpret an inconclusive result as evidence that the segment has no incremental value.

What Has to Be Frozen Before Day One, and What If Your Automation Will Not Freeze?

A segmented brand holdout is only credible if the intended advertising difference remains intact throughout the experiment. Campaign overlap, automated bidding, shared budgets, and unrelated account changes can alter which branded searches receive ads, making it difficult to attribute sales differences to the search condition being tested.

Isolating navigational or consideration-oriented searches on paper is not enough. The paid search team must verify that the intended advertising difference survives actual campaign delivery.

A retailer is testing whether ads on navigational brand searches generate incremental sales. The team suppresses those ads in selected markets while leaving consideration-oriented brand advertising active.

If another Search campaign begins serving the excluded queries, the holdout no longer represents a clean reduction in navigational advertising. If a shared budget reallocates spend toward the control markets, the experiment may also change advertising pressure outside the intended treatment.

In both cases, the test may produce a statistically precise result for an advertising change the business never intended to measure.

Validate the Comparison Before Launch

An A/A test checks whether the proposed treatment and control markets produce an effect consistent with zero when no advertising change has been applied. It can reveal problems with geographic comparability or measurement before the holdout begins.

Analytics should assign the planned market groups, keep advertising conditions unchanged, and run the intended measurement method over a pre-treatment validation period.

A large or persistent estimated difference deserves investigation. The team should check geographic sales attribution, promotions, historical market trends, and campaign coverage.

An A/A result near zero is not proof of validity. The estimated effect must also be interpreted alongside its uncertainty, and the test does not establish that the intended query group will remain isolated once advertising is suppressed.

For a segmented brand holdout, the paid search team should therefore validate both the geographic comparison and the campaign-level treatment.

Lock the Advertising Treatment and Monitor Delivery

The preflight review should prevent unrelated changes from affecting the experiment and confirm that excluded branded searches do not continue receiving ads through overlapping campaigns. Treatment integrity must be monitored after launch, not assumed from campaign settings.

The paid search lead should use Google Ads Change history to audit account changes and review auto-applied recommendations under the account's Recommendations and Auto-apply settings.

During the test, monitor actual ad delivery in both market groups, including spend, impressions, campaign coverage, and available search-term evidence. Search terms reporting omits some queries, so it cannot prove perfect isolation. Google leaves out queries without enough activity to meet its privacy standards, and search terms insights group those queries into categories without showing them, so a clean-looking report cannot confirm that every excluded query stayed dark. Unexpected delivery should trigger an investigation rather than an automatic assumption that the experiment remains valid.

Decide What Happens When Automation Reacts

Automated systems can run during a brand holdout as long as their response does not change the intended advertising treatment or cause changes in one geographic group to affect another.

A hypothetical retailer uses a script to manage a shared budget across markets.

When navigational brand ads are paused in the treatment markets, the script detects lower spending and reallocates budget toward the control markets. The experiment now measures more than the absence of navigational ads: it also includes increased advertising pressure elsewhere.

Google's guidance on implementing geo experiments describes how budget constraints can create these cross-market effects. In budget-constrained campaigns, removing the test geos triggers immediate budget shifting, and the system redistributes the unspent funds to the control regions and inflates the baseline. Google recommends individual campaign budgets instead of shared or portfolio budgets, and says shared budgets and portfolio bidding should never be used for go-dark tests. The resulting problem is not simply additional statistical noise. The estimated advertising effect may be biased because the groups no longer operate independently.

Google defines iROAS for Conversion Lift based on geography as incremental conversion value divided by incremental cost, so the denominator should be the spend that actually changed. If automation moved budget into control markets, measure that change rather than assuming the planned pause was the whole treatment.

The team needs to decide which question it is testing.

If the objective is navigational brand incrementality, prevent reactive rules, shared budgets, or other systems from changing the treatment and control conditions in unintended ways. This may require separating budgets, suspending selected rules, or using predetermined settings during the test. Google's guidance on implementing geo experiments includes setups for Smart Bidding campaigns, so Smart Bidding does not necessarily need to be disabled, but its interaction with the experimental design must be assessed.

If the reactive management policy cannot be separated from the advertising change, the experiment may instead evaluate that broader policy, provided the alternative policies are explicitly defined and assigned. The result would not isolate the incremental value of navigational brand ads.

A test that changes both query-level advertising exposure and budget allocation cannot support a clean conclusion about the query segment alone.

Set the Reporting Cooldown Before Launch

The experiment's results should not be finalized until enough time has passed for test-period conversions and revenue to mature. The cooldown should reflect actual conversion and offline-import delays rather than a fixed waiting period.

Analytics should review historical conversion lag and any delays from customer relationship management (CRM) imports or revenue reconciliation. The team should select its primary sales outcome before launch and document differences between Google Ads, Google Analytics 4 (GA4), and transaction-system reporting.

Advertising can return to its normal operating state while the team waits for test-period outcomes to mature. The reporting cooldown does not necessarily require extending the advertising pause. Google's guidance on implementing geo experiments includes cooldown protocols for after the intervention, and Conversion Lift based on geography reports a cooldown date range with its results.

A segmented holdout only answers a segment-level question when the intended search condition remains the actual treatment. If campaign overlap or automation changes what was withheld, the team must qualify the result rather than present it as clean evidence of that segment's incremental value.

How Do You Read the Result Without Fooling Yourself?

A brand holdout should be interpreted against the specific advertising decision it was designed to answer. Incremental revenue, statistical uncertainty, and contribution profit determine whether the tested advertising was economically valuable; segment-level conclusions require separate, valid evidence for each search condition.

A statistically significant overall brand result can still leave the original segmentation problem unresolved.

If navigational and consideration-oriented searches were tested together, the experiment estimates their combined effect under the tested conditions. It cannot establish whether one group generated most of the incremental revenue or whether both justified their advertising cost.

That distinction should determine how the results are reported.

Compare Confidence Intervals With the Financial Threshold

A confidence interval expresses uncertainty around the estimated advertising effect. The business should compare that interval with the financial break-even point, not judge the result solely by whether the interval includes zero.

Return to the hypothetical retailer with a 40% contribution margin before advertising costs.

For every $1 spent on advertising, the retailer needs $2.50 in incremental revenue to cover that cost:

$1 ÷ 40% = 2.5 break-even incremental return on ad spend (iROAS).

Here, iROAS means experimentally estimated incremental revenue divided by incremental advertising spend, not the return on ad spend reported through Google Ads attribution.

An estimated iROAS above 2.5 suggests that the tested advertising generated enough contribution profit to cover its cost. An estimate below 2.5 suggests the opposite. But the confidence interval determines how much certainty the experiment provides around that conclusion.

A result with a point estimate of 2.0 and an interval spanning 1.2 to 2.8 would not resolve the profitability question. The interval includes outcomes on both sides of break-even.

Likewise, a result that is not statistically distinguishable from zero does not automatically prove the advertising has no value. If the interval is wide enough to include financially meaningful positive effects, the test remains inconclusive.

The analytics team should therefore report both the estimated effect and whether the uncertainty interval supports the decision threshold established before launch.

Read the Two Search Segments as Two Different Decisions

Separately validated experiments can reveal materially different incremental returns across branded search conditions. A navigational segment may fail to justify its advertising cost while a consideration-oriented segment generates sufficient incremental revenue to support continued investment.

A hypothetical retailer has isolated navigational and consideration-oriented brand advertising and run valid experiments for each treatment.

The iROAS calculation is straightforward:

Incremental revenue ÷ incremental advertising spend = iROAS.

For navigational searches, the experiment estimates $2,000 in incremental revenue from $5,000 in spend, producing 0.4 iROAS.

The illustrative confidence interval ranges from −0.2 to 1.0. Although the interval includes zero, its upper bound remains below the retailer's 2.5 break-even threshold.

Under the assumed valid experimental design, the evidence supports the conclusion that the tested navigational advertising did not generate enough incremental contribution profit to justify its cost.

For consideration-oriented searches, the experiment estimates $20,000 in incremental revenue from the same $5,000 in spend, producing 4.0 iROAS.

Its illustrative interval ranges from 3.0 to 5.0, entirely above the 2.5 break-even threshold. That supports maintaining the tested advertising exposure under the retailer's assumed economics.

The two experiments therefore support different decisions.

The retailer has evidence to reconsider the tested navigational advertising while maintaining the tested consideration-oriented advertising. Neither result automatically identifies the optimal bid or budget, because a holdout measures the effect of a defined advertising change rather than every possible spending level.

And these conclusions depend on the experiments actually isolating the two treatments. A blended brand holdout cannot be divided into equivalent segment-level causal estimates afterward using Search terms reporting.

Distinguish an Inconclusive Result From Evidence of a Small Effect

A holdout that fails to detect a statistically significant effect may still be too imprecise to support a budget decision. The relevant question is whether the confidence interval rules out effects large enough to change contribution profit.

The navigational example illustrates an important distinction.

Its iROAS interval includes zero, so the experiment does not establish a positive incremental revenue effect. But the interval's upper bound is still below financial break-even. That makes the result useful for the tested budget decision despite the absence of a statistically clear positive lift.

Suppose the navigational interval were much wider, for example, −0.2 to 4.0.

That hypothetical interval would include both unprofitable and profitable outcomes. The same 0.4 point estimate would no longer justify a confident financial conclusion.

This is why the MDE, power analysis, and test duration from the design stage matter. An experiment that cannot distinguish financially meaningful effects from normal sales variation should not be reported as proof that advertising is nonincremental.

A properly powered experiment can still be inconclusive. The final interpretation must reflect the uncertainty actually observed.

Use Synthetic Control When Geographic Comparisons Need Improvement

Synthetic control methods can construct a counterfactual from a weighted combination of comparison markets when no individual market provides a suitable match. They can improve geographic measurement, but they cannot recover search-segment effects that were never separately tested.

The analytics team should assess whether the synthetic comparison reproduces historical sales patterns before the advertising change and whether its uncertainty estimates are appropriate for the experimental design.

Google's Meridian GeoX is an open-source library for geographic experiments that uses time-based regression. The availability of more sophisticated measurement tools does not remove the need to validate the treatment, account for geographic interference, or distinguish overall brand effects from segment-specific effects.

The final report should identify the search condition tested, estimated incremental revenue, incremental advertising spend, iROAS and its uncertainty interval, and the contribution-profit decision supported by the evidence.

How Do You Turn One Brand Holdout Into a Standing Measurement Program?

A brand holdout measures incrementality under a specific set of advertising conditions, not permanently. A standing measurement program preserves the scope of each result, updates estimates when search behavior or campaign conditions change, and gives marketing, analytics, and finance a consistent basis for allocating brand spend.

If the business conducts one overall brand holdout, it has one overall incrementality estimate. If it conducts valid experiments on separately isolated search conditions, it may have evidence for different budget decisions across those conditions.

The measurement program should preserve those differences rather than collapse them into a single number.

Maintain Versioned Incrementality Factors by Measured Scope

An incrementality factor compares experimentally estimated incremental revenue with platform-attributed revenue for the same advertising scope. The factor can inform performance reporting between experiments, but it must not be applied to search segments the experiment did not separately measure.

Suppose a hypothetical brand holdout estimates $200,000 in incremental revenue while Google Ads attributes $500,000 in revenue to the tested campaigns.

The estimated incrementality factor is:

$200,000 ÷ $500,000 = 0.40.

If Google Ads subsequently reports a 5.0 return on ad spend (ROAS) for comparable campaigns, the business could apply that factor to estimate:

5.0 platform ROAS × 0.40 = 2.0 incrementality-adjusted ROAS.

Its validity depends on consistent revenue definitions, campaign scope, spend denominators, and advertising conditions.

The segmentation constraint is critical.

A factor estimated from an overall brand holdout applies to the combined advertising treatment tested. It cannot establish separate incrementality factors for navigational and consideration-oriented searches.

If the retailer has independently measured those search conditions, analytics can maintain separate factors for each. Otherwise, the business should retain the blended factor and explicitly document the segment-level questions it cannot answer.

Each approved factor should have a versioned record containing:

New experiments should create new factor versions rather than overwrite previous estimates. Historical reporting can then be reconciled to the assumptions used at the time.

Retest When the Advertising Conditions or Financial Stakes Change

Incrementality estimates should be reviewed when the advertising conditions they describe materially change. Retesting frequency should reflect spend exposure, conversion volume, geographic feasibility, and the risk of making decisions with outdated evidence.

Branded search conditions do not remain fixed.

A competitor may become more active in important auctions. Broad match expansion or campaign restructuring may change which queries receive ads. Performance Max coverage may overlap with previously isolated brand campaigns. Customer behavior and organic search visibility can also shift.

Any of those changes can weaken the case for applying an older experiment to current spending.

The analytics and paid search teams should therefore review the validity of existing factors when the query mix, auction conditions, campaign structure, or conversion measurement changes materially.

An annual measurement plan can help organize that work, but there is no universal testing frequency:

For repeat geo experiments, analytics should reassess historical market comparability and rotate eligible markets where doing so preserves experimental quality. Rotation should not take priority over statistical power or treatment integrity.

Retest to find out whether the evidence supporting current budget allocations still applies.

Give Finance and Marketing Clear Decision Rights

Platform-attributed ROAS and experimental iROAS answer different questions. A standing measurement program should preserve both metrics while making incremental contribution profit, not attributed revenue alone, the basis for major brand budget decisions.

Google Ads reports revenue attributed to advertising under its configured measurement rules. A holdout estimates how much revenue changed because the tested advertising was present.

Both figures can be legitimate even when they differ substantially.

Leadership should see a consistent bridge between platform ROAS, experimentally measured iROAS, and any incrementality-adjusted estimate used between tests. Analytics must also document differences between Google Ads, GA4, and transaction or CRM data rather than applying factors across incompatible revenue definitions.

Ownership should be explicit:

A new factor should be approved when credible evidence supports a change, not simply because platform ROAS moved or a stakeholder prefers a different estimate.

When evidence is inconclusive, finance can evaluate a range of potential outcomes rather than treating the point estimate as certain.

Most importantly, the organization must preserve the boundary between what it measured and what it inferred.

One blended brand holdout can support an overall brand budget decision. Separately validated segment experiments can support different decisions for navigational and consideration-oriented searches. A standing measurement program keeps that distinction intact as budgets, campaigns, and auction conditions change.

Frequently Asked Questions

Why does a brand holdout test often show almost no lift?

A brand holdout may show little incremental lift because customers who already intend to visit the advertiser's website switch from paid ads to organic results when advertising disappears. However, a small estimated effect can also reflect insufficient statistical power. The confidence interval determines whether the result supports a conclusion of limited incrementality or remains inconclusive.

How much spend do I have to put at risk to get a readable result?

There is no universal minimum spend requirement for a brand holdout. The necessary advertising exposure depends on historical sales variability, the minimum detectable effect, available geographic markets, and test duration. Analytics should calculate statistical power before launch, while finance evaluates whether the potential revenue loss from withholding advertising is acceptable.

Can I run a holdout test while automated bidding tools are active?

Yes, with conditions. Automated bidding can remain active if its behavior does not compromise the intended treatment and control conditions. The paid search team must monitor Smart Bidding, shared budgets, scripts, and overlapping Search or Performance Max campaigns for unintended changes in delivery. If automation materially alters the treatment, the experiment may no longer isolate the advertising effect being tested.

Ready to see if you’re overpaying for branded search?

Request a demo to start the risk-free, 30-day AdAi trial to quantify your trapped capital and reclaim budget to fund your growth campaigns.