Did trust rise, or did the sample change?
Use a consistent comparison before claiming brand progress.
The question
Harbor tracks trust in a fictional service brand. Two independent survey waves ask the same question. The report claims a 15-point improvement. Sampling weights and the customer mix have not yet been provided.
| Respondents | Trust the brand | |
|---|---|---|
| Wave 1 | 400 | 160 |
| Wave 2 | 400 | 220 |
The counts are invented. A change in a tracker does not by itself establish the effect of any brand activity.
“Choose your next action and explain the evidence behind it.”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Describe the raw result
What would you do here? Open the move you’d make.
Report a 15-percentage-point raw rise.
The strong move. A percentage-point change compares rates directly.
Call it a 15% relative increase.
Tempting, but it misses something. The relative increase is 15/40 = 37.5%; 15 describes percentage points.
See how a strong candidate says it
“The raw trust rate rises from 160/400 = 40% to 220/400 = 55%, a rise of 15 percentage points. I would call this an unadjusted result.”
Why it works: A percentage-point change compares rates directly.
Step 2: Check comparability
What would you do here? Open the move you’d make.
Check the sample mix and weighting.
The strong move. Different respondents can change the total without a change within either group.
Accept comparability because both samples contain 400 people.
Tempting, but it misses something. Equal totals say nothing about the proportion of existing customers in each wave.
See how a strong candidate says it
“Before calling this brand progress, I would ask whether the customer mix, sampling method, question and fieldwork period are comparable. Equal sample sizes do not make the samples equivalent.”
Why it works: Different respondents can change the total without a change within either group.
Step 3: Use the same mix
Wave 1 contains 200 existing customers, of whom 140 trust the brand, and 200 noncustomers, of whom 20 trust it. Wave 2 contains 300 existing customers, of whom 210 trust it, and 100 noncustomers, of whom 10 trust it.
What would you do here? Open the move you’d make.
Show raw and fixed-mix results together.
The strong move. The reported total changed because the sample composition changed.
Claim that trust rose within both groups.
Tempting, but it misses something. Neither group’s rate rose; the customer share changed from half to three quarters.
See how a strong candidate says it
“Existing-customer trust is 70% in both waves; noncustomer trust is 10% in both. At a 50/50 mix, wave 2 is 40%, the same as wave 1. The raw rise is explained by the larger customer share.”
Why it works: The reported total changed because the sample composition changed.
Step 4: Keep the longer view honest
What would you do here? Open the move you’d make.
Use stable tracking and separate commercial outcomes.
The strong move. A consistent measure supports learning without inventing a financial result.
Translate the 15-point headline into brand value growth.
Tempting, but it misses something. No valuation model or attributable cash-flow change is supplied.
See how a strong candidate says it
“I would keep the audience definition and weighting stable, repeat the measure, and compare it with actual customer outcomes. One tracker cannot establish brand valuation or the return on the brand budget.”
Why it works: A consistent measure supports learning without inventing a financial result.
The answer, in short
- Decision
- Withdraw the claim of a comparable trust improvement.
- Because
- Both segment rates are unchanged; the fixed-mix rate remains 40%.
- What would change it
- A later comparable wave showing within-group gains could support a progress claim.
Common traps
- Confusing points with relative percent.
- Letting audience mix masquerade as brand progress.
Check your answer
- I calculated both raw rates.
- I asked about the sample.
- I reweighted to a fixed mix.
- I avoided a valuation claim.
Last reviewed Nov 10, 2026