Here’s a very short horror story: Overdose deaths increased 50% last year.
That’s it. That’s the horror story.
Is that bad? Probably? Should we be concerned? Maybe? Should someone alert the media?
We have absolutely no idea.
Overdose deaths could have increased from two to three. Or from 2,000 to 3,000. We don’t even know who or where we’re talking about yet.
And yet, give any of us a statistic with a percentage sign attached to it and we’re remarkably willing to accept that it means something.
When it comes to interpreting data, that’s where things can go sideways quickly. A number can be completely accurate and still leave us with the wrong impression.
Which brings us to one of the least exciting questions in data analysis and one of the most useful: What’s the denominator?
Why Denominators Matter When Interpreting Data
Most of us remember denominators from elementary school fractions, where they were primarily useful for determining how much pizza our imaginary classmates ate.
In real-world data, the denominator tells us what our number is actually relative to.
Say County A recorded 120 overdose deaths last year, while County B recorded 60.
County A has the bigger problem, right? Not necessarily.
If County A has 500,000 residents and County B has 50,000, those raw numbers suddenly tell a very different story. County A still experienced more overdose deaths, but relative to population size, the burden in County B is much greater.
The count wasn’t wrong. Our interpretation was.
That doesn’t mean the count doesn’t matter. County A still had twice as many deaths, which may be extremely important if you’re planning services or trying to understand how many people and families are affected.
The rate answers a different question about the relative burden on each community.
Neither is the “right” number. They’re just telling you different things.
“30%” Is Not a Complete Sentence
Percentages are particularly good at sounding authoritative: Thirty percent of people reported difficulty accessing healthcare.
Okay.
Thirty percent of whom? All residents? Adults? Patients? People who answered a community survey? The 10 people who showed up to Tuesday night’s meeting?
Those are very different findings.
If 30% of survey participants report difficulty accessing healthcare, that’s what we know. Depending on how the survey was conducted and who participated, we may not be able to say that 30% of the entire community has that experience.
As Pew Research Center explains, defining the population represented in survey data is essential to interpreting the results accurately.
“Thirty percent of survey participants” might sound less dramatic than “30% of residents.” But it is considerably more useful if it’s true.
Small Numbers Can Make Big Drama
Denominators get especially interesting when they’re small.
Imagine the number of people experiencing a particular outcome increases from four to eight.
That’s a 100% increase.
Technically correct. Also, the kind of statistic that deserves some context before anyone puts “100% INCREASE” in 36-point red font on a PowerPoint slide or on a news banner.
Small numbers can produce enormous percentage changes. That doesn’t make the change unimportant. It just means we need the underlying numbers to understand what happened.
The same caution applies to small survey subgroups, small communities, rare events, and population estimates. A percentage can look remarkably respectable while being based on very few actual people.
Ask the Annoying Question when Interpreting Data
This is why good analysts spend an unreasonable amount of time reading footnotes, methodology notes, and table definitions apparently designed to test the limits of human attention.
That’s often where the denominator is hiding.
You don’t need to be a statistician to get better at interpreting data. You just need to become the person who asks slightly annoying questions:
Out of how many?
Percent of whom?
From what to what?
Are these two numbers actually measuring the same thing?
Those questions won’t make every dataset easy to interpret. But they’ll prevent a surprising number of bad conclusions. Because good data storytelling isn’t about finding the biggest number, the most dramatic percentage, or the scariest trend.
It’s about understanding what the number actually says before deciding what story it tells. And sometimes, unfortunately for all of us, that means checking the denominator.
At Crescendo, we know that good research is about more than collecting data. It’s about understanding what the numbers mean and what they can tell us about a community. Whether you need a comprehensive Community Needs Assessment or more targeted Population Health and Market Research, we can help you make sense of the data. Contact us to learn more.