Re-evaluating Old Norms In An Automated Era
Blog | August 29, 2024
By Terence McCarron
Every time I question some long-accepted concept, it tends to spark some Q&A from our clients. Incidence Rate is by far the most popular.
I wrote a detailed piece suggesting we revisit assumptions on incidence rate (IR). It sparked a debate on LinkedIn about whether this was semantics.
I can tell you right now, it is not semantics. It has a direct impact on MRX project profitability and even quality.
IR in the CATI Era
When I began in this industry, quant was heavily executed using the phone. In that method, the usage of Incidence Rate had a direct link on 2 major cost drivers:
- Call Center hours required to fulfill a study sample needs.
- Estimation of sample records needed to complete the study.
This was truly a cost-plus model and incidence rate was the most important variable.
IR in the Online Access Panel Phase
In the heyday of Online Panels, when response rates were predictably and sustainably high, the costs of sending a few thousand invites for a project were minimal. Even if email costs were expensive, deep respondent profiles made this more efficient.
As a result, IR cost impact was minimized and became a competitive lever for panel companies vying for a project.
IR in the Programmatic Era
Early into the beginnings of OpinionRoute, I played a little game where I tried to recreate the IR Formulas for each survey tool and sample company. I rarely could recreate the survey tools’ approach without cheating. The sample companies were easier to figure out. In short, many weren’t relying on incidence at all. What they often work with is “Conversion Rate”.
This is Not an Academic Exercise
So, why does it really matter? I focus on things like this because the subtle shift to conversion rate means ANY reason for a respondent not completing threatens the CPI. When lower conversion rates mean a higher CPI, this has real gross profit implications.
For example:
- Researcher-reviewed QC removals lower the conversion rate.
- Demographic terminates lower the conversion rate. (We call this a targetable term.)
- Overquotas lower the rate. (even demographic).
MRX’s profitability can suffer. quality is not a priority that’s assured through the economic model.
Wrong Incentives
If we focus on the data quality chain, you must ask how each source in the chain is incentivized. In the programmatic world, the originator of the respondent entering the survey ecosystem is often an affiliate network, loyalty program or mobile app. Here’s how this disconnect plays out in the field.
- MR Firms want: a real, qualified respondent who gives reasonable answers to our questions.
- Programmatic Sample Companies provide: a marketplace for online traffic sources to monetize clicks in surveys.
- Subsources focus on: driving as many clicks as possible.
If the subsources lack a shared goal with the ultimate buyer, is it reasonable to expect them to deliver quality? Also, are we doing this right if poor targeting and bad quality increases the payout?
With OpinionRoute as your partner, we wage this battle on every project for you. We don’t have any illusions of re-inventing the wheel. But we believe in fighting back on things that make projects harder for the Market Researchers. We show up with this on every project.
Want a partner to help make things easier for you? Let’s chat!