Quality Corner | June 2024 | Are your speed checks still catching the bad guys?
Blog | June 26, 2024
By Rich Ratcliff
With our new release of ResponseID, our in-survey fraud detection technology, we have data to shed light on unseen survey fraud in a way that makes us rethink the usual programmatic survey checks.
Over the years, there have been evolutionary moments when the research process and how we protect it have changed. At one point in our evolution, someone said, ‘We need to record how long it takes the participant to take the survey’; thus, the term ‘Speeders’ was born. The idea here was that the data shows as “straight-lined,” an obvious sign of rushing through a series of questions.
After our most recent upgrade to ResponseID, we carved out a subset of 10,000 responses across clients and projects to measure impact. One glaring finding: researchers relying solely on post-survey evaluation will miss significant fraudulent behavior.
Today, Fraudsters are exerting more effort to present open-ended answers that look better. This is slowing down the speed of their exercise and rendering speed checks less effective. ResponseID data reveals that ~5% of responses exhibited clear markers of fraud, such as copy/paste patterns, real-time translation usage, and open development consoles during their survey activities. The verbatims were far from ‘bad content’. These fraudulent responses are designed to look authentic, making them difficult to detect through traditional methods.
One telling metric stems from a comparison of LOI between the fraudsters and the real people. When comparing the response lengths of those who were QC’d versus those who were marked as Qualified Complete, the lengths differed by only 3%.
This minor difference highlights how yesterday’s quality checks may be outdated. Today’s bad actor has learned those old tricks.
Implementing real-time fraud prevention and removing 5% from the completion process can significantly reduce downstream impact. This approach enhances data quality and believability and offers substantial operational benefits for market researchers.
Here’s how:
Time Savings: By identifying and excluding fraudulent responses in real-time, researchers can save significant amounts of time that would otherwise be spent on post-survey fraud detection, correction and replacement.
Improved Data Quality, Clarity and Believability: Removing fraudulent responses results in a cleaner, more reliable dataset.
Operational Efficiency: Reducing fraudulent responses minimizes the need for extensive data cleaning and validation, allowing researchers to focus more on analysis and interpretation.
Cost Reduction: Early fraud detection reduces the resources spent on dealing with fraudulent data, including both financial and opportunity costs.
We often see the QC metric as a singular, simplified number. However, we have grown to see it in a more nuanced way. ResponseID is a valuable link in your data quality chain. It provides in-depth insights into respondent behaviors so researchers can make better decisions faster.