One of the types of qualitative research we’ve carried out the most over these 25 years in the market is the pre-testing of advertising campaigns — whether for films, concepts, or even graphic pieces.
Our analyses and results seem to be on the right track, judging by the volume of work and the steady return of agencies and advertisers for other projects over time.
Starting Point: Focus Groups
Like most players in the market, we typically rely on focus groups, usually segmented by age group, socioeconomic class, and — most of the time — geographic region.
For smaller-scale evaluations, we run an average of around 9 groups. For larger national campaigns, demand often exceeds 20 focus groups.
The use of verbatim quotes with edited audio — capturing participants’ actual statements and positions — was an innovation we developed that added a great deal of information and understanding to the comments and recommendations we present. Beyond boosting persuasive power, these audio clips consistently made testimonials feel real and credible, adding feeling and human nuance to the situations and considerably enriching our presentations.
So much so that, on countless occasions, the interest of agencies and advertisers ended up concentrated almost entirely on the audio clips themselves, given the strength of the messages and information coming directly from the target audience.
The Limits of Focus Groups
Even so, in several of these pre-tests it wasn’t always possible to get a clear, unambiguous answer to questions of a subjective or emotional nature about the campaign concepts.
The overriding impression was that a more adequate, reliable metric was still missing to support these kinds of questions, since focus groups alone weren’t always enough to resolve every doubt — for advertisers and agencies alike.
Among the cases where responses weren’t always satisfactory: the impact of the campaign on those watching the commercial, or the extent to which certain positions were actually perceived as “security,” “trust,” or even what message really got through.
On an exploratory basis, we tried complementing focus group assessments with individual footage of participants, evaluating their reactions through changes in facial expression — but without much success, since the variations were barely noticeable.
We also tried other biometric measurement techniques, such as GSR (galvanic skin response), used abroad to predict box office performance for films and plays — but again with results that fell short of expectations and were hard to interpret for non-specialists.
The Path to Implicit Testing
At international conferences, both ESOMAR’s and NMSBA’s, we’ve consistently looked for new options that could add supporting elements to these questions in a way that’s transparent and clear for agencies and advertisers alike.
This drive at Checon Pesquisa to innovate and diversify our approach — in pursuit of a better understanding of how people react to advertising campaigns — led us to implicit testing.
Implicit association tests were originally developed as a tool to explore the unconscious roots of human thought and emotion, since people often struggle to express their own emotions, or don’t even know for certain what’s going on in their own heads.
These challenges become even more pronounced with lower levels of education, or when people are shy — and social desirability bias (the tendency to give the “politically correct” answer) also shows up fairly often, risking the accuracy of the evaluation.
The case for using implicit association tests in pre-test evaluations rests on their ability to measure responses, ideas and beliefs at a subconscious level, without relying on rationalization that isn’t always “genuine.” What’s more, the results are easy to interpret, with no psychology background required, while still being flexible enough to dig into the specific aspects of each individual campaign.
The Checon Methodology
Under our methodology, implicit tests take place after the focus groups, which serve as a reference point for developing the testing protocol — pinpointing exactly which aspects still haven’t been properly clarified.
We recommend running implicit association tests with the same participant profile, split into a control group (people who haven’t seen the campaign) and a second group who see the material (film, spot, or graphic pieces) right before testing. This makes it possible to set a very clear comparison standard for assessing the impact of aspects that focus groups didn’t fully clarify.
Each implicit association test generally runs 3 to 5 minutes and can assess up to 5 distinct attributes — usually more than enough for the demands we’ve encountered.
We recommend a minimum of 30 tests: 15 typically with the control group, and another 15 with participants who watched the same material presented and tested in the focus groups. This number can be adjusted for specific situations that call for more granular results (by social class or age group, for example), or when comparisons against competitors are also needed.
Advantages of the Method
Among the advantages worth highlighting: needing far fewer focus groups, avoiding the repetition that typically sets in from around the 10th group onward, and focusing effort on measuring very specific aspects.
Another strong point is having a tool that can evaluate beyond the conscious and rational — with metrics that, while not intended to replace quantitative research, complement focus group results.
The result is savings in both processing and analysis time, as well as in overall investment, since implicit association test results are available immediately.
