Knowing what we are doing: The case for growth focused creative pre-testing
Creative pre-testing takes a fraction of the budget for an ad campaign to test whether it will positively effect sales before it is put out into the world. As a marketer it means you can have some reassurance you are investing in marketing, not just spending on it. Sounds simple and sensible doesn’t it? So why are so many brands ignoring it?
“If we knew what we were doing it wouldn’t be called research.” - Albert Einstein
TESTING TIMES
Creative pre-testing takes a fraction of the budget for an ad campaign to test whether it will positively effect sales before it is put out into the world. As a marketer it means you can have some reassurance you are investing in marketing, not just spending on it. Sounds simple and sensible doesn’t it?
Pre-testing became prominent in the 1960’s alongside TV ads and Stephen King of JWTs ADMAP article¹ that advanced thinking in pre-testing that endures to this day. Since then, continuous revisions of the practice have endured and all-manner of methodologies for testing ads, including brain scanning and facial coding, mean its potential could be greater than ever.
But despite all this innovation, it is still divisive. After all, shouldn’t we, as experienced marketers be able to just look at an ad and
‘gut-feel’ it? Prof. Kennedy of the Ehrenberg-Bass Institute for Marketing Science decided to test that assertion². Our intuition, it turns out, is not particularly reliable.
Professor Rachel Kennedy, Director, Product Development at the Ehrenberg-Bass Institute for Marketing Science
Over 700 marketers assessed two ads; one a proven winner, and the other a proven loser – the research showed that marketers could not, in fact, “gut feel” it at all. Proof if it were ever needed that accountability needs rigour, not just experience.
ACCOUNTABILITY
In 2007 Les Binet and Peter Field helped change the view of marketers who believed that marketing was a purely instinctive endeavour with their hugely influential paper “Marketing in the Era of Accountability”³. It empirically identified the marketing practices that increase profitability; encouraging the marketing world to add more science to the ‘art’.
What then, was their advice on pre-testing? Surely if going on gut feel is the opposite of accountability, their paper should have recommended it strongly.
Binet and Field – Marketing in the Era of Accountability
Surprisingly, it didn’t – on every measure, where advertising was positively pre-tested, business effects were less pronounced; pre-testing seemed to somehow hamper the effectiveness of campaigns.
This was incendiary, and Binet and Field didn’t stop there. In their 2013 paper “The Long and Short of It”⁴, new data re-enforced pre-tested campaigns as less likely to achieve long term business success. The evidence was stacking up.
So why wasn’t pre-testing working?
METHOD IN THE MADNESS
To be fair to Binet and Field, they always alluded to the fact that research wasn’t entirely worthless, but that the methodologies were flawed. Particularly in setting objectives, and in reference to the low-attention processing model of Robert Heath’s work⁵ - how advertising works over the long term.
Binet and Field – Marketing in the Era of Accountability
In “The Long and Short of it” they offered deeper insight; for short term campaigns of less than 6 months, pre-testing did in fact have a positive effect (+10%), but this effectiveness became negative (-10%) for campaigns lasting 2+ years.
Pre-testing seemed to work really well for short term campaigns, and poorly for campaigns focused on long term effectiveness.
Alarmed by these findings, leading market research companies set out to fix things. As it turned out, persuasion-led creative pre-testing actually discriminated against ads that had more long-term effectiveness potential.⁷
Orlando Wood – System¹
Some ads simply didn’t stand a chance in traditional pre-testing, because it was testing if they could generate quick sales, not long-term effectiveness.⁷ We needed a new methodology for long-term growth.
TEST MATCH
Since then, the majority of research agencies have revolutionised their approach to pre-testing, allowing them to assess short term persuasiveness alongside long term recall and emotion to give better predictability from testing.
The IPSOS Equity Effect Index⁵, Kantar Millward Brown’s investment in neuroscience⁸ and System1’s much touted star rating⁹ for emotional response have all dramatically improved predictive measures for pre-testing in recent years.
But if we have learned anything from the chequered history of creative pre-testing innovation, it is that new methods don’t solve the problem alone. The most important element is ensuring we match the right kind of testing, to the objectives of the brief.
Darren Poole – Millward Brown
To that end, we need solid, evidence-based briefs with clear business objectives - so we know what an ad is supposed to achieve, and that needs to be shared collaboratively between agency, client and the research agency to ensure the research agency can tailor the study outputs and de-brief.
As the old saying goes – rubbish in, rubbish out.
TESTING, TESTING, 1,2,3
Some advice for creative pre- testing then:
Avoid just “gut feeling” an ad. It’s gambling with your business’s money. Testing gives us accountability for creative decisions.
Creative pre-testing does support growth, but it needs the right methodology aligned to the objective of the campaign. Persuasion for short-term results, and emotion and recall for effective long-term growth.
Tight, business objective-led briefs and strong collaboration ensures every partner is working to the same goal and that the right tests, outputs and debriefs are developed.
Einstein knew even he couldn’t predict outcomes. If you are serious about creativity for growth, it’s time for science to come to the art.
Ringo Moss, Chief Strategy Officer, McCann Demand
Sources
1. Stephen King: “Can research evaluate the creative content of advertising?” - ADMAP 1967.
2. Rachel Kennedy: “Ehrenberg-Bass on the science of pre-testing campaigns” - WARC 2018
3. Les Binet and Peter Field: “Marketing in the Era of Accoutnability” – WARC 2007
4. Les Binet and Peter Field: “The Long and Short of It” – IPA 2013
5. Robert Heath.: “50 years using the wrong model of TV advertising” - MRS 2007
6. Adam Sheridan: “Selling Creative Research Short?” – IPSOS 2019
7. Orlando Wood: “How Emotional Tugs Trump Rational Pushes” - System1 2012
8. Dominic Twose: “How to measure emotional response in advertising” – Admap 2017
9. Stephen Whiteside: “System1’s five tips to deliver more effective television ads” – WARC 2020
10. Darren Poole: “How to use copy-testing or Pre-Testing” – Millward Brown / WARC 2016