Coke v Pepsi and the sip test problem
In 1972 Coke had 18% of the soft drinks market while Pepsi had a mere 4%. By the early 80s, Cokes’ share of the market had fallen to 12% while Pepsi had grown to 11%, despite Coke spending more than $100m per year on advertising and having superior vending machine and retail reach.
Around the same time, Pepsi started promoting the Pepsi Challenge. Consumers were asked to take a sip of Coke and Pepsi in glasses marked Q and M and choose their favourite. 57% picked Pepsi and 43% picked Coke. Coke disputed the findings at first but eventually ran its own tests and found the feedback to be accurate. Pepsi won in an A/B sip test.
Coke set about improving their formula to fight back against the slide of market share to Pepsi. They worked restlessly to get a modified flavour and in September 1984, Coke ran sip tests with hundreds of thousands of Americans and came out between 6% and 8% ahead of Pepsi. New Coke was born.
On the day of its launch Cokes’ CEO Roberto Goizueta announced, “This is the surest move the company has ever made” and there was little reason to doubt what he said. The research was in, the stats showed New Coke would win the taste fight.
Within months of the launch, New Coke failed. Coke drinkers rose up and Coke were forced to bring back Classic Coke. But oddly enough Pepsi never managed to capitalise on the rebellion of Coke drinkers and to this day still lags behind Coke in terms of popularity.
So how did Coke get it so wrong? They had all of the statistics that any manager would need to make an informed decision. The problem came about because Coke took the wrong data. In a sip test, you only take a mouth full of Coke and a mouth full of Pepsi. Pepsi is sweeter and when taking a sip will win the preference of most tasters. But we don’t just sip a Coke or Pepsi, we drink a can, bottle or glass of the stuff. So when the new, sweeter flavour came into being, it was rejected, as the overall experience wasn’t as pleasurable as the original Coke.
The same thing happens when we review our online marketing stats. Page impressions, unique viewers and open rates blind us. Finding followers and even goal pages seem to be a sensible measure of our success. I argue however, that this is just another sip test. It gives us a sense of how we are performing but it doesn’t give us the whole picture.
So what should we measure? When creating an online marketing review I believe that the new metrics and the measurement of success or failure should be based upon the quality (not quantity) of your audience, their level of engagement, their loyalty, your influence over the audience and whether or not they are spreading your good word.
By combining your metrics to give a holistic overview of your audience and your influence then you are drinking the full can of Coke. Looking at click-to-sell ratios is just sip tasting.
