This paper examines how advertising effectiveness can be defined and measured. We propose a 3D-effectiveness key performance indicator involving the weighted prominence of three factors: recall, brand image and buying intention. In this way, the indicator proposed improves the metrics of this type known in the Spanish market. Using experimental data about 20 fixed brands and other 12 variable brands collected along the 52 weeks of a year, we find that this indicator reflects consumer’s perceptions and that it fits advertising company’s needs. We have developed a web tool to provide the market with a benchmark measure of advertising effectiveness. Finally, we discuss future research lines.
One of the greatest challenges facing advertisers and agencies continues to be measuring the effectiveness of their advertising campaigns, typically one of the largest line items in the marketing budget. The situation is fur- ther complicated by the emergence of new advertising formats and platforms [
In parallel, advertisers are thirsty for metrics to help them with their day-to-day decision-making [
Before we can decide whether a communication campaign has been effective or not, we need to clarify first what we actually mean by effectiveness, against what objectives should advertising effectiveness be evaluated [
where ai are the weights that each advertiser determines based on the objectives which their communication had for each of the Di dimensions. This new way of defining effectiveness is better suited to the reality of advertisers, who sometimes run campaigns whose main objective is to achieve media coverage by getting many people to recall having seen the advertising, other times what they want is to improve brand image and at other times, however, all they want is to achieve rapid sales increases.
The objectives that we intend to address through this research are five: 1) Creating a concept of effectiveness that suits the needs of each advertiser (as seen in the previous section) and is valid for the different communica- tion strategies that may be applicable; 2) That such effectiveness, that will be measured through fieldwork and therefore will be redefined as “perceived effectiveness”, will be a type of effectiveness without the frequent bi- ases that these kinds of methodologies have; 3) That we may set a benchmark in the market, so when comparing effectiveness between different brands, there will be a point of reference against which to measure performance; 4) That this information is open and free to everyone; 5) And finally, that the data may be available in a flexible way, so that we might speak of a “just-in-time” measurement of perceived effectiveness.
Since January 2012, we have conducted a weekly tracking with CAWI methodology (Internet interviews), aimed at a representative sample of the Spanish population aged between 16 and 55. We have carried out 500 weekly interviews, with a total of about 26,000 annual interviews, based on quota sampling by age, sex and region. The sampling error is 4.4%. Each week the 20 fixed brands have always been tested and another 12 that have been changing each month. The 20 fixed brands were selected based on the Top of Mind advertising awareness 2011 ranking of IMOP (study done by the Public Opinion and Marketing Institute in Spain, which measures spontaneous advertising recall by media, without suggesting a brand or a product line), and the adver- tising investment ranking by INFOADEX company (source of investment data in Spain) also for 2011, with a maximum of 4 brands per sector. The performance indicators for these 20 brands have been tracked over the 52 weeks of the year. The other 12 brands that have rotated through each month represent two different sectors, so that in six months we have investigated the 12 sectors with the largest investment and enhanced awareness in the country. This has allowed us to have a full picture of each sector twice a year (see
. Brands
Renault |
---|
Mercedes |
Seat |
Audi |
Carrefour |
El Corte Ingles |
Media Markt |
Lidl |
Telepizza |
McDonald’s |
ING Direct |
Movistar |
Vodafone |
Orange |
Jazztel |
Coca-Cola |
Danone |
Nike |
Adidas |
L’Oreal |
. Sectors
Automotive |
---|
Energy |
Retail |
Catering |
Finance |
Insurance |
Telecommunications |
Food |
Beverages |
Electronics & IT |
Textile |
Beauty & Cleanliness |
The questionnaire covers the three axes of the perceived 3D-effectiveness: Recall, Image and Business, as well as questions about the media in which they recalled having seen the advertising, product consumption, overall assessment of the brand, and finally, and most importantly, a battery of questions on media consumption, which will help us to make a study of coincidence against the Media General Study (EGM) official source of audience measurement in Spain, which will give greater strength to our study ensuring that the media consumption is also representative of the general population (See an outline of the questionnaire used in
When calculating the effectiveness index, we eliminated three common biases that characterize this type of research: 1) The brands engaged in a campaign in the week in which the fieldwork is carried out and have a large investment on that week, always have a higher recall than the ones which have a smaller investment. Therefore, a brand “A”, out of the 20 analyzed each week, can have a high recall on a given week via a large investment versus another brand “B” having a smaller recall, because it had a very small advertising investment on that week. If we understand that having a high recall to be equated to having great effectiveness, we will then be penalizing brand “B”. To avoid falling into this bias when estimating effectiveness, we performed some sim- ple regression models between investments and recall levels,
where
Thus D1 is the effectiveness dimension corresponding to recall and there will be campaigns with positive ef- fectiveness and campaigns with negative effectiveness, what we will standardize at a later stage; 2) Also with simple regression models, we note that there is a high correlation between the value of a brand and how people perceive the advertising of that brand with a view to improving the image consumers had of the brand. If brand A is a highly valued brand, consumers will tend to claim that its advertising helps them improve the image they have of the brand, and vice versa when a brand is poorly valued. The way to overcome this bias in our effec- tiveness measurement will again be to compare the expected value according to the model, with the actual value obtained through the questionnaire; 3) Finally, in the business axis we can see the multivariate dependence of the brand image perception and its market share. A brand with a high value image and a high market share will always be favored in such questionnaires, falsifying the data unless this condition is isolated. With these three kinds of models, we obtained three effectiveness measurement indicators free from some of the main biases that characterize this type of research.
. Outline of the questionnaire
Questionnaire |
---|
Prompted Advertising Awareness |
When-Prompted Advertising Awareness |
Media-Prompted Advertising Awareness |
Impact of Advertising on Brand Image |
Impact of Advertising on Buying Intention |
Clients/Non-Clients Identification |
Brand Valuation |
Media Usage |
Media Usage Frequency |
Region |
Gender |
Age |
Level of Studies |
In no circumstances this method could compete with research carried out by advertisers by means of question- naires over long periods of time, most of the time face-to-face, exclusively designed to measure the effective- ness of communication campaigns one by one. Neither is designed nor intended to measure all the important attributes of the image of the brand, or going into the details of the variables such as buying intention or recom- mendation of a product.
The 3D-effectiveness key performance indicator proposed in this paper involves the weighted importance of three factors: recall, brand image and buying intention. In this way, this new metric reflects consumer’s percep- tions and it fits advertising company’s needs. The indicator is built using experimental data about 20 fixed brands and other 12 variable brands collected along the 52 weeks of a year. The results obtained over the months have been enriching and in the following paragraphs we will discuss some of the main results.
Effectiveness KPI. First, we obtain a perceived effectiveness KPI, week by week, for the top 20 brands in Spain throughout the year 2012 (
Changing Concept of Effectiveness. Second, for each of the analyzed brands we can get the value of the 3 ef- fectiveness axes, and, based on the principle that for each advertiser the concept of effectiveness may vary, each one of them can allocate the weights they may deemed valid. For example, if Nike considers that for their cam- paigns the weights should be 0.25 to recall and image, and 0.50 to business, they would move from the eleventh position, on the 26/11/212 week, to third position (
Top-20 Effectiveness. Third, we perform a weekly effectiveness tracking of the 20 top brands in Spain consi- dering advertising awareness and investment. Such tracking will allow us to know how effectiveness changes
Perceived 3D-effectiveness (26/11/2012-02/12/2012)
Perceived 3D-effectiveness (19/11/12-25/11/12)
3D-effectiveness. Recall. Brand. Purchase
over time, besides being a benchmark for this indicator in the Spanish market, and which can be crossed with other more general variables. This will give rise to future research such as: what does effectiveness depend on, and whether it is affected, for example, by such variables as the consumer confidence index, reflecting a situa- tion of crisis.
Effectiveness by Sector. Fourth, each month we have analyzed two sectors and every six months we repeated the study, so that each sector has been analyzed twice within a year. Based on this, we analyzed effectiveness by sectors, being able to compare each brand, no longer with the Top-20, but with their 6 or 7 main competitors. As future research lines in this case we want to emphasize the comparison of the evolution of the effectiveness by sectors and their drivers.
Effectiveness without Biases. Fifth, as explained above, we have eliminated some of the biases that frequently contaminate these researches. In
Effectiveness
Effectiveness without biases
recall close to the average. Our experience gathered throughout the 12-month study is that in general the reading that is obtained of effectiveness with or without biases comes out as significantly different. We believe that, as far as possible, it is important to eliminate the biases, and to this end we propose to use the methodology de- scribed above.
Targeting Based on Effectiveness. Sixth, considering that each week the sample size is large enough, each month we can turn to the study of the effectiveness based on such variables as sex, age, educational level, region, media consumption or the fact of being a consumer or not of the brand. This will allow advertisers to address that segment of the target audience with low effectiveness and analyze the drivers of those sectors for which ad- vertising has been highly effective.
Effectiveness for Each Media. Finally, from knowing which media the interviewees recall having seen the advertising, we can estimate the perceived 3D-effectiveness for each media.
Effectiveness for each media
This project has largely achieved its initial objectives. We have been able to create an effectiveness indicator to suit the needs of each advertiser and which is valid for the different communication strategies that may exist (achieving coverage (recall), improving brand image and increasing buying intention). This indicator improves the metrics of this type known in the Spanish market and this improvement occurs in two ways: Firstly it ad- dresses not only the traditional advertising recall, but also concepts as important for the advertiser as brand im- age or buying intention, and, secondly, it eliminates some of the biases that commonly crop up in these mea- surements.
Another handicap that these types of studies usually have, is the amount of time one has to wait until data are available, 2 or 3 weeks, at the earliest, after the end of the campaign. In our case, we have pursued speed and re- liability of results, making them available on the week following the end of the campaign being tested.
Furthermore, in an effort to make this KPI known to the market and thus being capable to set an effectiveness benchmark against which to compare performance, we have developed a tool that has web access and which is open to everyone [
All this makes this research and its results especially relevant for the Spanish market advertisers; they can now know a little better what the effectiveness of their campaigns has been, and also what the effectiveness of the different media has been. This does not mean that this study and its methodology may be valid for any other country that would want to implement it, but it could be adapted to each particular case.
As future lines of research, several options are open, highlighting that of analyzing the evolution of effective- ness and to discover what are its main drivers, as well as comparing effectiveness by sectors, looking for possi- ble explanations. In addition, as from January 2014, we will proceed to analyze the Top-100 instead of the Top-20.