testing ads adapted from j. scott armstrong from lectures related to persuasive advertising. file:...
TRANSCRIPT
Testing Ads
Adapted from J. Scott Armstrong fromlectures related to Persuasive Advertising.
File: TestingAds-R16In Lectures for AdPrin/Testing Ads
Updated January 2015
1
Learning Diary
The lectures follow an experiential learning experience. To make this work properly:1.Obtain a learning diary (paper). A 10 x 13 bound diary is suggested.2.Keep it up to date.3.Take the learning diary with you to all class sessions.4.For self-learners, use the diary to track your learning progress for all of your learning activities.
2Adprin.com
Objectives of this session
To understand and apply these principles and techniques (not to convince you). Ask for clarification as needed.
Set a goal for yourself on how many principles and techniques you plan to use by the end of this session. Even a goal of one will help you. Put this in your learning diary now.
Note: We will discuss only some of the slides. When you go through the lecture on your own, view it in “Slide Show” and follow the experiential procedures.
Adprin.com
Procedure
Focus on understanding.
Record questions in your learning diary that will help you to apply the techniques or principles, then, after you decide which ones you want to apply, try to answer these from the readings. If not clear, ask others for help.
4Adprin.com
The most important word in the vocabulary of advertising is test
• Test your premise • Test your headlines and illustrations• Test your media • Test your level of expenditure
“Never stop testing and your advertising will never stop improving.” David Ogilvy
Adprin.com
How would you test this headline for Tylenol?
“When not playing is not an option.”
6Adprin.com
Testing the effectiveness of advertisements
Assume that you were given 10 pairs of print ads and asked to identify which ad in each pair would be more persuasive. Each ad advertises a product with the same brand. In addition, you would like to suggest how to improve each of the ads.
You have a budget of $20,000. Write different approaches in your learning diary. Spend at least five minutes if you already know how to solve this problem, and ten minutes if you do not know. If working in a group, exchange your solutions with a partner.
When you finish, go to the next slide.7Adprin.com
Five approaches to testing advertisements:Listed by Effectiveness
All of these methods involve comparisons of responses for two or more alternative advertisements:1.Unaided ratings by novices (used for Super Bowl).2.Unaided ratings by experts (used for Super Bowl).3.Copy testing: Test which ads gain the best response.4.Persuasion Principles Audit: Rate how well ad applies evidence-based persuasion principles. This is analogous to how we evaluate medical doctors and engineers.5.Field experiment: Test direct-response ads to see which are most effective (e.g., as done by Bose radio ads).
Reference: Armstrong et al (2014)
8Adprin.com
Benefits of unaided judgment by non-experts*
1. Very low cost2. Very fast3. Combining helps to improve validity
9Adprin.com
Unaided judgments by experts
Experts are typically poor at forecasting effects of different approaches in complex uncertain situations.
TetlockSeer-suckerTheory
Experts in consumer behavior were poorer than marketing practitioners and high-school students in predicting consumer behavior (Armstrong 1991).
10Adprin.com
Persuasion Principles Audit
• Freeware and training module available at AdPrin.com
• Requires some knowledge of the principles(See AdPrin.com & Persuasive Advertising)
• Requires good cognitive skills, logic, and patience• Creativity helps• Practice helps• People differ in their ability to do this task. I suspect
that economists, lawyers, and engineers do well.• The audit leads to a Persuasion Principles Index (PPI)
11Adprin.com
AdPrin Audit procedure
• Complete 90-minute AdPrin.com training module • Rate print ads in about 30 minutes, motion ads in 40.
• Use five independent coders, then combine by using the mode
• When ratings differ “substantially” for “relevant principles” obtain a new rater
– for details see Predictive Validity of Evidence-based Advertising
12Adprin.com
Benefits of the AdPrin Audit
1. Relatively low cost2. Relatively fast3. Leads to evidence-based improvements
13Adprin.com
Copy testing: Selecting a representative sample
1. Who to test? • A convenience sample• Mall intercepts or student subjects, or Mechanical Turk at Amazon. • Screening questions for similarity to the target market
2. How large a sample? Start small, increase as the client desires or as you
think proper.
3. What approach?Experimental design, with random assignment of
subjects to two or more treatments.
14Adprin.com
Copy testing for advertisements
1. Compare responses to ads.2. Ads do not have to be in finished form.
Rough copy generally produces realistic results on the relative magnitude of effects.
3. High face validity convincing to clients.4. Are ads appropriate for all groups?
- Consider testing among members of interest groups.- To maintain security, ask subjects to role-play such interest
groups.
15Adprin.com
Testing procedure
1. Prepare Test Package– Degree of finish of the ads (“finished” not critical)– Context (e.g., in a magazine)– Number and length of exposures
2. Design of the test (e.g., pre- and post- tests; within subjects v. across subjects).
3. Use multiple measures of effect
Adprin.com
Testing responses
1. Describe the subjects’ role (e.g., “Assume you are planning to buy a car”)
2. Focus on behavioral change (e.g., purchase intentions, not “liking”)
3. Use a scale for responses when possible (e.g., use “0 to 100%” for intentions)
4. Use alternative ways of asking the questions5. Pretest the questions to ensure understanding6. Test elements of the ads
(e.g., picture alone versus headline alone)
17Adprin.com
19
Test the illustration: What is the product?
20
Complete ad
Administering copy tests
1. Written testing instruments and instructions.2. Self-administration of tests by individuals.3. Those administering the instrument should be
unaware of the hypotheses if they interact with the subjects.
4. Prepare written instructions for administrators. 5. Take notes about departures from the script.
21Adprin.com
What not to do for copy testing
1. Do not ask customers how they make decisions when purchasing products. They do not know even when making important decisions. (See meta-analysis by Nisbett & Wilson 1977). Even if they did know, they would not remember (Kaasa, Morris & Loftus 2011).
2. Consumers are not experts at advertising, so do not ask them how an ad might be improved or if they liked an ad.
3. Do not use focus groups. 22Adprin.com
Review: Benefits of copy testing
1. Useful for comparing responses to ads.2. The ads do not have to be in finished form. Rough
copy generally produces realistic results on the relative magnitude of the ads’ effects.
3. Copy testing has high face validity and is thus convincing to clients.
4. Testing can also assess whether the ads are appropriate to all interest groups.
23Adprin.com
Experimental Designs (Campbell and Stanley)
“BEFORE”MEASURES TREATMENT “AFTER”
MEASURES
“AFTER ONLY”
“BEFORE-AFTER”
“BEFORE-AFTER WITH CONTROL”
EXP.
CONTROL
“FOUR GROUP- SIX STUDY
EXP I
EXP II
CONTROL I
CONTROL II
After Only Design
• Use time series• Prepare (obtain) predictions with ad campaign • Compare results with prediction
PA
Provide Contingency PlansAdprin.com
Laboratory or field experiments?
1. Laboratory experiments are typically less expensive and offer more control.
2. Field experiments are more realistic.3. Findings from each method are comparable.
For a brief summary, see Persuasive Advertising, pp.7-13.
See Locke’s Generalizing from Laboratory to Field Settings
26Adprin.com
What would you do in this asbestos case?
492 X-rays submitted as court evidence in asbestos lawsuits. X-ray readers found lung damage in 96% of the cases. You have been hired as an expert witness for all cases. What would you do? Spend five minutes to answer this in your learning diary.
If working in a group, share your ideas with a learning partner.
When finished, go to the next slide.
28Adprin.com
Resolving the asbestos case
Replicate: Ask independent experts to read the x-rays.Result? Only 4.5% of the X-rays were read as showing damage. (See Gitlin study)
The plaintiffs will claim that you are biased. How would you protect against that? •Click after you have written your answer in your learning diary.•Take steps to ensure that the independent experts do not know why they are being asked to do the readings.
29Adprin.com
Reporting on your research1. Bring in research as needed to support
recommendations.
2. Use statistics to communicate information (See The Visual Display of Quantitative Information by E. Tufte).
3. Do not use statistical significance. Use prediction intervals instead.
4. Report on shortcomings of your research.
5. Provide full disclosure on data and methods.
See checklist for oral and written reports30Adprin.com
Applications: Tylenol ad
Apply material from this lecture to test this Tylenol headline.
“When not playing is not an option.”
If working in a group setting, exchange your applications with a learning partner.
Then click for some answers
Adprin.com
Tylenol headlineIt did poorly on copy tests.
Also does poorly on principles. For example:1. No brand name2. Two negative words3. Contains a period4. Social responsibility and legal issues*
* For example, they might get sued if an unfortunate event occurs.
32Adprin.com
Ideas for Applications of Techniques for Testing Ads
If you are in a class, test your advertisements for the “house ad” by using the techniques discussed in this lecture.
If you are a self-learner apply the testing techniques to advertisements in you organization, or to a cause that is of interest to you.
33Adprin.com
Techniques
In your diary, describe the techniques that you were able to use to test your advertisement and rate your success (e.g., creativity, objective setting)
34Adprin.com
Advice on learning techniques
One study found that fewer than 10% of students were successful in applying new knowledge.•This went to 20% if they actively applied what they were taught during a class session.•It went to 90% when they worked with a learning partner and coached each other.
Select techniques to apply
35Adprin.com