Concept Testing

Concept TestingConjoint Analysis

February 15 & 20, 2007

Evaluating with Customers Concept Testing

is used to help screen and refine new product ideas

Conjoint Analysis used to determine the

combination of attributes that maximizes appeal

Concept Testing A concept is composed of attributes

and benefits for a particular usage situation

Attributes incorporate a specific product form and technology

DetermineDetermineCustomerCustomer InterestInterest

see Page and Rosenbaum (1992), “Developing an Effective Concept Testing Program for Durables,” J Product Innovation Mgmt

Purposes of Concept Testing

To identify very poor concepts so that they can be eliminated.

To estimate (at least crudely) the sales or trial rate the product would enjoy (buying intentions, early projection of market share).

To help develop the idea (e.g. make tradeoffs among attributes).

Procedure for a Concept Test

Prepare concept statement Clarify specific purposes Decide format(s) Select commercialization Determine price(s) Select respondent type(s) Select response situation Define the interview Conduct trial interviews Interview, tabulate, analyze

Concept Testing Cautions and Concerns

If the prime benefit is a personal sense (aroma, taste).

If the concept involves new art and entertainment.

If the concept embodies a new technology that users cannot visualize.

If concept testing is mishandled by management, then blamed for product failure.

If customers simply do not know what problems they have.

The Concept Statement

The Customer Value Proposition: FOR {the ideal customer} WHO {have the following problem} MY PRODUCT IS A {product category} THAT {key differentiating benefit} UNLIKE {the major competitor}

UsageUsageSituationSituation CustomerCustomer

ProductProduct

The Concept Statement

Format Narrative

Narrative

Here is a tasty, sparkling beverage that quenches thirst, refreshes, and makes the mouth tingle with a delightful flavor blend of orange, mint, and lime.

It helps adults (and kids too) control weight by reducing the craving for sweets and between-meal snacks. And, best of all, it contains absolutely no calories.

Comes in 12-ounce cans or bottles and costs 60 cents each.

1. How different, if at all, do you think this diet soft drink would be from other available products now on the market that might be compared with it?

Very different ( ) ( ) ( ) ( ) Not at all different

2. Assuming you tried the product described above and liked it, about how often do you think you would buy it?

More than once a week ( ) ( ) ( ) ( ) ( ) ( ) Would never buy it

The Concept Statement

Narrative

The Concept Statement

Format Narrative Drawing / Diagram

Drawing/Diagram

The Concept Statement

Format Narrative Drawing / Diagram Model / Prototype Virtual Reality

Virtual Reality Information Acceleration http://www.wharton.upenn.edu/learning/

futureview/

Developing Concepts to Test

Time to prepareTime to prepare test materialstest materials

Number of itemsNumber of items testedtested 11MoreMoreMostMost

PreferredPreferred

LeastLeastPreferredPreferred

Paper &Paper &PencilPencil

ComputerComputer

PrototypePrototype

Working ModelWorking Model

e.g., www.acupoll.com

What is generally tested?

BUYERBUYER

Does it solve Does it solve a “problem”?a “problem”?

yesIs itIs it““believable”?believable”?

yesIs itIs it““unique”?unique”?

yes

Would it be Would it be bought at onebought at oneof several testedof several testedprice points?price points?

yes

Can measure potential customer reactions using: (1) 5-pt “definitely not” - “definitely” scales(2) sorting tasks

Considerations in the Concept Test

Core Idea vs. Positioning/Commercial Concept Statement

New Brand vs. Old Brand vs. No Brand Concept statement: narrative, drawing, model? Respondent group: Lead users? Large users? Response situation: Where? How? Purchase Measure Decisions:

Buyer Intent Frequency Price

Product Diagnostics Attribute Diagnostics

Ask the right people...

time

SalesSalesThe ChasmThe Chasm

Early MarketEarly Market Mainstream MarketMainstream Market

TechnologyTechnologyEnthusiastsEnthusiasts

VisionariesVisionaries PragmatistsPragmatists ConservativesConservatives

See (1) Rogers (1995) Diffusion of Innovations (2) Moore (1991) Crossing the Chasm (3) www.chasmgroup.com

Lead U

sers

and In

novat

ors

vs. M

ainstr

eam

Mar

ket

Ask the right questions... How important is the product

“experience”? Does the customer have to “touch

& feel” the product to understand the benefits offered?

““Simulate” the ExperienceSimulate” the Experience

How can concepts be tested?

Focus Groups One-on-One Personal Interviews Mall Intercept Phone Interviews Postal Surveys Internet Surveys Hybrids (e.g., phone-mail-phone)

Compare in terms of:Compare in terms of:sample control, concept flexibility, costsample control, concept flexibility, cost

Compare in terms of:Compare in terms of:sample control, concept flexibility, costsample control, concept flexibility, cost

see: (1) Pope (1993), Practical Marketing Research (2) McQuarrie (1996) The Market Research Toolbox

Typical AnalysisCategory or Industry

Purchase Intent Concept Norm

Definitely Would Buy 27% 20%Probably Would Buy 43 40Top Two BoxTop Two Box 70% 70% 60% 60%

Might or Might Not Buy 22% Probably Would Not Buy 5Definitely Would Not Buy 3

Summary of Concept Testing Advantages

relatively easy to get customer input can be used as an early screen for new

product ideas

Limitations not that helpful for the design and

development of specific product forms not as reliable for discontinuous

innovations

Conjoint Analysis

Primary benefit in addition to (or in lieu of) concept tests:

forces a trade-off

Conjoint Analysis

Can be used to quantify the relative importance of attributes

Can be used to help determine the combination of attributes that maximizes appeal

Relatively easy for incremental innovation Requires experts or information

accelerationfor discontinuous innovations

see (1) Page and Rosenbaum (1987), “Redesigning Product Lines With Conjoint Analysis,” J Product Innovation Mgmt (2) www.sawtooth.com {Sawtooth Software}

Major Assumptions An offering is a bundle of attributes and

benefits. An offering can be decomposed into a bundle of “features” for which “utility values” can be calculated.

The utility value of an offering is some simple function of the utilities of the offering’s “feature” levels.

Customers prefer the offering with the highest utility value.

Conjoint: Steps 1 and 2

Identify Relevant Attributes Survey/Focus Group/Intuition Salsa Example

(Thickness, Color, Spiciness)

Identify Relevant Levels of Each Attribute

Thickness: Regular, Thick, Extra-Thick Color: Red, Green Spiciness: Mild, Medium-Hot, Extra Hot

Create Profiles for each Combination

3 thickness (reg., thick, extra-thick)

2 color (red, green) 3 spiciness (mild, med/hot, extra

hot) Leads to 3X2X3 = 18 Profiles

Conjoint: Step 3

Choose a Sample Considerations:

Consumer Involvement Typicality Diversity (if multiple segments) Expertise (if complex or discontinuous)

Conjoint: Step 4 Obtain Customer Judgements

Rank Order Sort into categories Rank the profiles within each category

Pair-wise Comparisons Use a computer package to quickly hone

in on important attributes

Conjoint: Step 4 - Example

Conjoint: Step 5 Compute Individual Value Systems Use MONANOVA for rank order data

Output in the form of standardized utilities

Output Example

Conjoint: Step 6 Find the average utilities (part-worths)

for each attribute Intuition: Find the attribute with the biggest

range in utilities across the different levels Use graphs/calculations for importance measures Be careful with averages

Segments may exist Cluster Analysis can tell you

Output Example

Let’s consider golf balls...

• • distance and durabilitydistance and durability• • durability and pricedurability and price• • distance and controldistance and control

Conjoint Analysis Average Average Price

Driving Ball Life Distance

250 yards 54 holes $3.00

220 yards 36 holes $4.00

200 yards 18 holes $5.00

Your “Optimal” Product Design

$5/sleeve

Driving Distance of 200 yards

Average Ball LifeAverage Ball Lifeof 54 holesof 54 holes

See also Titleist’s Ball-Fitting and Wilson’s Custom Fit

How can conjoint analysis be conducted? One-on-One Personal Interviews

written or verbal concept descriptions multimedia presentation of concepts

RTI’s TradeOff VR; Sawtooth’s Sensus TradeOff; MIT’s Information Acceleration

networked computer facilities Moskowitz Jacobs

Mail written concept descriptions disk by mail

Internet the future??

Summary of Conjoint Analysis Advantages

the relative importance of product features can be quantified using customer input

only need to test a relatively small number of actual product designs

Limitations output is usually not directly linked to actual

purchase