Socrates : Our trade-off survey toolset

An introduction and example screens

Trade-off surveys

• Trade-off surveys differ from standard surveys in that they ask

participants to choose between various product features rather than

grade individual features.

• They can offer a more complete picture of what is truly important to

a consumer or employee.

• They are also the best way to measure factors like price sensitivity

• There are various types of trade-off surveys:

– Discrete choice• Participants are presented with attributes choices packaged together and choose their preference or

none of the above.

– Non-packaged• Participants are presented with all the attribute choices and asked to rate them and assign importance

– Conjoint• Attributes are grouped in to packages and then ranked or rated

• There are variants of conjoint (e.g. pairwise, partial, full profile) that either limit the number of choices

in each package, or limit the combination of attributes that are permitted.

• However, trade-off surveys are often perceived as complex to

perform and this could explain why they are less commonly used.

What is Socrates?

• Socrates is a survey toolset that allows rich-featured

trade-off surveys to be created, managed, delivered and

completed over the internet.

• Socrates was created with two central principles¹:

– Pragmatism

• We have implemented the features that we believe will bring the

greatest benefit soonest

– Ease of use

• Our intention was to create a toolset that will remove barriers to

constructing and participating in trade-off surveys.

• Socrates is available to be used without installation and

on a fixed-cost or a pay-as-you-use basis.

¹ more on how these principles have informed our design below

Functional view (simplified)

DEFINTION

• The design, and creation of a

survey.

• This will be undertaken by skilled

marketing / customer insight

strategists

COLLECTION

• This is an end-user performing the

survey and entering their choices

• Collection can be remote or guided

by the analyst

ANALYSIS

• Real-time monitoring of the

survey’s progress

• Analysis of the final results and

rankings

REPORTING

• Presentation of the results of the

analysis to the customer

• These can be delivered online or

through traditional means (e,g.

Slideshows etc)

The diagram above outlines the four key areas in an online survey

toolset and breaks them down in to smaller functional components.

Not all components are shown, and this diagram is oriented

towards trade-off surveys rather than all surveys in general.

FOUR PROCESS STEPS

Socrates 2.1 – Functional view

• We have implemented the

functions that will make

the biggest difference to

the marketing strategist

and customer

• We have left the specialist

analytical components to

existing tools, preferring to

allow data to be imported

and exported integrate in

an open way

• We have focused on

areas that we can apply

our skills in usability and

implementation to create a

better toolset.

The orange-bordered boxes highlight the features Socrates 2.1

implement.

Those with a grey border are on our planned product roadmap.

Those shaded in grey are features we plan either not to

implement, or integrate other solutions.

OUR DESIGN RATIONALE

Socrates – Design Principles

• We focus on the functions that allow

trade-off surveys to be created,

managed, and performed.

• There are many software solutions

already ideally suited to perform the

more complex features, especially

around statistical analysis

• We enabled our solution to easily

integrate with these rather than

replicate features they do better.

• We wanted to remove the cost of

installation from our product and also

get great performance and widest

access, so we chose to implement on

the internet.

• We also wanted our solution to be

secure and scalable so we chose

Google’s hosting platform rather than

attempt to implement ourselves.

• We believe that marketing strategy

departments will have specialists

capable of designing meaningful trade-

off surveys

• Our goal was to ensure these

specialists had clear and intuitive tools

to design and manage the surveys.

• Socrates guides both the designer and

participant through simple steps

enhancing their confidence in the tool

and the process.

• Our experience in Usability and online

customer feedback shows that end-

users will abandon surveys they feel

are difficult to use

• We have spent time usability and

accessibility testing our applications

with real users in order to remove

barriers to participation.

PRAGMATIC APPROACH EASE OF USE

Our experience with online surveys

• Major multinational telecommunications corporation

– We provide hosted online customer satisfaction surveys for one of the best

known names in mobile technology

– This year we’ve run 12 surveys in 8 countries with over 60,000 invitees

• 360-degree survey for large investment Bank

– We’ve previously created a 360-degree survey tool for a large Investment Bank

– We hosted the survey which was a major factor in the yearly compensation

round for over 15,000 employees

• Online survey tools for direct marketing organisation

– We created a custom survey tool that enabled a Direct Marketing organisation to

better connect with their customers.

• Architecture and design advice to market research company

– We provided software architecture and design advice to a market research

company seeking to create a new survey product suite.

– Our recommendations ensured their system could perform and scale to support

their existing and future business needs and also provided performance

improvements of several orders of magnitude.

The Socrates survey process

A walkthrough with screenshots

Terminology used in Socrates

Before the walkthrough this is the terminology we use

• Designer

– A marketing strategist who devises the survey

• Participant

– An end-user (usually a customer or employee) who performs the survey

• Attribute

– A feature of a product or service (e.g. Price, Free minutes, Basic salary, Holidays)

• Package

– A group of attributes collected together, each with a value assigned (e.g. Price=£39.99; Free

minutes=300)

• Iteration (Page)

– A set of packages presented to the participant for ranking or rating

• Ranking

– Where packages are selected in order

• Rating

– Where packages are given a score either out of a fixed amount (e.g. Score out of 10) or

where a certain number of points is shared between the packages.

The survey process – Designer perspective

Designer: Creating a new survey

• The designer creates a

new survey from the

dashboard

• They then can enter

the basic details for the

survey– Its name

– A description

– The text of the email that

will be sent to the

participant when inviting

them

– The welcome text that

will greet the participant

when they follow the link

to their survey page

– ...

• Once finished the

survey is stored and

ready to add attributes

Designer: Defining attributes

• Attributes are the basis

of any survey

• Here the designer is

able to add, view, and

edit the attributes in

place

• The screen opposite

shows three existing

attributes and the

ability to add more

• Attributes have a

name, description, and

optionally units (e.g.

GBP, minutes, etc)

Designer: Defining packages

• Once the set of

attributes is created the

Designer can group

them together in to

packages

• The attributes are

made in to columns

and the designer sets

values for them in the

particular package.

• Most importantly the

designer is able to see

and change the values

that she/he has added

making it easy to

ensure the right

package combinations

are there.

Designer: Defining pages

• Once the packages have

been created, they can be

grouped together in to

“pages”

• These are iterations in the

surveys where the

participant will be

presented with a number

of packages to choose

between.

• The upper table shows the

pages already defined and

the packages therein

• Highlighted items show

what has just been

changed

• The lower portion allows

the designer to drag and

drop packages in to the

workspace (grey) to create

a new page or edit another

Designer: Launching surveys

• Once the survey has been

defined, and participants

added it is ready to be

launched

• Surveys are launched via

the dashboard – along

with any other actions that

can be performed (close,

archive, create new,

delete)

• The dashboard has a

summary of alerts relevant

to the designer as well as

real-time monitoring of

survey take-up.

• Once a survey is

launched, Socrates will

email the participants and

provide them with a

unique link to their

personalised survey pages

The survey process – Participant perspective

Participant: Welcome

• The participant will

have a unique URL to

their survey page in

their personalised

email.

• When they follow this

URL they will see a

welcome page.

• This can contain a

personalised greeting

(if set up by the

designer) and

instructions on how to

complete.

• The participant presses

“Start” to begin the

survey.

Participant: Ranking packages

• Each of the pages in

the survey correspond

to a page set up by the

designer

• In the example the

participant is on the

first (of three) pages.

• This is indicated by the

highlighted progress

bar

• The participant drags

and drops the package

boxes in order to rank

them

• The attributes and their

values are shown in

each box

Participant: Rating packages

• This example shows

an alternative where

the designer has asked

the participant to rate

the packages

• This rating is from a

total allocation of 20

points

• The participant has

allocated 8 points to

Package A and 2 to

Package B

• The boxes at the foot

of the screen clearly

show how many more

points there are to

allocate.

Technical Overview: A cloud-based solution hosted on Google’s infrastructure

• No installation needed

– Application is ready to

use

• Secure

– Survey data is protected

per client

• Highly scalable

– Can grow to the needs of

the survey. From 1

participant to 1000s

• Pay for what you use

– Cloud-based and

charged on usage

• Accessible through

multiple devices

– Browser

– Smartphone

• Global

– Google’s infrastructure

means good

performance worldwide

FEATURES

Find out more

• Read about zico our standard online survey tool

– http://truenorth.gb.com/products/zico

• Latest news about Socrates on its product page

– http://truenorth.gb.com/products/socrates

• Contact us about using Socrates– Sid Haniff, sid@truenorth.gb.com, 07968 277 116

– Mark Thomsit, mark@truenorth.gb.com, 07801 480 820