alan avidan, exec. director & chief beezzzdev

32
Analytical Optimization Technologies for Games & Apps Analytics, A/B Testing, Segmentation & Dynamic Best-Fit Alan Avidan, Exec. Director & Chief BeezzzDev

Upload: pepper

Post on 12-Feb-2016

51 views

Category:

Documents


0 download

DESCRIPTION

Analytical Optimization Technologies for Games & Apps Analytics, A/B Testing, Segmentation & Dynamic Best-Fit. Alan Avidan, Exec. Director & Chief BeezzzDev. Points We’ll Cover. What is optimization What can be measured and optimized Optimization t echnologies for games and apps Analytics - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Alan Avidan, Exec. Director & Chief BeezzzDev

Analytical Optimization Technologies for Games & AppsAnalytics, A/B Testing, Segmentation &

Dynamic Best-Fit

Alan Avidan, Exec. Director & Chief BeezzzDev

Page 2: Alan Avidan, Exec. Director & Chief BeezzzDev

Points We’ll Cover

What is optimizationWhat can be measured and optimized

Optimization technologies for games and apps AnalyticsA/B TestingUser SegmentationDynamic Best-Fit

Let’s get started!

Page 3: Alan Avidan, Exec. Director & Chief BeezzzDev

Optimization Family Tree

Page 4: Alan Avidan, Exec. Director & Chief BeezzzDev

What is Optimization?

Data-driven efforts formulated and designedto maximize Key Performance Indicators (KPI)by enhancing in-game/app conversions

Max Z {f(x)} ≡ f(Engagement, Retention, Monetization, Virality) X

s.t. g(x)=0, h(x)<0

Page 5: Alan Avidan, Exec. Director & Chief BeezzzDev

Which Key Performance Indicatorsshould you target for optimization?

Monetization Engagement Retention Virality

Page 6: Alan Avidan, Exec. Director & Chief BeezzzDev

Analytics and Optimization Companies

Page 7: Alan Avidan, Exec. Director & Chief BeezzzDev

Optimization Results88.9% improvement on landing page

Page 8: Alan Avidan, Exec. Director & Chief BeezzzDev

Which Game Elements Can Be Optimized?

New Features Arts (Creative) Message Wordings Game Mechanics Game Flow Landing Pages

Promotions

Page 9: Alan Avidan, Exec. Director & Chief BeezzzDev

Optimization Technologies We Use

Analytics A/B Testing (Split Testing)

User Segmentation Dynamic Best-Fit

Page 10: Alan Avidan, Exec. Director & Chief BeezzzDev

 The process of developing optimal or realistic decision recommendations based on insights derived throughthe application of statistical models and analysisagainst existing and/or simulated future data - Wikipedia

Typical uses of Analytics

Engagement Tracking Funnel Analysis

Measure, Display, Analyze, Change, Repeat

Analytics

Page 11: Alan Avidan, Exec. Director & Chief BeezzzDev

Analytics - Bottom Line Upside• Monitor, record, & display Key Performance

Indicators (KPI) • Measure effectiveness of game mechanics and

monetization Efforts • Access and display data to understand how

users interact with game/app; decide where improvements are needed

Downside The capture and storage of data, followed by

analytics and visualization is tedious, provides retroactive information about the “Average User.”

Page 12: Alan Avidan, Exec. Director & Chief BeezzzDev

A/B Testing

Credit: Steve Collins, Swrve

Page 13: Alan Avidan, Exec. Director & Chief BeezzzDev

A/B Testing Uses

New features are introduced to a selection of users, and their reactions measured. Features remain only if users engage with them - Wooga

Photo: Spencer Higgins; Illustration: Si Scott

Page 14: Alan Avidan, Exec. Director & Chief BeezzzDev

Q: A/B Testing: What are the most unexpected things people have learned from A/B tests?

Answer Wiki1.Make sure that the test is statistically significant - run it for long enough, and with enough traffic to make it count2.I have learned how dramatically, and ridiculously wrong my most basic assumptions were3.It's empirically proven that you should let the data tell you what works or not and you should constantly be testing4.That the devil is in the detail - a minor change can generate a significant result

Page 15: Alan Avidan, Exec. Director & Chief BeezzzDev

A/B Testing – Bottom line

Upside Simple; understandable; can achieve very

good results

Downside:• One size fit all

Page 16: Alan Avidan, Exec. Director & Chief BeezzzDev

User-Base Segmentation

A Priori Segmentation:• Geographic - states, regions, countries• Demographic - age, gender, education• Psychographic - lifestyle, personality, values• Positive - similar wants or needs

Clustering Segmentation:• Behavioral - similarities of behavioral patterns and like-

properties

Page 17: Alan Avidan, Exec. Director & Chief BeezzzDev

Segmentation - Uses

Cohort Analysis – Track over time users with common reference featureTargeting - Serve different treatments for each segment to maximize KPIs

Page 18: Alan Avidan, Exec. Director & Chief BeezzzDev

Segmentation – the bottom line

Upside Can be effective especially reaching out to

groups identifiable by known attributes Downside:

– Clusters are predefined and thus remain unchanged during the analysis

– Requires storage of terabytes of data– privacy issues

Page 19: Alan Avidan, Exec. Director & Chief BeezzzDev

Dynamic Best-FitReal-Time Automated Action Optimization

A predictive algorithmic technology used to serve each user the page option they are most likely to convert on at any feature point

Page 20: Alan Avidan, Exec. Director & Chief BeezzzDev

DNA Signature Attributes

Geo-Demographic attributes: age, gender, education, country

Facebook attributes: Friends, Likes, Interests, Posts, Events

Behavioral attributes: level, spending, score, progress, custom

Session attributes: time of day, day, duration

Proprietary attributes: novice, high-bidder, risk-averse

3rd Party attributes: income level, education

Page 21: Alan Avidan, Exec. Director & Chief BeezzzDev

How Dynamic Best-Fit Works

Advanced statistical algorithms find strong correlations between user DNA data

and past conversions

Page 22: Alan Avidan, Exec. Director & Chief BeezzzDev

Best-Fit Wording

Page 23: Alan Avidan, Exec. Director & Chief BeezzzDev

Best-Fit OptionsPayment Pages: Different Ranges

Page 24: Alan Avidan, Exec. Director & Chief BeezzzDev

Best-Fit OptionsPayment Pages: Different Incentives

Page 25: Alan Avidan, Exec. Director & Chief BeezzzDev

Best-Fit: Game Flows

Option 1 Option 2Open page

Full tutorial

Stage 1

Open page

Short tutorial

Stage 1

Option 3Open page

No tutorial

Stage 2

Page 26: Alan Avidan, Exec. Director & Chief BeezzzDev

Best-Fit: Payouts

Only large and less frequent

winnings*

*The sum of all winnings are the same

Mostly small but more

Frequent winnings*

Page 27: Alan Avidan, Exec. Director & Chief BeezzzDev

Best-Fit:Invite Friends - Different Layouts

Page 28: Alan Avidan, Exec. Director & Chief BeezzzDev

Best-Fit: Promotions

Go VIP

Buy VIP card for 5 EURand enjoy 30% more coins

for all future buys

Buy 1 Get 1 FREE

Receive twice theamount of goldfor regular price

Triple your money

Buy 100 GoldGet 200 Free

Page 29: Alan Avidan, Exec. Director & Chief BeezzzDev

Best-Fit Arts

Page 30: Alan Avidan, Exec. Director & Chief BeezzzDev

Dynamic Best-Fit: Results

“Total Friends” attribute as conversion indicator in payment pageInsight: users with less than 100 friends more readily reach the payment page, and moreover convert better

“Like” attribute as conversion indicator in payment pageInsight: users with more than 25% of Likes associated with apps monetize much better, and moreover clearly prefer Layout 2

Increases conversions and KPIs

Gain Valuable new insights to improve app design and user targeting

Page 31: Alan Avidan, Exec. Director & Chief BeezzzDev

Review

• Optimization is vital to your game/app’s success • Retrofit existing games and plan for future

games• Match objectives with technologies: Different

technologies have different uses; Require a different level of involvement; and produce different Uplift results

• Future? -- Lots and lots moredata. Those that will learn toharness it will succeed