OVERVIEW

When the game developers behind one of the gaming world’s most well-known franchises saw attrition rates in their MMO continuing to rise week after week, they knew they had a problem. They just didn’t know what it was. Everyone had their own theory, but without hard data to guide decision making, paralysis set in. Each team continued down the same well-worn paths without addressing the core problem.

While the marketing team pushed developers for the information they needed, the development team was simply overwhelmed by the day-to-day demands of keeping the game up and running. And even if they’d had the luxury of more time, it wasn’t entirely clear where to start. Just like any successful MMO or social networking application, the game was producing so much raw data that merely getting it into a workable format was a challenge.

More importantly, the team didn’t know the right questions to ask. While the data can tell many stories, there are only a handful that really matter. And that’s where things get really interesting.

In a span of two weeks, starting from raw data files and a few discussions with the right people, we helped this company identify and understand the factors contributing to player attrition. Today, they are able to closely monitor attrition-related data using a series of simple, targeted tools that give them the visibility they need, when they need it.

CHALLENGES

The scale of the data was the first major challenge. With more than 250,000 avatars in the game, identifying useful trends and targeted data analysis required a highly scalable and powerful system for managing all that data. The tools required to dig into such massive amounts of data were well beyond the resources that the developers had on their desktops. And while commercial tools such as Google Analytics have their place, they’re simply not adaptable for the needs of game developers.

Apart from handling massive amount of data, here are a few challenges that we think are relevant to gaming.

• When data is constantly evolving in a dynamic environment, even a week-old snapshot of the game becomes out-of-date so quickly that the insights it offered would be of little use.

• Measurement is an ongoing process, not a one-shot proposition. Making repeated measurements based on all users for the entire lifecycle of the application introduces new obstacles.

• Not knowing which information you’ll need in the future makes extracting the right data from the beginning that much more important.

• The ongoing collection and maintenance of data generated by tens of thousands of users isn’t enough—mining the data for meaningful metrics is what it’s all about in the end.

CASE STUDY

THE RULES OF ENGAGEMENTHow one of the world’s leading game companies used Socialesque to transform raw data and uninformed hunches into game-changing insights about player engagement.

WHAT WE DID

We started with a series of massive log files containing avatar names, basic activity reports, “Whispers”—messages from one avatar to another, including time stamps, “Active friends”—links between avatars, and more. Some of these files had over 1.5 million records.

From here, we analyzed the data using common—and some not-so-common—social science principles designed to uncover useful nuances in the data. As an example, while we were able to map friendships using links between avatars, we were more interested in understanding the strengths of those friendships. We created a measure of “cohesiveness” using a number of other data inputs, allowing us to more accurately gauge the real impact of social connections within the game.

The most important question for us to answer was whether or not we could assess and monitor the health of individual social networks to detect which were in danger of fizzling out. Put simply, if your friends begin to leave the game, does that mean you probably will, too? While the answer may seem self-evident, being able to determine which social networks are at risk opens the door to even richer insights. Are specific content challenges leading different networks to fail? Are there shared attributes among failing networks that can be addressed by the game? Hardware problems? Understanding these linkages is crucial to understanding audience engagement.

OUR APPROACH

Our “Measured Design” approach helps you tackle this challenge. In our view, there are three kinds of analyses or searches: naïve, primed, and focused. Naïve search is exploratory—when you don’t necessarily know exactly what you’re looking for, but are following a hunch. Primed search is when you have a general idea where something interesting might be, but you don’t know exactly what it will look like. Focused search is when you really know exactly what you’re looking for and want specific information about something—and you have a very good idea where to look.

Building sufficient infrastructure to let you measure everything without overwhelming your resources or hamstringing your analytic options is an exceptionally

complicated trifecta for most development projects. This means that you either measure less and expect mediocre analysis or you measure everything but don’t know where to look when problems arise. Our approach allows you to move fluidly between all three, using the right tools within the right context.

RESULTS TO DATE

Social network analysis is even more intensive than the traditional statistical analysis. This is partly because of the inherent network effects that must be addressed to understand the topography of the social graph. Rather than just counting ‘number of friends,’ the company now understands the true drivers of their game. Today they can monitor the health of the game using a set of advanced dashboards that we built for them. Instead of reacting to subscription drops, the company is able to predict and prevent attrition within the game—and directly improve their bottom line.

Socialesque specializes in measuring

engagement, with a focus on gaming

and social networking applications. Our

proprietary large-scale data mining and

visualization technology helps deliver the

insights developers need to make richer,

more engaging content—maximizing the

business value of virtual human networks.

We’d love to talk with you so we can help

you understand how to engage your

audience and get more from your game.

Please contact Varun Nayak at

varun@socialesque.com.

www.socialesque.com