mail online android application at droidcon - turin - italy

Mail Online Android AppMassimo Carli

Head of Mobile

Who is Massimo Carli

• Born in Rovigo (Italy)• Graduated at Padua

University • Co-founder of Mokabyte• http://www.mokabyte.it

• Teacher for Sun/Oracle• Author of the first Italian book

about Android• Head of Mobile for Mail

Online• Twitter: massimocarli

What is Mail Online

• UK-born MailOnline (www.dailymail.co.uk) is the world’s largest English-language newspaper website– over 59.7 million unique monthly visitors globally

• The third largest newspaper website in America – over 21 million unique monthly U.S. visitors (comScore,

November 2013).• Offices in London, NewYork, Los Angeles and Sidney• 700 articles and 8000 photos every day

Mail Online Android App

• One of Google’s “Best Apps of 2013” with over 1.9 million downloads

• Over 15 channels of must-read articles and pictures

• Over 600+ stories every day• Offline mode• Content related to location• Comments• Different display options

Architecture Goals

• Universal Application– Both smartphone and tablet supported– Easy evolution on TV

• Leverage on existing Android Components– Avoid reinvent the wheel– Performance improvements

• Create a common library– Optimize tools for commons stuff– Opportunity to create new things

Design Principles

• Avoid classic Antipattern– Spaghetti code– Golden Hammer– Reinvent the wheel

• Design for change• Open Close Principle– Opened for the extension– Closed for the changes

• Test first

Key Components

• Data Synchronization– Article data should be persisted to permit offline

reading– Download should NOT reduce application

responsiveness • Image download and optimization– Prevent classic Android OutOfMemory Error

• Content personalization– See WHAT the user wants to see and WHEN

High Level Architecture

User Interface

MOL ContentProvider

SQLite

getData() getMetadata()

AsyncImageView

Image ContentProvider

FS

Text Custom Components

getImage()

Article Data Synchronization

SyncAdapter

Profile Settings

ProfileData

Profile Manager

DataManager

REST Method Library

Internet

MOLContent

Rest Method Library• REST Services requests– GET, PUT, POST and DELETE supported

• Delegate result management to a Deserializer<?>• Http Error code management• Use of HttpClient and UrlConnection based on the

Api Level• Traffic measures• Executed in the caller Thread– @See Command Framework later

An Example: Get RequestRestCommand postCommand =

RestCommandBuilder.get(url).addParam(”param1”,”value 1”).addParam(”param2”,”value 2”).addHeader(”header1”,”value 1”).addHeader(”header2”,”value 2”).build();

RestCommandResult<LoginData> result = RestExecutor.get().execute(ctx, command,

loginDataDeserializer);LoginData loginData = result.getResult();

An Example: Post RequestRestCommand postCommand =

RestCommandBuilder.post(url).withStringDocument(inputJson)

.asJson().build();

RestCommandResult<LoginData> result = RestExecutor.get().execute(ctx, command,

loginDataDeserializer);LoginData loginData = result.getResult();

Deserializer<?>// Abstraction of the object who read bytes from an // InputStream and create, if possible, an instance of// type E

// It’s “Android Context” dependent

public interface Deserialiser<E> {

E realise(InputStream inputStream, Context context) throws IOException;

}

Deserializer for Stringpublic final class StringHttpDeserialiser implements HttpDeserialiser<String> {

- - -

@Override public String realise(InputStream inputStream, Context context)

throws IOException { String encodedString = null; if (TextUtils.isEmpty(mEncoding)) { encodedString = IOUtils.toString(inputStream); } else { encodedString = IOUtils.toString(inputStream, mEncoding); } IOUtils.closeQuietly(inputStream); return encodedString; }}

Deserializer for Bitmap// Deserializer implementation that reads Bitmap data from the // input stream// Warning: Possible OutOfMemoryError!!!

public final class BitmapDeserialiser implementsHttpDeserialiser<Bitmap> {

- - -

@Override public Bitmap realise(InputStream inputStream, Context context)

throws IOException { Bitmap image = BitmapFactory.decodeStream(inputStream); return image; }

}

Save Bitmap on File System// Data are read from the inputstream and written directly to // the File System. No extra memory used for Bitmap allocation

public final class FileDeserializer implements Deserializer<Void> {

@Override public Void realise(InputStream inputStream, Context context) throws IOException { // Prepare streams try { tmp = new FileOutputStream(tempDestinationFile); flushedInputStream = new BufferedInputStream(inputStream); fos = new BufferedOutputStream(tmp); IOUtils.copy(flushedInputStream, fos); } finally { // Close everything }

return null;}

}

Read data as JSon Object// Using StringDeserializer we create and return a JSONObject

public class JsonDeserializer implements Deserializer<JSONObject> { @Override public JSONObject realise(InputStream inputStream, Context context) throws IOException { final String jsonString = StringDeserializer.getDefault().realise(inputStream, context); try { final JSONObject jsonObject = new JSONObject(jsonString); return jsonObject; } catch (JSONException e) { e.printStackTrace(); } return null; }}

RestMethod GET example// We create a simple GET request for a given URL an we use // the RestExecutor to execute it. We get a RestCommandResult// object with the data or the error informations

RestCommand getCommand = RestCommandBuilder.get(url).build();

RestCommandResult<String> result = RestExecutor.get().execute(getContext(), getCommand,

int statusCode = result.getStatusCode();if (HttpStatus.SC_OK == statusCode) { String strResult = result.getResult();} else { Log.e(TAG_LOG, ”Error :” + result.getStatusMessage());}

RestMethod POST example// We send data with a POST and parse the result into a specific// object of type LoginData.LoginResponseDeserializer which is// a JSON deserializer specialisation

final LoginData.LoginResponseDeserializer deserializer = new LoginData.LoginResponseDeserializer();final RestCommand command =

RestCommandBuilder.post(loginUrlStr) .addParam("email", username)

.addParam("password”,password)

.addParam("deviceId", deviceId)

.build();final RestCommandResult<LoginData> result =

RestExecutor.get().execute(ctx, command, deserializer);

Bitmap Fetching with RestMethod// We send data with a POST and parse the result into a specific// object of type LoginData.LoginResponseDeserializer which is// a JSON deserializer specialisation

BitmapDeserializer bitmapDeserializer =

BitmapDeserializer.get();

RestCommand command = RestCommandBuilder.get(url).build();

final RestCommandResult<Bitmap> result = RestExecutor.get().execute(ctx, command, bitmapDeserializer );

// If okBitmap bitmap = result.getResult();

The Decorator Design Pattern

http://en.wikipedia.org/wiki/Decorator_pattern

Measure Traffic public class TrafficCounterDecorator<T> implements Deserializer<T> {

private Deserializer<? extends T> mDecoratee;

private long mDataCount;

public TrafficCounterDecorator(final Deserializer<? extends T> decoratee) { this.mDecoratee = decoratee; mDataCount = 0L; }

public final long getDataCount() { return mDataCount; }

public final void reset() { mDataCount = 0; }

public final Deserializer<? extends T> getDecoratee() { return mDecoratee; }}

Measure Traffic cont.public T realise(final InputStream inputStream, final Context context) throws IOException { return mDecoratee.realise(new InputStream() {

@Override public int read() throws IOException { mDataCount++; return inputStream.read(); }

@Override public int read(final byte[] buffer) throws IOException { final int dataRead = super.read(buffer); mDataCount += dataRead; return dataRead; }

@Override public int read(final byte[] buffer, final int offset, final int length) throws IOException { final int dataRead = super.read(buffer, offset, length); mDataCount += dataRead; return dataRead; }

}, context); }

Android Command Library• Framework we use to execute tasks in background • It’s based on Android Service Component– It stops when there’s anything to do

• May use Queue and priority Queue• Callback and result Management

public interface Command {

void execute(Context ctx, Intent intent);

}

Android Command Library

http://en.wikipedia.org/wiki/Command_pattern

Android Command Library// In a resource of type XML

<actions> <action name="testCommand" class="uk.co.mailonline…command.SimpleCommand"/></actions>

// In the AndroidManifest.xml

<service android:name="uk.co…service.CommandExecutorService”android:exported="false"><meta-data android:name="uk.co…

command.COMMAND_CONFIG" android:resource="@xml/commands" /></service>

A very simple Command// A simple Command

public class SimpleCommand implements Command {

public static final String TAG_LOG = SimpleCommand.class.getName();

@Override public void execute(Context ctx, Intent intent) { Log.i(TAG_LOG, "SimpleCommand executed!"); }

}

Executor// We use the normal Executor to execute the testCommand// without any parameters

public static class Executor {

public static void execute(Context ctx, String name, Bundle inputData){}

public static void executeIfConnected(Context ctx, String name, Bundle inputData){}

public static void executeAtLeastAfter(Context ctx, CommandBag commandBag,

String name, Bundle inputData){}

}

Executor// We can execute a simple Command given its name and // input Bundle (null in this case)

Command.Executor.execute(this, "testCommand", Bundle.EMPTY);

Audit Executor// We can execute a Command and get information on the// result. This is a simple Command that takes a while and// then completes

public void execute(Context ctx, Intent intent) {int times = intent.getExtras().getInt(InputBuilder.TIME_EXTRA, 1);

String message = intent.getExtras().getString(InputBuilder.MESSAGE_EXTRA);

if (TextUtils.isEmpty(message)) { message = DEFAULT_MESSAGE; } for (int i = 0; i< times ; i++) { Log.i(TAG_LOG, message + "#" + i); try{ Thread.sleep(500L); }catch(InterruptedException e){} }}

Audit Executor// We can execute a Command and get a feedback.

Bundle inputParams = InputParamsCommand.InputBuilder.create("Hello World!") .withNumberOfTimes(10).build();Command.AuditExecutor.execute(this, "simpleAuditCommand", inputParams, new CommandListener() {

@Override public void commandStarted(long commandId) { // Do something into the UI thread when the command starts mProgressBar.setVisibility(View.VISIBLE); }

@Override public void commandCompleted(long commandId, Bundle result) { // Do something into the UI thread when the command ends mProgressBar.setVisibility(View.GONE); } });

Audit Executor with Handler// If we need it we can receive the callback into a different// thread associated with a given Handler

public static class AuditExecutor {

public static long execute(final Context ctx, String name, Bundle inputData, final CommandListener

commandListener,final boolean tracked){...}

public static long execute(final Context ctx, String name,Bundle inputData, final Handler handler, final boolean tracked){...}

}

Audit Executor with Handler// If we need it we can receive the callback into a different// thread associated with a given Handler// Different WHAT are used

final android.os.Handler handler = new android.os.Handler() {public void handleMessage(Message msg) {

switch (msg.what) { case COMMAND_STARTED_WHAT: // Command starts

break;case COMMAND_PROGRESS_WHAT: // Command

progressbreak;

case COMMAND_COMPLETED_WHAT: // Command completes // Invoked when Command completes } }};

Audit with BroadcastReceiver

• The same events are notifier using a LocalBroadcastManager– Useful to manage configuration changes during

Command execution– Sometime better to use Fragment with

setRetainState(true)• Implemented for completeness

Audit Executor with Result// With an AuditCommand it’s possible to get the result of// the Command

/*** All the AuditCommand should be able to return it's result* at the end of their execution* @return The Bundle with the result of the command. Can be null.*/public Bundle getResult();

Audit Executor with Progress// The AuditCommand interface extends the Command interface// adding functionality for callback using a // LocalBroadcastManager injected by the AuditExecutor

public interface AuditCommand extends Command {

public void setLocalBroadcastManager(LocalBroadcastManager localBroadcastManager);

public void setRequestId(long requestId);

public Bundle getResult();

}

Audit Executor with Result// Into the commandCompleted() callback method we can have the// result of the Command. We can use the getResultValue() // utility method to get that information

@Overridepublic void commandCompleted(long requestId, Bundle result) {

long value = ResultAuditCommand.getResultValue(result); mProgressView.setText("Result: " + value); mProgressBar.setVisibility(View.GONE);}

Audit Executor with Progress// To manage progressState we implemented the// AbstractAuditCommand abstract class with the // setProgressingState() utility method

public class ProgressAuditCommand extends AbstractAuditCommand { - - - public void execute(Context ctx, Intent intent) {

// Read input paameters... for (int i = 0; i< times ; i++) { Log.i(TAG_LOG, message + "#" + i); sendProgressingState(getRequestId(), new Bundle()); try{ Thread.sleep(800L); }catch(InterruptedException e){} } }}

Queue Executor// Commands can be executed into a specific Queue defined into// a XML resource file// Different types of Queues for different types of data

<?xml version="1.0" encoding="UTF-8"?><queues> <queue name="commandQueue" class=“…queue.SynchronizedCommandQueue" />

<queue name="articleCommandQueue" class=”…queue.SynchronizedCommandQueue" />

<queue name="imageCommandQueue" class=”…queue.ImageCommandQueue" /></queues>

Different Queues// Commands can be executed into a specific Queue defined into// a XML resource file// Different types of Queues for different types of data

<?xml version="1.0" encoding="UTF-8"?><queues> <queue name="commandQueue" class=“…queue.SynchronizedCommandQueue" />

<queue name="articleCommandQueue" class=”…queue.SynchronizedCommandQueue" />

<queue name="imageCommandQueue" class=”…queue.ImageCommandQueue" /></queues>

Queue Executor// Commands can be executed into a specific Queue defined into// a XML resource file

public static class QueueExecutor { public static void execute(Context ctx, String queueName,

String commandName, Bundle inputData) {…}

}

ORM Android Library• We created a library to manage ORM into Android– An abstraction of ContentProvider– A way to map Entities into DB Table or

ContentProvider– Convention over configuration when possible

• Prebuilt query into XML documents• Smart way to execute query

ORM Android Library• We didn’t use it!!!• All the entities were in memory– Android Cursor implementation is a better solution– Reinvented the Wheel

• Other ORM tools have the same problems– We used Android specific APIs

• Is it useful abstract information into model entities?– Probably yes but only for better code style and for

OCP (Open Close Principle)

Images and AsyncImageView

setImageUrl()

Image Cached

showImage()

Image Not Cached

DnlImageCommand

Images and Memory

• We don’t create Bitmap instance during download• Images are Cached only if needed• We resize images based on screen size• Use of inPurgeable flag for devices < Api Level 11• If Bitmap memory needs to be reclaimed it is. If the

Bitmap needs to be reused it is re-encoded• Is not ignored for Api Level >= 11• Less memory -> more CPU -> less responsiveness

Two approaches for Article Details

• Unique Layout file with all the different sections• Easier to implement• Bad usage of memory. The user not always scroll to the bottom of the

news.• Bad responsiveness when user swipes the ViewPager

• A ListView with different types of cells• More verbose implementation• Better usage of memory• Better responsiveness when user swipes the ViewPager

Battery Usage

• Mail OnLine Android App needs to download a lot of data• 14 Channels with 50MB of data each during

synchronization• Each article can have between 3 and 100 images

• Our readers want to read articles in offline mode• We need to optimize the quantity of data based on• User preferences• Connection Type (more on WiFi, less on 3G)

• Our Command Executor needs to run only when necessary• When the queue is not empty

Tracking and Ads

• Our application is not free and needs Ads• Tracking is important for• User traffic statistics• Ads targeting

• We also use a tool for Crash reporting• Our crash rate is less that 0,03%

• All these third parties tools and SDKs make adds things our app has to manage• It’s very important to optimize everything

The ‘Probe’ Project

• To optimize the app we need to have information about the devices states• System information (memory, free space, etc.)• Find solution for specific problems

• Send notification for specific article based on user preferences• We use Profile informations

• Send sounds, images, actions, etc• Security is important

The ‘Probe’ Project

Mongo DB

Node.js

Internet

GCM

Our Projects on GitHub

• Rest Method Library• https://github.com/MailOnline/android_rest_framework

• Command Library• https://github.com/MailOnline/android_command_framework

• Stay tuned!!

Question?