TapLogger: Inferring User Inputs On Smartphone Touchscreens Using On-board Motion Sensors

Zhi Xu, Kun Bai, and Sencun Zhu

Tap Event Detection

Introduction

Reference: This poster is based on the paper “TapLogger: Inferring User Inputs On Smartphone Touchscreens Using On-board Motion Sensors," in Proc. o ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec’12)

TapLogger detects tap events by monitoring the acceleration changes (i.e. SqSum = Ax2 + Ay

2 + Az2)

Sensors equipped on a smartphone bring potential risks of leaking user’s private informationWe observe the correlations between the tapped position on the touchscreen and the motion changes of smartphones;

Attack Workflow Step 1: The user is tricked to install the TapLogger app;

Step 2: TapLogger learns the motion change patterns of tap events when the user is interacting with it;

Step 3: TapLogger runs in the background, stealthily monitor the motion changes, and uses the learnt tap event pattern to infer user inputs on touchscreens.

Note that, monitoring the readings of motion sensors requires no security permissions.

Pattern is user specific and device specific Experimental results of tap event detection Unique pattern of tap events

Tap Position Inference TapLogger infers the position tapped by monitoring the gesture changes (i.e. the readings of Roll and Pitch) Observed correlations Use extracted features to distinguish tap events

Proposed Applications With TapLogger

The training layout and target layout

Number Pad Logging Attack during the call Password Stealing Attack when unlocking the phone An example of inference

Evaluation with 20 sequences of tap inputs with length of 16

The distribution of inferred labels after entering the passwords “5 7 6 8” for 32 rounds

Evaluation with different passwords (30 rounds each)