activity recognition using cell phone accelerometersgweiss/papers/weiss... · 2010-07-22 · july...

July 25, 2010 SensorKDD 2010 1

Activity Recognition Using Activity Recognition Using Cell Phone AccelerometersCell Phone Accelerometers

Jennifer Kwapisz, Gary Weiss, Samuel MooreJennifer Kwapisz, Gary Weiss, Samuel MooreDepartment of Computer & Info. ScienceDepartment of Computer & Info. Science

Fordham UniversityFordham University

July 25, 2010 2SensorKDD 2010

We are Interested in WISDMWe are Interested in WISDM

WISDM: WISDM: WIrelessWIreless Sensor Data MiningSensor Data MiningPowerful portable wireless devices are becoming Powerful portable wireless devices are becoming common and are filled with sensors common and are filled with sensors Smart phones: Android phones, iPhoneSmart phones: Android phones, iPhoneMusic players: iPod TouchMusic players: iPod Touch

Sensors on smart phones include:Sensors on smart phones include:Microphone, camera, light sensor, proximity sensor, Microphone, camera, light sensor, proximity sensor, temperature sensor, GPS, compass, temperature sensor, GPS, compass, accelerometeraccelerometer


AccelerometerAccelerometer--Based Activity Based Activity RecognitionRecognition

The ProblemThe Problem: use accelerometer data to : use accelerometer data to determine a userdetermine a user’’s activitys activityActivities include:Activities include:

Walking and joggingWalking and joggingSitting and standingSitting and standingAscending and descending stairsAscending and descending stairsMore activities to be added in future workMore activities to be added in future work


Applications of Activity RecognitionApplications of Activity Recognition

Health ApplicationsHealth ApplicationsGenerate activity profile to monitor overall type and Generate activity profile to monitor overall type and quantity of activityquantity of activityParents can use it to monitor their childrenParents can use it to monitor their childrenCan be used to monitor the elderlyCan be used to monitor the elderly

Make the device contextMake the device context--sensitivesensitiveCell phone sends all calls to voice mail when joggingCell phone sends all calls to voice mail when joggingAdjust music based on the activityAdjust music based on the activity

Broadcast (Broadcast (FacebookFacebook) your every activity) your every activity


Our WISDM PlatformOur WISDM Platform

Platform based on Android cell phonesPlatform based on Android cell phonesAndroid is GoogleAndroid is Google’’s open source mobile computing OSs open source mobile computing OSEasy to program, free, will have a large market shareEasy to program, free, will have a large market share

Unlike most other work on activity recognition:Unlike most other work on activity recognition:No specialized equipmentNo specialized equipmentSingle device naturally placed on body (in pocket)Single device naturally placed on body (in pocket)


Our WISDM PlatformOur WISDM Platform

Current research was conducted offCurrent research was conducted off--linelineData was collected and later analyzed offData was collected and later analyzed off--lineline

In future our platform will operate in realIn future our platform will operate in real--timetimeIn June we released realIn June we released real--time sensor data time sensor data collection app to Android marketplacecollection app to Android marketplace

Currently collects accelerometer and GPS dataCurrently collects accelerometer and GPS data


AccelerometersAccelerometers

Included in most smart phones & other devicesIncluded in most smart phones & other devicesAll Android phones, iPhones, iPod Touches, etc.All Android phones, iPhones, iPod Touches, etc.TriTri--axial accelerometers that measure 3 dimensionsaxial accelerometers that measure 3 dimensions

Initially included for screen rotation and Initially included for screen rotation and advanced game playadvanced game play


Examples of Raw DataExamples of Raw Data

Next few slides show data for one user over a Next few slides show data for one user over a few seconds for various activitiesfew seconds for various activitiesCell phone is in userCell phone is in user’’s pockets pocketEarthEarth’’s gravity is registered as accelerations gravity is registered as accelerationAcceleration values relative to axes of the Acceleration values relative to axes of the device, not Earthdevice, not Earth

In theory we can correct this given that we can In theory we can correct this given that we can determine orientation of the devicedetermine orientation of the device


StandingStanding


SittingSitting


WalkingWalking


JoggingJogging


Descending StairsDescending Stairs


Ascending StairsAscending Stairs


Data Collection ProcedureData Collection Procedure

UserUser’’s move through a specific courses move through a specific coursePerform various activities for specific timesPerform various activities for specific timesData collected using Android phonesData collected using Android phonesActivities labeled using our Android appActivities labeled using our Android app

Data collection procedure approved by Data collection procedure approved by Fordham Institutional Review Board (IRB)Fordham Institutional Review Board (IRB)Collected data from 29 usersCollected data from 29 users


Data PreprocessingData Preprocessing

Need to convert time series data into examplesNeed to convert time series data into examplesUse a 10 second example duration (i.e., window)Use a 10 second example duration (i.e., window)

3 acceleration values every 50 ms (600 total values)3 acceleration values every 50 ms (600 total values)Generate 43 total featuresGenerate 43 total features

Ave. acceleration each axis (3)Ave. acceleration each axis (3)Standard deviation each axis (3)Standard deviation each axis (3)Binned/histogram distribution for each axis (30)Binned/histogram distribution for each axis (30)Time between peaks (3)Time between peaks (3)Ave. resultant acceleration (1)Ave. resultant acceleration (1)


Final Data SetFinal Data SetID Walk Jog Up Down Sit Stand Total1 74 15 13 25 17 7 151 2 48 15 30 20 0 0 113 3 62 58 25 23 13 9 190 4 65 57 25 22 6 8 183 5 65 54 25 25 77 27 273 6 62 54 16 19 11 8 170 7 61 55 13 11 9 4 153 8 57 54 12 13 0 0 136 9 31 59 27 23 13 10 163 10 62 52 20 12 16 9 171 11 64 55 13 12 8 9 161 12 36 63 0 0 8 6 113 13 60 62 24 15 0 0 161 14 62 0 7 8 15 10 102 15 61 32 18 18 9 8 146 16 65 61 24 20 0 8 178 17 70 0 15 15 7 7 114 18 66 59 20 20 0 0 165 19 69 66 41 15 0 0 191 20 31 62 16 15 4 3 131 21 54 62 15 16 12 9 168 22 33 61 25 10 0 0 129 23 30 5 8 10 7 0 60 24 62 0 23 21 8 15 129 25 67 64 21 16 8 7 183 26 85 52 0 0 14 17 168 27 84 70 24 21 11 13 223 28 32 19 26 22 8 15 122 29 65 55 19 18 8 14 179

Sum 1683 1321 545 465 289 223 4526% 37.2 29.2 12.0 10.2 6.4 5.0 100


Data Mining StepData Mining Step

Utilized three WEKA learning methods Utilized three WEKA learning methods Decision Tree (J48)Decision Tree (J48)Logistic RegressionLogistic RegressionNeural NetworkNeural Network

Results reported using 10Results reported using 10--fold cross validationfold cross validation


Summary ResultsSummary Results


J48 Confusion MatrixJ48 Confusion Matrix

20817214Stand

32703204Sit

21258921399Down

221073232388Up

111216127516Jog

028272141513WalkActual

Class

StandSitDownUpJogWalk

Predicted Class


ConclusionsConclusions

Able to identify activities with good accuracyAble to identify activities with good accuracyHard to differentiate between ascending and Hard to differentiate between ascending and descending stairs. To limited degree also looks like descending stairs. To limited degree also looks like walking.walking.Can accomplish this with a cell phone placed Can accomplish this with a cell phone placed naturally in pocketnaturally in pocketAccomplished with simple features and standard Accomplished with simple features and standard data mining methodsdata mining methods


Related WorkRelated Work

At least a dozen papers on activity recognition using At least a dozen papers on activity recognition using multiple sensors, mainly accelerometersmultiple sensors, mainly accelerometers

Typically studies only 10Typically studies only 10--20 users20 usersActivity recognition also done via computer visionActivity recognition also done via computer visionActigraphyActigraphy uses devices to study movementuses devices to study movement

Used by psychologists to study sleep disorders, ADDUsed by psychologists to study sleep disorders, ADDA few recent efforts use cell phonesA few recent efforts use cell phones

Yang (2009) used Nokia N95 and 4 usersYang (2009) used Nokia N95 and 4 usersBrezmesBrezmes (2009) used Nokia N95 with real(2009) used Nokia N95 with real--time recognitiontime recognition

One model per user (requires labeled data from each user)One model per user (requires labeled data from each user)


Future WorkFuture Work

Add more activities and usersAdd more activities and usersAdd more sophisticated featuresAdd more sophisticated featuresTry timeTry time--series based learning methodsseries based learning methodsGenerate results in real timeGenerate results in real timeDeploy higher level applications: activity profilerDeploy higher level applications: activity profiler


Other WISDM ResearchOther WISDM Research

Cell PhoneCell Phone--Based Biometric identificationBased Biometric identification11

Same accelerometer data and same generated features but Same accelerometer data and same generated features but added 7 users (36 in total)added 7 users (36 in total)If we group all of the test examples from one cell phone and If we group all of the test examples from one cell phone and apply majority voting, achieve 100% accuracyapply majority voting, achieve 100% accuracyCan be used for security or automatic personalizationCan be used for security or automatic personalization

Interested in GPS Interested in GPS spatiospatio--temporal data miningtemporal data mining

1 Kwapisz, Weiss, and Moore, Cell-Phone Based Biometric Identification, Proceedings of the IEEE 4th International Conference on Biometrics: Theory, Applications, and Systems (BTAS-10), September 2010.

July 25, 2010 SensorKDD 2010 25

Thank YouThank You

Questions?Questions?

activity recognition using cell phone accelerometersgweiss/papers/weiss... · 2010-07-22 · july...

Documents