a closed-loop system for pervasive...

A CLOSED-LOOP SYSTEM FOR PERVASIVE HEALTH

Jong Hyun Lim Defense Talk

19 October 2012 @ 2pm Maryland 214

1 Department of Computer Science

Chronic Diseases and Outcomes Increased life expectancy, increased chronic diseases

•  Developing countries: estimated $84 billion spent by 2015 [1] Traditional model of episodic care in clinic and hospital settings is not optimal. Recently spotlighted method: low-cost longitudinal monitoring using consumer medical devices


Pervasive Health Devices

The devices (mobile/static) continuously monitor peoples everyday condition.


Bayer Contour USB

Jawbone UP Withings wireless weight scale

Pervasive Health Applications

Web/Mobile apps to visualize/analyze collected data from underlying systems and devices

Fitbit Activity Monitor

Web Servers


The Characteristic of Chronic diseases Chronic diseases are complex and multi-factorial in nature (e.g., Obesity).

Pervasive health applications that can combine multiple sensing streams from devices are desirable.

Department of Computer Science 5

Challenge: Combining multiple sensor data


Activitymonitor

Weight Dietary intake & Location Device

Owners

Developers

..... .....

Applicationfor Obesity

WebServer1

WebServer2

WebServer3

Data streams

Challenge: low device utilization


Activitymonitor

Weight Dietary intake & Location Device

Owners

Developers

..... .....

Applicationfor Obesity

WebServer1

WebServer2

WebServer3

Data streams

CaregiversNo Connections to Caregivers !! (Open)

Low device utilization

Our Proposed Closed-Loop Model


Data Collection& Application Development

Activitymonitor

Weight Dietary intake & location

CaregiversProviders

All streams

Device Owners

Developers

Intervention &Event Detection

Feedback/Alerts/Triggers/Survey..... .....

Applications

•  Lim et al., A Closed-loop Approach for improving the Wellness of Low-income Elders at Home Using Game Consoles, IEEE Communications Magazine, Jan 2012.

Stovepipe Architecture [2]

An architecture of existing pervasive health systems

Activity Monitor

SQLite

Device Driver

HTTP, Security,OAuth

MySQL

Device Driver

HTTP, Security,

Basic

Wireless Weight Scale

WiFi BlueTooth

XML JSON


Impact on Application Development

Activity Monitor

SQLite

Device Driver

HTTP, Security,OAuth

MySQL

Device Driver

HTTP, Security,

Basic


WiFi BlueTooth

XML JSON

1 N

App for Obesity

Closed-Interface •  no/limited data accessibility for application

development •  At best, applications become a mixture of

heterogeneous protocols, message formats, and APIs

Low composability and maintenance challenge


Impact on Application Development (Cont’d)

Activity Monitor

MySQL

Device Driver

HTTP, Security,

Basic


WiFi

XML

App for Obesity

New Devices:

Vertical-Integration •  All systems components are tightly-

coupled for custom purpose •  To integrate heterogeneous devices,

developers need to modify all components in the system

•  Time and cost are non-negligible

System extensibility challenge


Question on Architecture What system architecture can improve application composability and system extensibility?


Our Approach Main Ideas

•  For extensibility •  Providing a set of modules to interface and integrate heterogeneous

devices easily

•  For composability •  Open, uniform data access interfaces •  Providing services that the application developers need


HealthOS*

HealthOS is a platform that consists of clients and servers to collect sensor data and support application development.

* Lim et al., HealthOS: A Platform For Pervasive Health Applications, mHealthsys, Nov 2012

App

Storage Space

Pipeline 3

Pipeline 1Adapter 2 Adapter NAdapter 1

SmartphoneGame ConsolePhysiological Monitor

HealthOS Server

Pipeline 2

Push

HealthOS Client


PC or mobile platforms

A process running in Internet (e.g., cloud)

HealthOS: Resource Exposition •  Resource: devices, users, sensing data

•  Principle: REpresentational State Transfer (REST) [3]

•  Interfaces are uniform, universally available (HTTP)

•  Resource Hierarchy


/<userID>/<device>/<deviceID>/<modality>/<command>

http://lim.cs.jhu.edu/lim/fitbit/547945/activity (HTTP request from applications)

HealthOS: Further Simplifying Application Development

RESTful Java APIs •  Serialization (converts XML resources to Java Object) •  Higher level abstraction (hides the details about communication and

security in HealthOS)


HealthOS: Sample Code


<User> <UserID>lim</UserID> <UserName> <LastName>Lim</LastName> <FirstName>Jonghyun</FirstName> </UserName> <DeviceName>Fitbit</DeviceName> <DeviceType>Activity Monitor</DeviceType> <DeviceID>547945</DeviceID> <DeviceName>Zeo Sleep Manager</DeviceName> …..

[Without API : retrieving user lim device information

1.  HTTP requests to http://lim.cs.jhu.edu/lim 2. Authentication/security

3. Parse XML Messages from HealthOS 4. Use device information

HealthOS: Sample Code


HealthOS healthos = new HealthOS(“http://lim.cs.jhu.edu", "lim", "password"); User user = healthos.getUserById("lim"); for(Device device : user.getDevices()) { System.out.println(device.getName()); System.out.println(device.getModalities()); }

[With RESTful API Code Sample]

6 lines of code

HealthOS: Applications’ Needs pervasive health applications and systems have diverse preferences/requirements for their purposes.

App

HealthOS Server

Push

Hospital EMR System

Pull

1.  Message Formats (CCD [2])

2.  Services (Push data) 3.  Protocols (FTP)

1.  Message formats (CCR [1]) 2.  Data Translation


[1] CCR (Continuity Care of Record) and [2] CCD (Continuity Care of Document)

HealthOS: Satisfying Applications’ Needs


HealthOS Server

BioHarness

Accelerometer data

Daily Activity Monitor

Database

I want step counts data in CCR XML

2. Connect database Get accelerometer data

3. Change accelerometer to step counts

4. Format in XML CCR

1. HTTP request

5. Return data in XML CCR: <Body> …. <Description> step count .. …. <Value> 20 </Value>

HealthOS: Pipeline

Reusable, configurable building blocks (called components) to support applications’ needs


App

HealthOS Server

Push

Hospital EMR System

Pipe

line

1

storage space

Pipe

line

2

Pipe

line

3components pool(HealthOS server)

assemble

HealthOS: Components in Pipeline Each components are built based on component-based software engineering (CBSE).

Component: a software entity encapsulating its implementation and interacting with each other through well-defined interfaces:

•  Each component defines provided and required interfaces and functions. •  Developers can easily reuse components.


DBConnectorCCR_translator . <<component>><<component>>

ProvDBDataCCRData

Required InterfaceProvided Interface

To client requestingdata

ReqDBData

HealthOS: Adapters Adapters: modules to interface integrate heterogeneous devices into HealthOS


Adapter 2 Adapter NAdapter 1

Exercise monitorGame exercisePhysiological monitor

HealthOS Client 1

PC

HealthOS Server

Tablet

HealthOS Client 2

Encrypted data

HealthOS: Adapter Development

{ "packet": { "sleep":[ ["time_in_deep", "uint", 2, 1], ["time_in_light", "uint", 2, 2], , …] }, "devicename":"zeo", "adaptername":"zeo_adapter”, "interface”:”Bluetooth” …. }

•  Device abstraction: JSON script describing the semantics of devices


time_in_deep time_in_light

Zeo Sleep Managers’ sleep message:

(2 bytes unsigned integer) …

HealthOS: Adapter Development (Cont’d)

Zeo ‘s Device Abstraction

Macro Scripts

[A new adapter for Zeo Sleep Manager]

Input


Output

Communication with HealthOS

Communication with Zeo

Message Parsing, Formatting and Securing data

HealthOS: Installing a New Device


{ "packet": { ”vitals": [ [”o2", "uint", 2, 1], [”heartrate", "uint", 2, 1], , …] }, "devicename":”miTag", "adaptername":”miTag_adapter”, "interface":”USBserial” }

Wireless Physiological Monitor miTag (o2, pulse, etc.)

1 Developers compose a new device abstraction

HealthOS: Installing a New Device


User “lim” with miTag ID 2 + miTag abstraction

2

3 miTag_adapter HealthOS client

HealthOS Web services

URL request from HealthOS application: https://lim.cs.jhu.edu/lim/miTag/2/vitals

5

lim_miTag_2_vitals[Database Table]

HealthOSserver

[Pipeline]lim_miTag_2_vitals

4

HealthOS: Management HealthOS provides device owners and caregivers additional management benefits.

RESTful API

LookingGlassCaregiver,

Data Owner

HealthOS....

Caregivers, Data Owner

O2, Pulse Weight Balance O2, Pulse Weight Balance

1 2 3 N.... 1(security and

authentication methods)

[Stovepipe systems] [HealthOS]


Intervention and Event Detection


HealthOS

activitylevel

weighttrend

dietary intake, location

CaregiversProviders

all streams

Device Owners

Developers

InterventionEvent Detection

feedback/alerts/triggers/survey..... .....

applications

Our Pervasive Health Study Using Game Consoles

Server

Elders' Residence

Balance &Weight Exercise

Game

Elders Clinicians'

Data Internet

Exercise

Interventions: visit or call

(Installed fortwo weeks)


Study goal: Improve lower body strength of 21 elders to prevent fall

Game design: users’ ability and motivator

Result: mean score of lower body function improvement: 24%

Problems Low device utilization

•  48.7% (for 16 participants who played the game more than 3 times)

No automatic way to detect and notify of events •  For example, when participants did not use devices yesterday, send

reminder.


DailyAlert* •  A mobile persuasion toolkit consisting of smartphone clients and

a Web server. •  Alert: A message generated by DailyAlert server to notify, remind, feedback,

and survey device users.

•  Alert Model: •  AlertTrigger (when): when to generate an alert. •  AlertFormat (how): a composite document built by using Open Data Kit

(ODK). [2]

AlertTrigger

Schedule-driven

Event-driven

Conditional

Message

Form

Text

Audio

Image

Periodic

AlertFormat

Immediate

...

* Zhan et al., DailyAlert: A Generic Mobile Persuasion Toolkit for Smartphones, PhoneSense 2011


DailyAlert Use Case

User: user-friendliness and Controllability

Caregivers: alert composability

3. Render the form in DailyAlert mobile

<Survey name=”survey1”> <Question> <Header>Is the pain</Header> <Options> <Option>very bad</Option> <Option>bad</Option> <Option>not too bad</Option> </Options> </Question>

2. Import XML (and Schema) in DailyAlert server and set AlertTrigger (everyday)


1. Compose an alert using ODK GUI and export the

alert to XML

Data Composability for Caregivers What DailyAlert lacks?

•  Caregivers’ composability on collected data •  e.g., “if weight > 200 lbs, generate alerts” (threshold-based)

•  Ability to describe disease models in the system for sophisticated data analysis and detection


Trend Finder: Design

DetectionAlgorithm

Rule Definition(based on a statistical

model or rule)

Trend Finder

Estimated Parameters

Input: sensor measurements

Output: alerts/notifications

Input: users' requirements

Trend Finder is a system to plug disease models and provide caregivers (data consumer) better composability.

•  Detection algorithm: known mathematical models to plug in the system

•  Rule Definition: a component to define on how to use collected data.


Trend Finder: Early Disease Detection

Input: weight measurement from HealthOS

1 Output: µ (an estimated parameter)

3

A rule based on a model: 10% weight loss in the past month e.g., µ(1) - µ(30) ≥ 0.1 µ(1), generate an alert

4

Department of Computer Science

Weight loss in Chronic Obstructive Pulmonary Disease (COPD) study •  Abnormal weight loss is indicative of high risk of having COPD.

Linear regression component

Time window (start date, end date) 2

3.3 Regression Methods 31

the computations will be done by statistical software and we will not pursue otherexpressions for and here.

Example

Consider the random walk process that was shown in Exhibit 2.1. Suppose we (mistak-enly) treat this as a linear time trend and estimate the slope and intercept byleast-squares regression. Using statistical software we obtain Exhibit 3.1.

Exhibit 3.1 Least Squares Regression Estimates for Linear Time Trend

> data(rwalk)> model1=lm(rwalk~time(rwalk))> summary(model1)

So here the estimated slope and intercept are = 0.1341 and = 1.008, respec-tively. Exhibit 3.2 displays the random walk with the least squares regression trend linesuperimposed. We will interpret more of the regression output later in Section 3.5 onpage 40 and see that fitting a line to these data is not appropriate.

Exhibit 3.2 Random Walk with Linear Time Trend

> win.graph(width=4.875, height=2.5,pointsize=8)> plot(rwalk,type='o',ylab='y')> abline(model1) # add the fitted least squares line from model1

Estimate Std. Error t value Pr(>|t|)

Intercept 1.008 0.2972 3.39 0.00126

Time 0.1341 0.00848 15.82 < 0.0001

^0

^1

^1

^0

! !!

!

!

!

!!

!!

! ! !!

! !!

!

!

!!

! !

!!

! !!

!!

!

!

! !

!!

!

! !! !

!

! !!

!!

!

!

! !

!

!

!

!! !

! !!

Time

y

0 10 20 30 40 50 60

!20

24

68

µ(lbs)

(date)

140

150

130 9/30

36

Trend Finder: Personal Health/Exercise Coach

37

* Pictures from APDM, Inc

Trend Finder Data from

HealthOS

1.  Gait, posture analysis algorithm: sit-to-stand, 7 meter walk-away performance, cadence, stride velocity, stride length, and etc.

2. Detection Rules: if stride velocity < 30cm/s or turning speed > 3 second, generate an alarm to me.

Too many turns and slopes, modify routes

John (patient, lower body limitation) Accelerometers, GPS

Matt (doctor)

Designing a personalized exercise routine For John

Department of Computer Science

Trend Finder: Fault Detection Fault

•  User fault: Users’ unexpected behavior or environment affecting the quality and state of measurements

•  Software and hardware faults: Anomalies in hardware or software


Trend Finder: Fault Detection Approach

•  Data-Driven: the data that the devices report is the primary source for detecting the presence of fault. Model data characteristics, such as spikes, glitches, and stuck-at value readings.

•  Incremental: use extra devices and combine multiple modalities.


Time(date)

EKG (stuck-at) EKG

Unusually high heart rate

Activity monitor

Summary


HealthOS

activitylevel

weighttrend


Caregivers/Providers

all streams

Device Owners

Developers

DailyAlertTrend Finder


Application

•  Combine multiple streams using adapters/pipeline/API easily

•  Focus on visualization

Summary (Cont’d)


HealthOS

activitylevel

weighttrend



all streams

Device Owners

Developers



Application

•  Find correlation among data •  In-depth user data analysis •  Easily compose/deliver alerts •  Detect events of interest •  Diminished management work

Summary (Cont’d)


HealthOS

activitylevel

weighttrend



all streams

Device Owners

Developers



Application

•  Timely triggers/feedback •  Increased device utility •  Connectedness •  Selectively secure and share data •  Improved health conditions

Acknowledgements •  Andreas Terzis •  Scott Smith •  Sarah Szanton (JHU School of Nursing) •  Gerald Masson •  Michelle Carson (JHU School of Public Health) •  Laura Fisher (JHU School of Nursing) •  I-Jeng Wang (JHU Applied Physics Lab) •  Cathy Thornton and Debbie DeFord •  Our HiNRG members

•  Razvan, Mike, Jayant, John, Yin, Marcus, Doug, Andong, Da, Victor (sorted by time each person joined, not by personal preference)

•  Nayoung Alice Kim


References


•  [1]: WHO, Noncommunicable diseases country profiles 2011.

•  [2] Estrin et al., Open mHealth Architecture: An Engine for Health Care Innovation. Science, 2010

•  [3] Roy Fielding., Architectural Styles and the Design of Network-based Software Architectures, doctoral dissertation, 2000

•  [4] Ko et. al., MEDiSN: Medical Emergency Detection in Sensor Networks, ACM Transactions on Embedded Computing Systems (TECS), Special Issue on Wireless Health Systems, 2010

References (cont’d)


•  [5] ABI Research. Wearable sports and fitness devices will hit 90 million shipments in 2017, 2012. (slide 3: 90 million wearable fitness devices in 2017)

•  [6] Szanton et al., Increasing Balance in disabled older adults using a Wii-based exer-game: a pilot study of feasibility and acceptability, Gait and Posture (in submission)

•  [7] Bishop & Associates Inc., Wireless Technologies Drive Use of Mobile Devices in Healthcare, 2012. (50 million wireless health devices in 2017 [1])

•  [8] WHO, Obesity: Preventing and Managing the Global Epidemic. Technical Report Series, No 894, 2009

•  [9] Guralnik et al., A short physical performance battery assessing lower extremity function:

association with self-reported disability and prediction of mortality and nursing home admission, J. Gerontology, 1994 (SPPB)

References (cont’d) •  [10] C.R. Lyles et al., Qualitative evaluation of a mobile phone and web-based collaborative

care intervention for patients with type 2 diabetes, Diabetes technology & therapeutics, 2011 (smartphone frustration)

•  [11] Open Data Kit, http://opendatakit.org/


a closed-loop system for pervasive...

Documents