united states patent patent us 8,011,229 b2 lieberman et al

mu uuuu ui iiui imi uui imi uiii um um um uuii uu uii mi

(12) United States PatentLieberman et al.

(54) DETERMINING POSTURAL STABILITY

(75) Inventors: Erez Lieberman, Cambridge, MA (US);Katharine E. Forth, Houston, TX (US);William H. Paloski, League City, TX(US)

(73) Assignees: Massachusetts Institute of Technology,Cambridge, MA (US); President andFellows of Harvard College,Cambridge, MA (US)

(*) Notice: Subject to any disclaimer, the term of thispatent is extended or adjusted under 35U.S.C. 154(b) by 15 days.

(21) Appl. No.: 12/323,912

(22) Filed: Nov. 26, 2008

(65) Prior Publication Data

US 2009/0260426 Al Oct. 22, 2009

Related U.S. Application Data

(60) Provisional application No. 60/990,817, filed on Nov.28, 2007.

(51) Int. Cl.

G0IMI/00 (2006.01)(52) U.S. Cl . ...................................................... 73/65.01(58) Field of Classification Search ....... 73/65.01-65.09

See application file for complete search history.

(56) References Cited

U.S. PATENT DOCUMENTS

5,209,240 A 5/1993 Jain et al.5,263,491 A * 11/1993 Thornton ...................... 600/587

5,388,591 A 2/1995 De Luca et al.

5,790,256 A 8/1998 Brown et al.

(1o) Patent No.: US 8,011,229 B2(45) Date of Patent: Sep. 6, 2011

5,919,149 A 7/1999 Allum5,987,982 A * 11/1999 Wenman et al. ........... 73/379.086,063,046 A 5/2000 Allum6,165,143 A 12/2000 van Lummel6,273,863 B18/2001 Avni et al.6,280,392 B1 * 8/2001 Yoshimi et al . ............... 600/5346,741,911 B2 * 5/2004 Simmons ...................... 700/2457,141,026 B2 * 11/2006 Aminian et al . .............. 600/595

2004/0173220 Al 9/2004 Harry et al.2006/0194178 Al8/2006 Goldstein2008/0078030 Al * 4/2008 Lee et al . .......................... 5/6162008/0108913 Al* 5/2008 Lengsfeld et al . ............ 600/5952009/0137933 Al* 5/2009 Lieberman et al. ........... 600/5952009/0187129 Al* 7/2009 Ben-Galim et al . ............ 602/1820 09/02 10 093 Al* 8/2009 Jacobsen et al ............... 700/260

(Continued)

FOREIGN PATENT DOCUMENTSAU 199461875 11/1993

(Continued)

OTHER PUBLICATIONS

Ishida, A and Miyazaki, S., Maximum Likelihood Identification of aPosture Control System, IEEE Transactions ofBiomedical Engineer-ing, vol. 34 (1987), pp. 1-5.

(Continued)

Primary Examiner Max Noori(74) Attorney, Agent, or Firm Proskauer Rose LLP

(57) ABSTRACT

A method for determining postural stability of a person caninclude acquiring a plurality of pressure data points over aperiod of time from at least one pressure sensor. The methodcan also include the step of identifying a postural state foreach pressure data point to generate a plurality of posturalstates. The method can include the step of determining apostural state of the person at a point in time based on at leastthe plurality of postural states.

23 Claims, 17 Drawing Sheets

200

HAND HELD DEVICE 240

TITY

SINGLE

220b

US 8,011,229 B2

U.S. PATENT DOCUMENTS2009/0247909 Al * 10/2009 Mukumoto ................... 600/592

FOREIGN PATENT DOCUMENTSCN 1 7718 86 5/2006FR 2 873 917 2/2006KR 2001 011583 7/1999KR 100599501 7/2006RU 2 185 094 7/2002RU 2 257 143 7/2005RU 2004 106 030 8/2005RU 2 266 705 12/2005SU 1126285 11/1984SU 1463220 3/1989TW 240624 10/2005WO 95/27185 10/1995WO 97/40471 10/1997WO 01/95980 12/2001WO 2007/094598 8/2007WO 2008/058048 5/2008

OTHER PUBLICATIONS

Nashner, L.M. and Peters, J.F., Dynamic Posturography in the Diag-nosis and Management of Dizziness and Balance Disorders,Neurologics Clinics, vol. 8, No. 2 (May 1990), pp. 331-349.Duarte, M. et al., Stabilographic Analysis ofUnconstrained Standing,Ergonomics, vol. 43, No. 11 (2000) pp. 1824-1839.Samuel, Eugenie, "Walking on Shaky Shoes", New Scientist (Nov. 2,2002) p. 22.Loram, I.D. and Lakie, M., Human Balancing of an Inverted Pendu-lum: Position Control by Small, Ballistic-Like, Throw and Catch

Movements, Journal of Physiology, vol. 540.3 (2002), pp. I 111-1124.Priplata, Atilla et al., "Noise-Enhanced Human Balance Control"Physical Review Letters, vol. 89, No. 23 (Dec. 2, 2002) pp. 238101-238104to 238101-238104.Moss, Frank et al. `Balancing the Unbalanced" Nature, vol. 425(2003) pp. 911-912.Tossavainen et al., "Development of Virtual Reality Stimuli for ForcePlatform Posturography", International Journal of MedicalInformatics, vol. 70, No. 2-3 (Jul. 1, 2003) pp. 277-283.Jeka, J et al., Controlling Human Upright Posture: Velocity Informa-tion Is More Accurate Than Position or Acceleration, Journal ofNeurophysiology, vol. 92(4) (2004), pp. 2368-2379.Tsuruoka,Y. and Shibasaki, R., Spectral Analysis in Walking Balanceby Elderly Subjects, Proceedings of the 28`h IEEE EMBS AnnualInternational Conference, (2006) pp. 5420-5423.Bamberg, Stacy et al., "Development of a Quantitative In-Shoe Mea-surement System for Assessing Balance: Sixteen-Sensor Insoles",Proceedings of the 28 `h IEEE EMBS Annual International Confer-ence, (2006) pp. 6041-6044.Forth, K.E. et al., Age Associated Differences in Postural Equilib-rium Control: A Comparison Between EQscore and Minimum Timeto Contact (TTC(min)), Gait and Posture, vol. 25(1) (2007), pp.56-62.Rasku et al., "Recognition of Balance Signals Between HealthySubjects and Otoneurological Patients With Hidden Markov Mod-els", Biomedical Signal Processing and Control, vol. 2, No. 1(Jan. 2,2007), pp. 1-8.International Search Report and Written Opinion for InternationalPatent Application No. PCT/US2008/084864 dated Oct. 5, 2009.

* cited by examiner

U.S. Patent

Sep. 6, 2011

Sheet 1 of 17

US 8,011,229 B2

110a

110b

sOUTPUTMODULE

(:J,-- 130a

I

s

STABILITYPROCESSINGMODULE150

sOUTPUTMODULE130b

^b

sCOMMUNI-

CATIONMODULE140

100

FIG. 1

200

210

^szP


DISPLAY MODULE242

220b

220a

FIG. 2

COMM.MODULE

244

STABILITYPROCESSING

MODULE246

U.S. Patent Sep. 6, 2011 Sheet 2 of 17 US 8,011,229 B2


31

DISPLAY VIBRATORSVIBRATORS

MODULE368

368 360

SPEAKER365

STABILITYSTABILITY

PROCESSING MANAGEMENT

MODULEMODULE

340350

355c

355a 300

355d

355b QJ— 320c

320d

320b

FIG. 3

320a


410

400

r


STABILITY COMM.PROCESSING MODULEMODULE 424422 ----

430

412c

412a 412b

FIG. 4

512

522a 522b

FIG. 5


IELD DEVICE 530

)D POSTURALSTANCEL

532

TYSING COMM.

'EMODULE

536

U.S. Patent Sep. 6, 2011 Sheet 6 of 17

US 8,011,229 B2

STABILITY STABILITYPROCESSING MANAGEMENT

610

MODULE MOD30ULE620 6 s

614c

600

614a

614b

6

612b

FIG. 6

U.S. Patent Sep. 6, 2011 Sheet 7 of 17

US 8,011,229 B2

Receive PressureInformation from

Sensor710

700

Store PressureInformation

740

Determine aCurrent Postural

State720

Determine a NextPostural State

730

Transmit Messagewith Next Postural

State760

Determine Rangeof Postural

Stability750

FIG. 7


Receive PressureInformation from

Sensor810

Determine aCurrent Postural

State820

Determine a NextPostural State

830

Stable Check NextPostural State

860

Unstable

Adjust WeightDevice

870

800

FIG. 8


FIG. 9

tt7

mLn6

Nt

C5


1025

1035 `r : ► ` ^.

ryt iw w :..

1030 l

r°i

1005

1001

hl-^

^ tU0 0 G7

FIG. 1.0

0

CD r-r4

e

A

FIG. 11


g

1230D ^r*^

1

1210

1235A

—1205

u}Ln

cs

Irru^

230E40UC3 X

C?


0)Lnci

cothC:3

12358

% s3M

1220+ *r

fnt^"1

coEh

^7

FIG. 12


. r

i ♦ ^^ I

* ^ ^ r • ^^^ ^ ice`.. ^

Eq • t a ^!^ 1 *.

an.

i I

y

^

^ .I^.^il ". s^F11Mli ^ i i !A

i ;ty^i T1YN

b ^ ^w f •

i •i

LncoLn

cy

cocoLf S0

Lnrl-Ln

C,

La 7 XM

•

•

LnLO

1305

C3UDLn

Q

3

Ln

Ln

cor-

lidI--

r-A

C? C] Ca C^ C'^ C7

FIG. 13


1405

* ,+' 1415

s

r

1401

1405

p!

w

sf

4

0kC5

LnLnc;

0LnCi

[t1

b

0.Itci

C'

0'$ V rt 0 (31 00114 1-4 114 (5 Q

A

FIG. 14

1513

1508

nL5O (

X

FIG. 15A

Y

1514



1520Acquire pressure data points from at

least one pressure sensor over aperiod of time (e.g., t_i, €_i+1 ... t_i+n)

1525Determine postural state for eachpressure data point to generate

plurality of postural states

1530Determine postural state at a different

point in time, (e.g., t_i+n+1)

1535Determine person/subject isposturaliy stable/unstable

,

Feedback to person/subject /1540

FIG. 15B

c-Lu

a

z


Sample Data - 120s

0 14

ss• s +10

• 4, 0

,Eyes Open ,Eyes Closed

1605

1610

FIG. 16

1601

US 8,011,229 B21

DETERMINING POSTURAL STABILITY

CROSS-REFERENCE TO RELATEDAPPLICATION

This application claims the benefit of and priority to U.S.provisional patent Application No. 60/990,817 filed Nov. 28,2007, which is owned by the assignee of the instant applica-tion and the disclosure of which is incorporated herein byreference in its entirety.

GOVERNMENT SUPPORT

This invention was made with government support undergrant number E001001 awarded by NASA. The governmenthas certain rights in this invention.

FIELD OF THE INVENTION

The invention generally relates to determining posturalstability. Specifically, the invention relates to a method andapparatus/system for determining a subject's postural state.

BACKGROUND OF THE INVENTION

Poor posture can lead to postural instability (e.g., lack ofbalance), for example, as a person ages and/or when theperson is injured. Other causes for postural instability caninclude the return of a person from a zero gravity environ-ment, a lack of exercise, and/or an injury. Detection andcorrection of a subject's (e.g., a person's) postural instabilitycan be challenging outside of a lab environment and/or on areal-time basis.

SUMMARY OF THE INVENTION

In one aspect, the invention features a method for deter-mining postural stability of a person and can include the stepof acquiring a plurality of pressure data points (e.g., pressureinformation) over a period of time from at least one pressuresensor. The method can include the step of identifying apostural state for each pressure data point to generate a plu-rality of postural states. The method can also include the stepof determining a postural state of the person at a point in timebased on at least the plurality of postural states.

In another aspect, the invention features a method for deter-mining a postural stability (e.g., of a person). The method caninclude the step of acquiring at least a first pressure data pointand a second pressure data point from at least one pressuresensor. The method can also include the step of identifying afirst postural state and a second postural state based on thefirst and second pressure data points. The method can alsoinclude the step of determining a postural state (e.g., posturalstate of the person) at a point in time based on at least the firstpostural state and the second postural state.

In yet another aspect, the invention features a system fordetermining a postural stability of a person. The system caninclude at least one pressure sensor coupled to the person thatacquires a plurality of pressure data points over a period oftime. The system can also include a means for identifying apostural state for each pressure data point and a means forgenerating a plurality of postural states of the person over theperiod of time. The system can also include a means fordetermining a postural state of the person at a point in timebased on the plurality of postural states.

In one aspect, the invention features a method to determinepostural stability. The method can include receiving pressure

2information (e.g., pressure data points) from a sensor coupledto a load bearing structure. A current postural state for astructure associated with the load bearing structure can bedetermined based on the received pressure information. A

5 next postural state of the structure canbe determined based ona range of postural stability, the current postural state, a prob-ability of the next postural state or any combination thereof.

In another aspect, the invention features a computer pro-

10 gram product to determine postural stability. The computerprogram product can be tangibly embodied in a computer ora removable storage device. The computer program productcan include instructions being operable to cause a data pro-cessing apparatus to receive pressure information from a sen-

15 sor coupled to a load bearing structure. A current posturalstate for a structure associated with the load bearing structurecan be determined based on the received pressure informa-tion. A next postural state of the structure can be determinedbased on a range of postural stability, the current postural

20 state, a probability of the next postural state or any combina-tion thereof.

In yet another aspect, the invention features a system fordetermining postural stability. The system includes a stabilityprocessing module. The stability processing module can be

25 configured to receive pressure information from a sensorcoupled to a load bearing structure. The stability processingmodule can be further configured to determine a current pos-tural state for a structure associated with the load bearingstructure based on the received pressure information. The

30 stability processing module can be further configured todetermine a next postural state of the structure based on arange of postural stability, the current postural state, a prob-ability of the next postural state, or any combination thereof.

In another aspect, the invention features a system to deter-35 mine postural stability. The system includes a means for

receiving pressure information from a sensor coupled to aload bearing structure. The system further includes a meansfor determining a current postural state for a structure asso-ciated with the load bearing structure based on the received

40 pressure information. The system further includes a meansfor determining a next postural state of the structure based ona range of postural stability, the current postural state, a prob-ability of the next postural state, or any combination thereof.

In other examples, any of the aspects above, or any appa-45 ratus or method described herein, can include one or more of

the following features.A range of postural stability states can include or be deter-

mined by the plurality of pressure data points. In someembodiments, a postural state of the person is at least one of

50 a static postural state or a dynamic postural state. A dynamicpostural state can be defined as when the person is movingfrom a first static postural state to a second static posturalstate. A person can be identified or determined to be postur-ally stable or posturally unstable based on a number of times

55 the person is in the dynamic postural state.In some embodiments, a plurality of postural states follows

a punctuated equilibrium where a continuous series of staticpostural states defines an equilibrium. In some embodiments,a person can be identified or determined to be posturally

60 stable or unstable based on a number of distinct equilibria.Determining a postural state of the person at the point in

time can be based on, at least, a probability of transitioningbetween the static postural state and the dynamic posturalstate. In some embodiments, the probability of transitioning

65 between the static postural state and the dynamic posturalstate can be calculated based on the plurality of postural statesof the person over the period of time (e.g., by looking at the

US 8,011,229 B23

trend of how the plurality of postural states varies between,for example, static or dynamic postural states).

In some embodiments, an acceleration of the person can beacquired over the period of time. A location of the person canalso be determined/acquired by a sensor (e.g., GPS locationdevice).

In some embodiments, a postural state of a person at a pointin time can be determined by using a machine learning tech-nique (e.g., a Hidden Markov Model or Bayesian segmenta-tion) on the plurality of pressure data points to generate theplurality of postural states. A postural state of a person at apoint in time can be determined by using a Hidden MarkovModel on the plurality of pressure data points to determine apostural state of a person at a point in time (e.g., based onpressure points and information acquired to generate a plu-rality of postural states). A Hidden Markov Model can be usedon the plurality of pressure data points to, for example, proj ector calculate a postural state of a person at a later point in time.A postural state of a person at a point in time (e.g., a future/subsequent postural state) can be determined by applyingBayesian segmentation to the plurality of pressure data points(e.g., which can be used to generate the plurality of posturalstates).

In some embodiments, a plurality of pressure data pointsare acquired from at least one pressure sensor on a shoe wornby the person, a sock, a sole insert, a cane, a crutch, a walker,a walking aid used by a person, a prosthetic leg, a robotic leg,a vehicle, or an axle connected to at least one wheel.

Each of the plurality of pressure data points can reflect alocation of a center of mass or a center of gravity/force of theperson at a point in time. A change of the location of the centerof mass of the person can be determined over the period oftime. In some embodiments, identifying a pressure data pointas corresponding to a static or dynamic postural state can bedone by grouping a selected set of the plurality of pressuredata points as corresponding to a static postural state, basedon the location of the center of mass/gravity/force of theperson of each of the plurality of pressure data points.

In some embodiments, determining a postural state of aperson at a point in time can include determining a subsequentpostural state of the person. A postural state of the person canbe determined either in real time or at a later point in time afterthe pressure points/postural states have been acquired/deter-mined/identified. For example, as the plurality of pressuredata points are being acquired, a postural state of a person ata point in time (e.g., a subsequent postural state of the person)can be determined in real time, based on the plurality ofpressure data points. In some embodiments, a postural state ofa person at a point in time (e.g., a current postural state or asubsequent postural state of the person) can be determinedbased on the plurality of pressure data points which wereacquired during a previous period in time (e.g., a current orsubsequent postural state of a person canbe determined basedon pressure data points/postural states that were acquired/determined previously, such as, for example, 6 months ago).

In some embodiments, a po stural state of a person at a pointin time caninclude determining a subsequent postural state ofthe person based a probability of transitioning between thestatic postural state and the dynamic postural state. The prob-ability of transitioning between the static postural state andthe dynamic postural state canbe based, at least in part, by theidentified first postural state and the identified second pos-tural state.

In some embodiments, a person can be identified (e.g.,determined to be) posturally stable or posturally unstable

4based on at least an identified first postural state and anidentified secondpostural state (e.g., identified from pressuredata points).

In some embodiments, a Hidden Markov Model calcula-5 tion is utilizedto determine a next postural state. Probabilities

of transitioning between a plurality of stable and unstablepostural states can be utilized to determine the next posturalstate. In some embodiments, the probabilities of transitioningbetween stable and unstable postural states are determined.

io The probabilities of transitioning between stable and unstablepostural states can be determined based on the range of pos-tural stability.

In some embodiments, the received pressure information isstored. The range of postural stability for a structure can be

15 determined based on the storedpressure information. In someembodiments, the current postural state is stored. A statistic,a score, and/or a simulation is determined based on the storedpostural state. A message is transmitted to a postural analysismodule using a network. The message includes the statistic,

20 the score, the simulation, the current postural state, the nextpostural state, and/or the range of postural stability.

In some embodiments, the current postural state includesactivity information, performance information, fatigue infor-mation, and/or diagnosis information. The range of postural

25 stability can be unique for the structure.In some embodiments, the structure is a human and the

load bearing structure is the human's lower extremities. Thestructure can be a human and the load bearing structure can bea cane, a crutch, a walker, a prosthetic leg, a walking aid, or

so any combination thereof. The structure can be a roboticdevice or a vehicle and the load bearing structure can be a legand/or an axle coupled to one or more wheels.

In some embodiments, the pressure information is receivedfrom a plurality of sensors which are coupled to the load

35 bearing structure. A message can be transmitted to the struc-ture. The message can relate to the next postural state. In someembodiments, a message is transmitted to a stability manage-ment module. The message can include supplemental sensoryinformation to stimulate the structure to modify the next

40 postural state from unstable to stable. In some embodiments,a message is transmitted to a stability management module.The message can include weight adjustment information tomodify the next postural state from unstable to stable.

In some embodiments, a safety device is activated based on45 the current postural state and/or the next postural state. A

message can be transmitted to a communication moduleusing a network. The message can include the current pos-tural state, the next postural state, and/or the range of posturalstability.

50 The system can also include a handheld portable stabilitydevice. The handheld portable stability device can include thestability processing module. In some embodiments, the sys-tem further includes a stability management module config-ured to adjust a weight device to modify the next postural state

55 from unstable to stable. In some embodiments, the systemfurther includes a safety device configured to activate basedon the current postural state and/or the next postural state. Thesensor can be positioned in or disposed relative to a sock, ashoe, a sole insert, a cane, a crutch, a walker, a walking aid, a

60 prosthetic leg, a robotic leg, a vehicle, and/or an axle con-nected to one or more wheels.

Shoes can be designed with pressure sensors (e.g., pressuresensors inside the shoes) that can be utilized to track how aperson is doing and how well the person is controlling their

65 balance throughout the day. A postural stability device can beportable and can be utilized on a daily basis while the personis conducting daily activities. A person can be notified regard-

US 8,011,229 B25

ing a pending unstable postural state, which can prevent fallsand/or injuries to the person. A postural stability of a personover a period of time can be tracked and monitored, which canallow for the monitoring and/or correction of the person'sdaily activities. Tracking of a person's postural stability can 5

allow for the assessment of medical and/or physical therapy.The determination of postural stability can be utilized inconnection with safety devices to protect against injuries.

Other aspects and advantages of the invention can becomeapparent from the following drawings and description, all of 10

which illustrate the principles of the invention, by way ofexample only.

BRIEF DESCRIPTION OF THE DRAWINGS15

The advantages of the invention described above, togetherwith further advantages, may be better understood by refer-ring to the following description taken in conjunction with theaccompanying drawings. The drawings are not necessarily to 20

scale, emphasis instead generally being placed upon illustrat-ing the principles of the invention.

FIG.1 is an exemplary diagram illustrating soles of a shoewhich include a stability processing module, according to anillustrative embodiment of the invention. 25

FIG. 2 is an exemplary diagram illustrating a human with ahandheld stability device, according to an illustrativeembodiment of the invention.

FIG. 3 is an exemplary diagram illustrating a walker with astability module, according to an illustrative embodiment of 30

the invention.FIG. 4 is an exemplary diagram illustrating a person with a

cane and a handheld stability device, according to an illustra-tive embodiment of the invention.

FIG. 5 is an exemplary diagram illustrating a person play- 35

ing golf with a handheld stability device, according to anillustrative embodiment of the invention.

FIG. 6 is an exemplary diagram illustrating a robot with astability module, according to an illustrative embodiment of 40the invention.

FIG. 7 is an exemplary flowchart illustrating the determi-nation of postural states, according to an illustrative embodi-ment of the invention.

FIG. 8 is an exemplary flowchart illustrating the checking 45

of postural states, according to an illustrative embodiment ofthe invention.

FIG. 9 shows a state graph, according to an illustrativeembodiment of the invention.

FIG. 10 is an exemplary chart illustrating equilibrium, 50

according to an illustrative embodiment of the invention.FIG. 11 is an exemplary chart illustrating a safe zone,

according to an illustrative embodiment of the invention.FIG. 12 is an exemplary chart illustrating equilibrium,

according to an illustrative embodiment of the invention. 55

FIG. 13 is an exemplary chart illustrating equilibrium,according to an illustrative embodiment of the invention.

FIG. 14 is an exemplary chart illustrating equilibrium,according to an illustrative embodiment of the invention. 60

FIG. 15A is an exemplary illustration of data acquired froma pressure sensor, according to an illustrative embodiment ofthe invention.

FIG. 15B illustrates a method for determining posturalstability of a person based on the exemplary illustration of the 65

data from FIG. 15A, according to an illustrative embodimentof the invention.

6FIG. 16 is an illustrative graph comparing the number of

equilibria from subjects.

DETAILED DESCRIPTION OF THE INVENTION

A postural state of a person or subject can be utilized tonotify the person and/or activate a safety device. The methodand system for determining a subject's postural state caninclude a pressure sensor. Pressure information is receivedfrom the pressure sensor. The pressure sensor can be coupledto, for example, the sole of a person's shoe. The pressureinformation can be utilized to determine a current posturalstate of the person. A next postural state (e.g., a subsequentpostural state) of the person canbe determined using a HiddenMarkov Model (HMM) calculation. The HMM calculationcan utilize the current postural state of the person, a range ofpostural stability associated with the person, and/or prob-abilities of the transitions between the current postural stateand the next postural state. The next postural state can beutilized to take corrective action to change the next posturalstate from unstable to stable (e.g., weight management deviceutilized to redistribute weight, stimulate the feet of the per-son), to activate a safety device (e.g., airbag, inflatable undergarments), and/or to notify the person regarding the nextpostural state (e.g., "Warning You May Fall!," "Sit DownImmediately! ").

The illustrative embodiments as described herein can beutilized, for example, by physical therapists, doctors, athletes,astronauts, patients, and/or any person that needs to assessand/or correct postural stability. The illustrative embodi-ments as described herein can, for example, quantify thevestibular and other sensory feedback systems of the bodywhich are used to maintain balance during quiescent stand-ing, locomotion (e.g., running, walking), and/or other sen-sory-motor activities (e.g., dancing, kneeling). For example, aphysical therapist can utilize the postural states to gauge theprogress of a stroke patient as the patient relearns skills suchas standing, walking, and/or running. As another example, thepostural states could be utilized to help train athletes byquantifying their daily behavior (e.g., time spent running,time spent sitting) during training.

FIG.1 is an exemplary diagram 100 illustrating soles 110aand 110b (generally 110) of a shoe which can include astability processing module 150, input sensors 120a and 120b(generally 120), output modules 130a and 130b (generally130), and a communication module 140, according to anillustrative embodiment of the invention. The input sensors120 measure sensor information (e.g., pressure information).The sensor information from the input sensors 120b can betransmitted to the communication module 140. The transmis-sion from the input sensors 120b to the communication mod-ule 140 can be, for example, via a wire embedded into the sole110b. The communication module 140 can transmit the sen-sor information from the input sensors 120b to the stabilityprocessing module 150. The transmission from the commu-nication module 140 to the stability processing module 150can be, for example, via a personal area network (PAN). Thesensor information from the input sensors 120a can be trans-mitted to the stability processing module 150. The transmis-sion from the input sensors 120a to the stability processingmodule 150 can be, for example, via a wire embedded into thesole 110a.

The stability processing module 150 determines the cur-rent postural state of the person associated with the soles 110based on the sensor information. The stability processingmodule 150 can determine the next postural state of the per-son (e.g., a subsequent postural state of a person) based on a

US 8,011,229 B27

range of postural stability, the current postural state, and/or aprobability of the next postural state.

If the next postural state is determined to be unstable (e.g.,falling down, loss of equilibrium), then the stability process-ing module 150 can transmit a message to the output modules130. The message transmitted to the output modules 130 caninclude instructions to modify the next postural state fromunstable to stable. The output modules 130 can be, forexample, vibration modules which vibrate to notify the per-son of the pending unstable state and/or to increase blood flowto the person's feet. The increased blood flow to the person'sfeet can, for example, provide the muscular strength and/orflexibility for the person to adjust his/her postural state.

For example, the input sensors 120 are pressure sensors120 which sense the pressure of a person's foot in each of thesoles 110. The pressure information sensed by the pressuresensors 120 in each of the soles 110 can be transmitted to thestability processing module 150. The pressure sensors 120ain the sole 110a with the stability processing module 150 cantransmit the pressure information via a wire embedded intothe sole 110a. The pressure sensors 120b in the sole 110awithout the stability processing module 150 can transmit thepressure information via a wire embedded into the sole 110bto the communication module 140. The communication mod-ule 140 can transmit the pressure information wirelessly tothe stability processing module 150.

The stability processing module 150 can determine thecurrent postural state by calculating the center of force of theperson utilizing the pressure information. The determinationof the current postural state can utilize the HMM calculation.The HMM calculation can utilizes a set of probabilities foreach postural state to determine the next postural state. If thenext postural state is stable, then the stability processingmodule 150 can continue to monitor the person. If the nextpostural state is unstable, then the stability processing module150 can activate the output modules 130 to vibrate whichnotifies the person that they may fall and should take correc-tive action immediately.

In some embodiments, the soles 110 are inserts for shoes.The soles 110 can be, for example, integrated into shoes. Astable equilibrium in a person can be, for example, describedas a well-controlled posture while an unstable equilibriumcan be a poorly controlled posture.

In some embodiments, the input sensors 120 are pressuresensors that measure pressure information. The input sensor120 can be, for example, a motion sensor, a temperaturesensor, a heat sensor, a dielectric sensor, an electrical sensor,a magnetic sensor, a flow sensor, a humidity sensor, a chemi-cal sensor, a light sensor, a sound sensor, and/or any other typeof sensor.

In some embodiments, the transmission of the sensor infor-mation from the communication module 140 to the stabilityprocessing module 150 is through a network. The networkcan be, for example, a network (e.g., wired, wireless).

In some embodiments, the output modules 130 are electri-cal output modules which output electrical pulses and/ormechanical output modules which output mechanical pulsesand/or stimuli. The electrical pulses and/or the mechanicalpulses can, for example, notify the person of the next posturalstate and/or can stimulate the person to modify the next pos-tural state from unstable to stable. The stimulation can occurin real-time after the stability processing module 150 deter-mines that the next postural state is unstable, which can pre-vent the person from falling while and also improve circula-tion and/or muscle tone.

In some embodiments, the stability processing module 150transmits the next postural state, the current postural state, the

8sensor information, and/or the range of postural stability to apostural analysis module (not shown). The postural analysismodule can be, for example, a handheld portable device foruse by the person wearing the sensor device, a handheld

5 portable device for use by a person monitoring the personwearing the sensor device, part of a computing device (e.g.,computer at doctor's office, computer at the person's home)used to automatically monitor the person wearing the sensordevice, or any combination thereof. The transmission to the

to postural analysis module can be, for example, through a net-work (e.g., public switched telephone network (PSTN), alocal area network (LAN), a radio area network (RAN), apersonal area network (PAN), the internet). The postural

15 analysis module can, for example, utilize the transmittedinformation to monitor, track, and/or notify the personregarding their postural states. The postural analysis modulecan, for example, store the transmitted information for his-torical analysis by the person being monitored and/or a third

20 party monitoring the person (e.g., doctor, physical therapist).In some embodiments, the stability processing module 150

transmits the current postural state and/or the sensor informa-tion to the postural analysis module. The postural analysismodule can, for example, store the current postural state

25 and/or the sensor information. The postural analysis modulecan, for example, determine a statistic (e.g., percentage oftime running, percentage of time sitting), a score (e.g., num-ber of falls per day, average number of falls per month), asimulation (e.g., with increased physical therapy will the

so number of falls decrease, with increased training can theathlete distribute his/her mass better), and/or any other type ofmetric based on the stored postural state and/or the storedsensor information. The postural analysis module can, forexample, display the statistic, the score, and/or the simulation

35 for use by the person being monitoring and/or the third party.The postural analysis module can, for example, store thestatistic, the score, and/or the simulation.

In some embodiments, the postural state is stable orunstable. The postural state can, for example, include activity

40 information (e.g., walking, running, sitting), performanceinformation (e.g., time spent walking, time spent running),fatigue information (e.g., time spent close to outer range ofpostural stability, time spent close to center of postural sta-bility), and/or diagnosis information (e.g., limp, lameness,

45 neural condition, muscular condition, vision-related condi-tion).

In some embodiments, the determination of a next posturalstate, current postural state, and/orpastpostural states utilizesa posterior decoding algorithm, a Bayesian segmentation, a

50 graphical model, a choice-point method, and/or any othertype of algorithm that classifies time periods into static and/ordynamic periods. A dynamic Bayesian network can be, forexample, utilized to determine the next and/or past posturalstates based on the current postural state, the range of postural

55 stability, and/or the probabilities of the next postural state.In some embodiments, the determination of the next, cur-

rent, and/or past postural states utilizes a forward algorithm,a Viterbi algorithm, a forwards-backwards algorithm, Baum-Welch algorithm, and/or any other type of algorithm that

60 classifies time periods into static and/or dynamic periods. Theforwards-backwards algorithm can be, for example, utilizedto determine the probability of the next state (e.g., dynamic,equilibrium). The Viterbi algorithm can be, for example, uti-lized to determine the probability of the next state. The Baum-

65 Welch algorithm can be, for example, utilized to determinethe range of postural stability and/or the probabilities of tran-sitioning between states.

US 8,011,229 B29

In some embodiments, the HMM calculation determinesthe next state, the current state, and/or one or more past states(e.g., five, ten). The HMM calculation can be, for example,utilized to determine the probabilities of the sequence of thepast states, the current state, and/or the next state. Thesequence of the past states can be, for example, utilized tocalculate the probability of the next state. The sequence of thepast states can be, for example, utilized to determine thescore, the statistic, and/or the simulation.

Although FIG. 1 illustrates soles 110 with two input sen-sors 120 each, the soles 110 can have a plurality of inputsensors (e.g., four, ten, twenty, etc.). In some embodiments,only one of the soles 110 has input sensors 120. In someembodiments, there is only one input sensor 120 utilized forsensor information (i.e., there is only one input sensor 120between the two soles 110).

In some embodiments, the range of postural stability isdetermined for the person utilizing the soles 110. Therange ofpostural stability can be, for example, unique for the personsince the range of postural stability can be affected by age,activity level, postural stance, weight, medical history, and/orany other factor that can affect a person's posture.

In some embodiments, the range of postural stability isdetermined based on sensor information which is stored bythe stability processing module 150. The range of posturalstability can be determined, for example, by processing thestable postural states to determine the range of stable posturalstates. The determination of the range of the postural stabilitycan occur, for example, in real-time while the user is wearingthe shoes with the soles 110.

In some embodiments, the range of postural stability isbased on a person's center of gravity. A person's center ofgravity can vary, for example, in a range because a human canbe modeled as an inverted pendulum in which an uprightstance is an unstable equilibrium. Since small natural centerof mass deviations (e.g., breathing, limb movements, headmovement) can disrupt the equilibrium, then the pendulum(i.e., which can represent the person) can top over withoutappropriate sensory-motor control system. Generally, stand-ing posture utilizes subconscious sensory feedback mecha-nisms (e.g., vision, tactile sensations, vestibular organs) tomaintain upright stance (i.e., a stable postural state). Anadvantage of determining the postural stability of a person isthat the person can have a real-time readout of their capacityto balance at any given point in time.

In some embodiments, the range of postural stability ispre-determined for the person based on preset parameters.For example, there can be preset parameters based on a per-son's age, weight, height, activity level, and/or any other typeof parameter associated with posture.

In some embodiments, the range of postural stability is aglobal optimum. The global optimum indicates, for example,that there is a single optimal point for upright posture. If aperson is not at the optimum, then the person's body alwaysdirects the person towards the optimum.

In other embodiments, the range of postural stability is asafe zone. The safe zone can be, for example, a zone ofupright posture. Inside this zone, a person is stable in regardsto postural stability and a person moves around this zone atrandom. Every person can, for example, have a safe zone. Thesafe zone for every person can be, for example, unique fromother safe zones.

In some embodiments, the range of postural stability is apunctuated equilibrium. The punctuated equilibrium can be,for example, a safe zone with a constant turnover of equilib-ria. This transient equilibria form, persist, and dissipates fol-

10lowing control failure (e.g., not in equilibrium, includingfalling down). Following a control failure, a new equilibriaforms and control is restored.

Although FIG. 1 illustrates a person wearing a shoe, the5 input sensors 120 can be coupled to a load bearing structure

associated with a structure such that the postural states aredetermined for the structure. The structure can be, forexample, a vehicle and/or any other type of structure associ-ated with a load bearing structure. The load bearing structure

io can be, for example, a leg, an axle couple to one or morewheels, and/or any other type of structure that is load bearing.In some embodiments, the structure is a car and the loadbearing structures are the axles coupled to the wheels, wherethe input sensors 120 are coupled to the axles.

15 In some embodiments, the input sensors 120 are coupled toa bed to receive input information associated with the personon the bed. The input sensors 120 can be, for example,coupled to a seat of a chair to receive input informationassociated with the person sitting in the chair. For example,

20 the input sensors 120 can be coupled to the seat in a car. Whenthe stability processing module 150 determines that the cur-rent postural state and/or the next postural state is the personbeing ejected from the seat (e.g., in an accident), then thestability processing module 150 can activate a safety device

25 (e.g., air bag, anti-lock brakes).In some embodiments, the output module 130 is coupled to

the bed and/or chair. The output module 130 can be, forexample, a mechanical stimulator which can activate whenthe person has been sitting and/or lying for a set period of time

30 (e.g., one minute, twenty minutes). For example, the stabilityprocessing module 150 is set to activate the mechanicalstimulator for one minute every sixty minutes of sitting.When the stability processing module 150 determines that theperson has been sitting for sixty minutes (e.g., continuous

35 sitting, accumulative sitting), then the mechanical stimulatoris activated for one minute. A person's muscles can be auto-matically stimulated based on inactivity to prevent muscledecay due to the inactivity.

FIG. 2 is an exemplary diagram 200 illustrating a person4o 210 with a handheld stability device 240, shoes 220a and

220b (generally 220), and input sensors 222a and 222b (gen-erally 222), according to an illustrative embodiment of theinvention. In this embodiment, the person 210 is wearingshoes 220. Each of the shoes 220 can have input sensors 222.

45 The input sensors 222 can receive sensor information associ-ated with the person. The input sensors 222 can transmit thesensor information to a communication module 244 in thehandheld device 240. The communication module 244 cantransmit the sensor information to the stability processing

50 module 246. The stability processing module 246 can deter-mine the current postural state and the next postural state. Thecurrent postural state and the next postural state can be dis-played on the display module 242.

In some embodiments, a message is transmitted from the55 stability processing module 246 to the display module 242.

The message can be displayed on the display module 242.The message can include information informing the person ofa stable equilibrium (e.g., "Great Stance!," "StandStraight!"), instructional information informing the person to

60 take action to not fall (e.g., "Sit Down!," "Warning You mayFall!"), reporting information (e.g., "You ran two hourstoday," "You have been sitting too long today."), and/or anyother type of information collected and/or utilized by thehandheld device. The reporting information can include

65 information generated and/or determined by the stability pro-cessing module 246 which can include the score, the statistic,and/or the simulation.

US 8,011,229 B211

12In some embodiments, the communication module 244

The stability processing module 340 can determine the

receives sensor information through a wireless network (e.g., next postural state based on the current postural state, thePAN, RAN) from the input sensors 222. range of postural stability, and/or the probabilities of the next

In some embodiments, the display module 242 is a liquid

postural state. If the next postural state is unstable, then thecrystal display (LCD) device which displays information 5 stability processing module 340 can send a message to the(e.g., corrective information, current posture information). stability management module 350. Based on this message,The display module 242 can be, for example, a light emitting the stability management module 350 can transmit instruc-diode (LED) and/or any other type of display which notifies tions for the speaker 365 to announce a warning (e.g., "Sitthe person 210 of the postural states. In some embodiments, Down!," "Unstable State!"), for the display module 360 tothe legs of the person 210 are the load bearing structures and io display a warning, for the vibrators 368 to vibrate, and/or forthe person 210 is the structure. the weight devices 355 to adjust to stabilize the walker 310

In some embodiments, the handheld device 240 and the and/or the person. The weight devices 355 can be coupled toinput sensors 222 are utilized to determine the postural sta- each of the legs of the walker 310.bility of an astronaut after a spaceflight. The changes in gravi- In some embodiments, the current postural state can betational field strength during a spaceflight can, for example, 15 transmitted to the display module 360. The display moduledisrupt the postural stability of an astronaut. The handheld

360 can display the current postural state. For example, if the

device 240 can allow for the tracking of the astronaut's pos- current postural state is stable, then the display can be "Stabletural stability on a long-term basis without interfering with

Posture." If the current postural state is unstable, then the

the astronaut's daily activities. In some embodiments, the

display can be "Unstable Sit Down Immediately!"handheld device 240 and the input sensors 222 are utilized to 20 In some embodiments, the stability processing module 240monitor the postural state of an astronaut during spaceflight and/or the stability management module 350 are utilized toand/or environments of altered gravity (e.g., on the moon, on auto balance the walker 310. The auto balance can utilizemars). weight devices 355 to balance the walker 310.

In some embodiments, the handheld device 240 is a por- Although FIG. 3 illustrates a weight device (e.g., 355a)table handheld device 240. The portable handheld device 240 25 coupled to each of the legs of the walker 310, the walker 310can be utilized during a person's daily activities and may not can have one weight device (e.g., 355) that could be centrallyinterfere with the collection of sensor information and/or mounted on the walker 310. The centrally mounted weightpostural states during normal activity. device (e.g., 355a) can be utilized to modify the postural state

Although FIG. 2 illustrates the handheld device 240 asso- of the walker 310 and/or person from unstable to stable.ciated with the person 210 that is associated with the sensor 30 FIG. 4 is an exemplary diagram 400 illustrating a personinformation, the handheld device 240 can be utilized by a

410 with a cane 430, a handheld stability device 420, andthird party (e.g., physical therapist, doctor) to track the pos- input sensors 412a, 412b, and 412c (generally 412), accord-tural states of the person (e.g., a patient, an athlete). The third

ing to an illustrative embodiment of the invention. The person

parties can track the progress of the patient as the patient

410 can utilize cane 430 and the person's feet as load bearingrelearns skills such as standing, walking, and/or running via 35 structures. The input sensors 412 can be coupled to the per-the handheld device 240. son's shoes and the cane 430, respectively. The input sensors

In some embodiments, the input sensors 222 communicate

412 can transmit input information to the communicationsvia a wireless network (e.g., PAN, RAN) to a remote com- module 424 via a network (e.g., PAN, RAN). The communi-puting device (not shown) which is utilized by third parties cations module 424 can transmit the input information to the(e.g., caregiver, doctor) to monitor the person's postural sta- 40 stability processing module 422. The stability processingbility. The remote computing device can, for example, store module 422 can determine the current postural state and thethe sensor information. The remote computing device can, for next postural state of the person.example, determine the statistic, the score, the simulation, Although FIG. 4 illustrates the person 410 utilizing theand/or any other type of information based on the sensor cane 430, the person 410 could be utilizing a crutch, a pros-information. 45 thetic leg, and/or any other walking aid. The input sensors 412

In some embodiments, the input sensors 222 are coupled to can be coupled to any of these load bearing structures.any load bearing structure (e.g., person's lower extremities, FIG. 5 is an exemplary diagram 500 illustrating a personsocks) associated with the structure (in this example, the

510 playing golf utilizing a golf club 512, a handheld stabilityperson). device 530, a stability processing module 534, a communi-

Although FIG. 2 illustrates a person 210 with shoes 220, 50 cation module 536, and input sensors 522a and 522b (gener-the person 210 can be in a spacesuit and the shoes 220 can be ally 522), according to an illustrative embodiment of thethe boots of the spacesuit. The input sensors 222 can be

invention. The person's shoes 510 can include input sensors

coupled to the boots and/or the other lower extremities of the

522 which receive sensor information. The sensor informa-spacesuit. The handheld device 240 can be integrated into the tion can be transmitted to the communications module 536other devices and/or modules of the spacesuit. 55 via a network (e.g., PAN, RAN). The communications mod-

FIG. 3 is an exemplary diagram 300 illustrating a walker ule 536 can transmit the sensor information to the stability310 with a stability processing module 340, a stability man- processing module 534.agement module 350, a display module 360, a speaker 365, The stability processing module 534 can determine thevibrators 368, input sensors 320a, 320b, 320c, and 320c (gen- current postural state based on the sensor information. Theerally 320), and weight devices 355a, 355b, 355c, and 355d 60 current postural state canbe transmitted to the display module(generally 355), according to an illustrative embodiment of

532 which displays the postural state (e.g., "Good Postural

the invention. The walker can include input sensors 320

Stance!"). The stability processing module 534 can process awhich are coupled to each leg of the walker 310. The input simulation to determine if any changes can be made to thesensors 320 can receive input information and can transmit person's posture tothe input information to the stability processing module 340. 65 The stability processing module 534 can determine theThe stability processing module 340 can determine the cur- next postural state based on the current postural state, therent postural state. range of postural stability, and/or the probability of the next

US 8,011,229 B213

14postural state. The probability of the next postural state canbe, for example, customized according to sporting activity.For example, since football includes external forces acting onthe person (i.e., one or more third parties tackling the person),then the probabilities of the next postural state can take thoseexternal forces into account (e.g., the probability that the nextpostural state changes from running to tackled may be 0.9 fora football player while the probability is only 0.1 for a trackrunner).

Although FIG. 5 illustrates the handheld postural device530 utilized to analyze the person's golf game, the handheldpostural device 530 can be utilized for other activities (e.g.,sports). For example, the handheld postural device 530 can beutilized for track and field, American football, baseball,cricket, soccer, basketball, hockey, bowling, gymnastics, ski-ing, figure skating, dance and/or any other type of sportingactivity.

FIG. 6 is an exemplary diagram 600 illustrating a robot 610with a stability processing module 620, a stability manage-ment module 630, input sensors 612a and 612b (generally612), and weight devices 614a, 614b, and 614c (generally614), according to an illustrative embodiment of the inven-tion. The robot 610 can have input sensors 612. The inputsensors 612 can be coupled to the load bearing structures ofthe robot 610 (i.e., the legs of the robot 610). The inputsensors 612 can receive input information (e.g., pressureinformation from each leg of the robot 610). The input sensors612 can transmit the input information to the stability pro-cessing module 620. The stability processing module 620 candetermine the current postural state of the robot 610 based onthe input information. If the current postural state is unstable,then the stability processing module 620 can transmit a mes-sage to the stability management module 630. The messagecan include information regarding the instability (e.g., all ofthe weight is on the left leg, weight is not distributed evenly).Based on the message, the stability management module 630can transmit a message to the weight devices 614 to adjust theweight balance of the robot 610. These adjustments by thestability management module 630 can be used to modify thepostural state from unstable to stable.

The stability processing module 620 can determine thenext postural state based on the current postural state, therange of postural stability, and/or the probability of the nextpostural state.

In some embodiments, the range of postural stability ispredetermined for the model of the robot 610. For example,Model R2 robots can have a set range of postural stabilitybased on the distribution of their components. In someembodiments, the range of postural stability is determined inreal-time based on the configuration of the robot 610. Forexample, the robot 610 can determine which modules (e.g.,extra battery, welder attachment, voice processor) it containsand determines the range of postural stability based on itsincluded modules.

FIG. 7 is an exemplary flowchart 700 illustrating the deter-mination of postural states utilizing the exemplary diagram200 of FIG. 2, according to an illustrative embodiment of theinvention. The communication module 244 receives (710) thepressure information (e.g., pressure data points) from thesensors 222. The pressure information can be acquired over aperiod of time. The pressure information can be communi-cated to the stability processing module 246. The stabilityprocessing module 246 can determine (720) the current pos-tural state of the person 210 based on the pressure informationand store (740) the pressure information. Based on the storedpressure information, the stability processing module 246determines (750) the range of postural stability.

The stability processing module 246 can determine (730)the next postural state of the person 210 (e.g., a subsequentpostural state of the person) based on the current posturalstate, the range of postural stability, and/or the probability of

5 the next postural state. The stability processing module 246can transmit (760) a message which includes the next posturalstate to the display module 242 for display to the person 210.After the stability processing module 246 determines (730)the next postural state, the processing can receive (710) the

10 next set of pressure information from the sensors 222.FIG. 8 is an exemplary flowchart 800 illustrating the check-

ing of postural states utilizing the exemplary diagram 600 ofFIG. 6, according to an illustrative embodiment of the inven-tion. The stability processing module 620 can receive (810)

15 the pressure information from the sensors 612. The stabilityprocessing module 620 can determine (820) the current pos-tural state of the robot 610 based on the pressure information.

The stability processing module 620 can determine (830)the next postural state of the robot 610 based on the current

20 postural state, the range of postural stability, and/or the prob-ability of the next postural state. The stability processingmodule 620 can check (860) the next postural state. If the nextpostural state is stable, then the processing can continue byreceiving (810) the next set of pressure information from the

25 sensors 612. If the next postural state is unstable, then thestability processing module 620 can utilize the stability man-agement module to adjust (870) the weight devices 614. Theadjustment (870) of the weight devices 614 can modify thenext postural state from unstable to stable.

30 Although FIG. 8 illustrates the adjustment (870) of weightdevices 612 in unstable states, the weight devices can also beadjusted (870) to modify the next postural state from one stateto another state (e.g., standing to sitting, dynamic to equilib-rium, running to walking).

35 In some embodiments, the probability of transitioningbetween postural states is determined. The probabilities canbe, for example, determined based on the stored sensor infor-mation and/or the range of postural stability.

For example, the HMM calculation utilizes various states40 (e.g., current state, one or more past states) and the probabili-

ties of the hidden states (e.g., current state, one or more paststates) to determine the next postural state. FIG. 9 depicts astate graph, according to an illustrative embodiment of theinvention. By way of example, if the state graph in FIG. 9 is

45 utilized with tables I and 2, then the probability of the nextpostural state can be determined. As shown in FIG. 9 for eachstate 905-920 there can be corresponding observations 925-940. Si (905), S i^, (910), Sz+z (915), and Sz+3 (920) representthe states and 0, (925), O i^, (930), Oz+z (935), and 0z+3 (940)

50 represent the possible observations.

TABLE I

Emission Probability

55Si

Equilibrium Dynamic

O, Fast velocity 0.2 0.7Slow velocity 0.8 0.3

60

TABLE 2

Transition Probability65

S,,,

US 8,011,229 B215

16TABLE 3-continued

Equilibrium DynamicEmission Probability

S iEquilibrium 0.98 0.02Dynamic 0.32 0.68

In some embodiments, the observation O, (925) is utilizedwith the past states (e.g., Si-11 S z _21 S,_ 3) and the probabilitiesof the sequence of the past states to determine the probabilityof the current state S, (905). The probabilities of the observa-tion O, (925) in the emission probability, table 1 (e.g., whichcan be used to determine an emission matrix), and the tran-sition probability, table 2 (e.g., which can be used to deter-mine a transition matrix), can be, for example, utilizedtogether to determine the probability of the current state Sand/or the next state S,,,. For example, if the past four stateswere equilibrium, then the probability that the current statewill stay in equilibrium, the probability that the current statewill change to dynamic, and the probability of the state asso-ciated with the observation is utilized to determine the currentstate and/or the next state. As another example, if thesequence of the past four states is equilibrium, dynamic,equilibrium, and equilibrium, then the probability of thesetransitions in relation to each other, the probability of thecurrent state changing or staying the same (in this example,equilibrium), and the probability of the state associated withthe observation is utilized to determine the current state and/or the next state. The context of the transitions and/or notransitions (e.g., the lack of transitioning) between the paststates is utilized to determine the next state, thereby providingthe calculation with a history.

In some embodiments, the dynamic state represents threepossible outcomes: return to present equilibrium, transition tonew equilibrium, or falling down. In some embodiments, theobservations include sitting, standing, kneeling, lying down,falling, and/or any other postural position of a structure (e.g.,person, robot). In some embodiments, the observations (e.g.,O,) include any type of observation of postural state (e.g.,falling, standing, running, walking).

In some embodiments, velocity includes the center of massvelocity for the structure. The center of mass velocity can be,for example, measured by the input sensors (e.g., pressuresensors). The slow velocity and fast velocity can be, forexample, relative (e.g., determined by comparing the mea-sured center of mass velocity).

In some embodiments, the HMM calculation utilizes vari-ous states and the probabilities of the next hidden state todetermine the next postural state. By way of example, if thestate in FIG. 9 is utilized with tables 3 and 4, then the prob-ability of the next postural state can be determined.

TABLE 3

Emission Probability

Dynamic EquilibriumStanding Standing Walking Running

O, Fast Gait and 0.15 0.01 0.04 0.60Fast VelocityFast Gait and 0.20 0.03 0.30 0.06Slow VelocitySlow Gait and 0.40 0.01 0.06 0.32Fast VelocitySlow Gait and 0.05 0.05 0.60 0.02Slow VelocitySlow Gait and 0.20 0.30 0.00 0.00

S;5


No VelocityNo Gait and No 0.00 0.60 0.00 0.00

10 Velocity

TABLE 4

15 Transition Probability

S;_,


20S i Dynamic 0.60 0.30 0.05 0.05

StandingEquilibrium 0.25 0.60 0.10 0.05StandingWalking 0.20 0.05 0.60 0.15Running 0.12 0.10 0.18 0.60

25

In some embodiments, the stability processing module(e.g., 246 of FIG. 2) analyzes the range of postural stability todetermine the current postural state and/or the next postural

30 state. If the received pressure information is within set param-eters of the range (e.g., 25% to 75%, 10% to 90%), then thestability processing module can determine that the next pos-tural state is equilibrium (e.g., equilibrium running, equilib-rium walking). If the received pressure information is not

35 within the set parameters of the range, then the stabilityprocessing module can determine that the next postural stateis dynamic (e.g., dynamic falling, dynamic walking).

In some embodiments, the probabilities of two or morepossible next postural states are the same and/or substantially

40 similar, so the next postural state may not be determined. Thestability processing module (e.g., 246 of FIG. 2) can processthe input information received from the input sensors, therange of postural stability, and/or the current postural state todetermine the next postural state.

45 In some embodiments, the stability processing module(e.g., 246 of FIG. 2) processes the input information receivedfrom the input sensors, the range of postural stability, thecurrent postural state, and/or the next postural state to deter-mine if activity (e.g., alarm, email, notification) should be

50 initiated based on the processing.In some embodiments, the processing applies one or more

rules to determine if a condition occurs. By way of example,a rule can include determining whether the person entered adynamic state more than ten times in a thirty minute period. A

55 rule can also include determining if, for example, the personhas been in a dynamic state for 75% of the time over the pasttwenty four hours. The rules can be, for example, predeter-mined (e.g., set of rules based on age, set of rules based on amedical condition) and/or automatically generated (e.g., the

60 person is usually in equilibrium 90% of the time in a two hourperiod so any percentage less than 90% in a two hour periodsends an email to the person's caregiver). The automaticallygenerated rules can be, for example, based on individualcharacteristics of the structure (e.g., specific percentage of

65 state over time, number of times in dynamic state per hour),general characteristics of the structure (e.g., age range, medi-cal condition), and/or any other metric associated with the

US 8,011,229 B217

18structure. The activity initiated can be, for example, setting receive the predetermined range of postural stability prepro-off the alarm, notifying the structure, notifying the third party grammed into their stability processing module.(e.g., sending an email to the doctor, sending a text message to

FIG. 12 illustrates an exemplary punctuated equilibrium

the caregiver), and/or any other type of notification and/or

during quiescent standing with eyes closed based on pressurealarm. In some embodiments, an average velocity (e.g., of the 5 points acquired from a sensor (e.g., sensor from the shoe,person or other load bearing structure) can be measured dur- walker, prosthetic leg, walking aid, portion of arobotic deviceing a window of time and the rule can be based on, for or other load bearing structure as described herein), accordingexample, a number of velocity emission states out of a num- to an illustrative embodiment. In this example, the sensor wasber of time period (e.g., whether there are more than 100

located on a shoe worn by a person/subj ect. The subj ect stood

high-velocity emission states out of 150 time periods, etc.) io for two minutes on an EquiTest platform. Pressure wasFIG. 10 illustrates an exemplary punctuated equilibrium sampled at the right toe, the right heel, the left toe, and the left

during quiescent standing based on pressure points acquired

heel, at a rate of one hundred Hz. The sensor data was clas-from a sensor (e.g., sensor from the shoe, walker, prosthetic sified by the HMM. Center-of-force from left foot (0.0) toleg, walking aid, portion of a robotic device or other load

right foot (1.0) is shown on the x-axis. Center-of-force in the

bearing structure as described herein), according to an illus- 15 anterior (1.0) to posterior (0.0) direction was plotted on thetrative embodiment. In this example, the sensor was located

y-axis, normalized by foot length. Earlier to later equilibria

on a shoe womby aperson/subject (i.e., person) that stood for 1201, 1205, 1210, 1215, 1220 and 1225 are illustrated by the

two minutes on an EquiTest platform (available from Neuro- different shadings/regions (i.e., as indicated by lighter shad-Com International of Clackamas, Oreg.). Pressure was

ing to dark shading). Pentagons were used to mark the center-

sampled at the right toe, the right heel, the left toe, and the left 20 of-force of each equilibrium. Pentagon size can correspond toheel, at a rate of one hundred Hz. The sensor data was clas- dwell time. The length of the embedded line can be the aver-sified by the HMM. Center-of-force from left foot (0.0) to age distance of each point in the equilibrium from the equi-right foot (1.0) is shown on the x-axis. Center-of-force in the

librium center-of-mass, a measure of the compactness of the

anterior (1.0) to posterior (0.0) direction is plotted on the equilibrium. Dynamic trajectories (e.g., trajectories 1230A-y-axis, normalized by foot length. The subject's center-of- 25 E) that can revert to the equilibrium are bordered in a darkforce was initially dynamic 1001 (e.g., as indicated by light

border. Trajectories which lead to escape from the equilib-

shading), but settled into an initial equilibrium 1005 (e.g., as rium (e.g., trajectories 1235A-B) are bordered by a lightindicated by lighter shading). A dynamic escape trajectory

border. In this case, the subject in this example oscillated

1010 (e.g., as indicated by lighter shading with light border)

through six equilibria 1201, 1205, 1210, 1215, 1220 and 1225led to settling in a second equilibrium 1015 (e.g., as indicated 30 in two minutes.by medium shading). A dynamic reversion trajectory 1020

FIG. 13 illustrates an exemplary punctuated equilibrium

(e.g., as indicated by medium shading with dark border) did

during quiescent standing with eyes open based on pressurenot disturb the equilibrium, but a subsequent escape trajec- points acquired from a sensor (e.g., sensor from the shoe,tory 1025 (e.g., as indicated by medium shading with light walker, prosthetic leg, walking aid, portion of arobotic deviceborder) led the subject to settle into a final equilibrium 1030 35 or other load bearing structure as described herein), according(e.g., as indicated by dark shading). Towards the end of the to an illustrative embodiment. In this example, the sensor wastwo minutes, the subject entered a dynamic trajectory 1035

located on a shoe worn by a person/subject. The same subject

(e.g., as indicated by dark shading with light border). In FIG. as in FIG. 12 stood for two minutes on an EquiTest platform,10, pentagons were used to mark the center-of-force of each

this time with eyes open. Pressure was sampled at the right

equilibrium. The pentagon size can correspond to dwell time. 40 toe, the right heel, the left toe, and the left heel, at a rate of oneThe length of the embedded line can be the average distance

hundred Hz. The sensor data was classified by the HMM.

of each point in the equilibrium from the equilibrium center- Center-of-force from left foot (0.0) to right foot (1.0) is shownof-mass, a measure of the compactness of the equilibrium. In on the x-axis. Center-of-force in the anterior (1.0) to posteriorsome embodiments, the center-of-force information is uti- (0.0) direction is plotted on the y-axis, normalized by footlized for the range of postural stability. For example, the range 45 length. A single equilibrium 1301 is seen. A dynamic trajec-of pressures collected from the subject while the subject is tory 1305 (e.g., as indicated by dark border) roughly midwaystanding is used to create the range of postural stability. The through the protocol ultimately reverts to the equilibrium.range of postural stability can be utilized to determine the

When eyes are open, visual feedback can enable greater sta-

next postural state. bility.FIG. 11 illustrates an exemplary safe zone based on pres- 50 FIG. 14 illustrates an exemplary punctuated equilibrium

sure points acquired from a sensor (e.g., sensor from the shoe, during perturbed standing based on pressure points acquiredwalker, prosthetic leg, walking aid, portion of a robotic device

from a sensor (e.g., sensor from the shoe, walker, prosthetic

or other load bearing structure as described herein), according

leg, walking aid, portion of a robotic device or other loadto an illustrative embodiment. In this example, the sensor was

bearing structure as described herein), according to an illus-

located on a shoe worn by a person/subject. Eighteen subjects 55 trative embodiment. In this example, the sensor was locatedstood for two intervals of two minutes each on an EquiTest on a shoe worn by a person/subject. The subject stood for twoplatform. Weight distribution from left foot (0.0) to right foot minutes on an EquiTest platform, squatting after forty sec-(1.0) is shown on the x-axis. Center-of-force in the anterior onds and lifting arms after eighty seconds. Pressure was(1.0) to posterior (0.0) direction is plotted on the y-axis, sampled at the right toe, the right heel, the left toe, and the leftnormalized by foot length. The different shadings correspond 6o heel, at a rate of one hundred Hz. The sensor data was clas-to different individuals. In FIG. 11, the safe zone 1105 is sified by a Hidden Markov Model. Center-of-force from leftelliptical in shape, but is far larger than the region encom- foot (0.0) to right foot (1.0) is shown on the x-axis. Center-passed by a single equilibrium or any two minute standing of-force in the anterior (1.0) to posterior (0.0) direction isinterval. The range of pressures collected from the eighteen plotted on the y-axis, normalized by foot length. The subjectsubjects can be utilized to create a predetermined range of 65 entered an initial equilibrium 1401 for forty seconds (e.g., aspostural stability. For example, people with the same charac- indicated by the lighter shading and pentagon with lighterteristics (e.g., age, height, weight) as the eighteen subjects shading), which was abruptly interrupted by a squat 1405

US 8,011,229 B219

20(e.g., as indicated by lighter shading with light border). This and a second postural state based on the first and secondled into a new equilibrium 1410 (e.g., as indicated by dark

pressure data points 1501 and 1502. The first and second

shading and pentagon with dark shading). The arm lift 1415 postural states can be individually identified as static or(e.g., as indicated by dark shading with dark border) did not

dynamic. The method can also include determining a postural

lead to deterioration of the equilibrium. The HMM was able 5 state (e.g., of the person) at a point in time (e.g., a subsequentto effectively identify both the squat and the arm-lift. postural state or a postural state of a person at a later point in

In some embodiments, the range of postural stability is time after the data points have been acquired) based on at leastdetermined by quantifying the dwell time, the size, the shape the first postural state and the second postural state. Forof the equilibrium, the dynamic trajectories, and/or the zone example, person can be identified as posturally stable orof stable equilibrium. FIGS. 10 through 14 illustrate exem- io unstable based on at least the identified first postural state andplary dwell time, size, shape of equilibrium, dynamic trajec- the identified second postural state (e.g., step 1535).tories, and zone of stable equilibrium that can be utilized to

The step (e.g., step 1525 from FIG. 15B) of identifying/

determine the range of postural stability of a subject/person. determining a postural state for each corresponding pressureAlthough the figures and examples described herein illus- data point (e.g., any one of pressure data points 1501-1512

trate people and robots, the input sensors (e.g., 120) can be 15 from FIG. 15A) can generate a plurality of postural states. Acoupled to any type of load bearing structure associated with

range of postural stability can be determined based on the

a structure. Based on the sensor information, the postural

plurality of pressure data points or the corresponding plural-states can be determined for the structure. The structure can

ity of postural states (e.g. from step 1525 from FIG. 15B).

be, for example, a vehicle, a person, a robot, and/or any other

In some embodiments, the postural state identified for eachtype of structure associated with a load bearing structure. The 20 pressure data points (e.g., any one of pressure data pointsload bearing structure can be, for example, a leg, a walking

1501-1512 from FIG. 15A) can be identified as a static pos-aid, an axle couple to one or more wheels, and/or any other tural state or a dynamic postural state. A dynamic posturaltype of structure that is load bearing. state can be defined as when the person is moving between

FIG. 15A illustrates data acquired from at least one pres- static postural states (e.g., moving from a first static posturalsure sensor over time, according to an illustrative embodi- 25 state to a second static postural state). By way of example, inment. FIG. 15B illustrates a method for determining postural

FIG. 15A, pressure data points 1503-1506 and pressure data

stability of a person based on the data acquired from the points 1509-1512 can be each identified as static posturalpressure sensor, according to an illustrative embodiment of

states and pressure data points 1501-1502 and pressure data

the invention. Pressure data points 1501-1512 can be acquired

points 1507-1508 can be each identified as dynamic posturalfrom a sensor (e.g., sensor/input sensor from a shoe worn by 30 states.the person, a walker, a prosthetic leg, walking aid, portion of

Threshold functions can be performed on the pressure data

a robotic device or other load bearing structure as described

points (e.g., pressure data points 1501-1512 from FIG. 15A).herein). The x-axis and the y-axis can correspond to locations

A person can be determined or identified as being posturally

of the pressure data points. For example, in the case where a stable or posturally unstable (e.g., step 1535 from FIG. 15B)pressure sensor is located on a shoe worn by a person/subject, 35 based on a number of times the person is in the dynamicthen the x-axis can represent the lateral locations (e.g., right postural state. For example, if the number of times the personand left foot) on the person and the y-axis can represent

transitions from a static postural state to a dynamic postural

fore-aft locations (e.g., toe and heel) on the person's foot. state exceeds a predetermined threshold, then the person canA method for determining postural stability (e.g., of a

be identified as being posturally unstable.

person or a robot or vehicle, etc.) can include acquiring a 40 In some embodiments, data points (e.g., pressure dataplurality of pressure data points (e.g., pressure data points points 1501-1512 from FIG. 15A) from input sensors (e.g.,1501-1512) over a period of time (e.g., at time t_i, t_i+1 ... pressure sensors) can be processed to determine if a conditiont_i+n) from at least one pressure sensor (step 1520). A pos- occurs. For example, a rule can include measuring an averagetural state can be identified for each pressure data point to velocity (e.g., of a person or other load bearing structure) overgenerate a plurality of postural states (e.g., which can include 45 a period/window of time. In some embodiments, the rule ora range of postural states) (step 1525). A postural state of a condition used to assess/determine postural stability can beperson (e.g., or other load bearing structure) at a point in time

based on the number of velocity related emission states out of

can be determined based on at least the plurality of postural

a number of points/periods of time (e.g., whether there arestates (step 1530). In some embodiments, the postural state a more than 100 high-velocity emission states out of 150 timepoint in time is the subsequent postural state of the person 50 periods, etc). In some embodiments, the rules based methods(e.g., at time t_i+n+l). In some embodiments, the postural

do not use the number of dynamic states to determine a

state of the person is determined at a later point in time, person's overall postural state (e.g., determining the posturalt_i+n+m where m is a value corresponding to how much time stability). A simple thresholding algorithm can take the out-has lapsed between when the pressure data points were puts themselves, (e.g., such as the velocity of a person) inacquired to when the data points are analyzed to determine the 55 eitherabinned form (e.g., inthe emissionmatrix) orunbinnedpostural state. In some embodiments, a person can be identi- form.fied as posturally stable or unstable based on the data points/

In some embodiments, the plurality of postural states based

postural states (step 1535). on the pressure data points 1501-1512 follows a punctuatedA method for determining postural stability (e.g., stability equilibrium (e.g., as described in FIG. 14). A continuous

of a person) can also include the step of acquiring at least a 60 series of static postural states can define an equilibrium. Byfirst pressure data point and a second pressure data point from way of example, each of the plurality of pressure data pointsat least one pressure sensor (e.g., sensor/input sensor from the

1501-1512 can correspond to the following exemplary/illus-shoe, walker, prosthetic leg, walking aid, portion of a robotic trative sequence of postural states, where "S" corresponds todevice or other load bearing structure as described herein). a static postural state and "D" corresponds to a dynamicThe first pressure data point can correspond to pressure data 65 postural state: D (e.g., data point 1501), D (e.g., data pointpoint 1501 and the second pressure data point can be 1502. 1502), S (e.g., data point 1503), S (e.g., data point 1504), SThe method can also include identifying a first postural state

(e.g., data point 1505), S (e.g., data point 1506), D (e.g., data

US 8,011,229 B221

point 1507), D (e.g., data point 1508), S (e.g., data point1509), S (e.g., data point 1510), S (e.g., data point 1511), S(e.g., data point 1512), etc. A series of static postural states(e.g., and corresponding pressure data points 1503-1506 and1509-1512) can be grouped together to define an equilibrium 5

1513 or 1514. A person can be determined/identified as beingposturally stable or posturally unstable (e.g., step 1535 fromFIG. 15B) based on a number of distinct equilibria 1513 and1514. By way of example, if the number of distinct equilibria(e.g., the number of series/groups of static postural states) 10

exceeds a certain threshold, then the person can be identifiedas being posturally unstable.

The "postural state of the person at a point in time" (e.g.,step 1530 from FIG. 15B) can be defined, for example, as apast postural state of the person, a current postural state of the 15

person or a subsequent postural state of the person. Any oneof a past postural state, current postural state or a subsequentpostural state of the person canbe determined by looking at oranalyzing the plurality of pressure data points 1501-1512acquired from pressure sensor. 20

In some embodiments, the postural state of the person at apoint in time (e.g., step 1530 from FIG. 15B) can be calcu-lated or determined in real time as the pressure data points1501-1512 are being acquired by a sensor. Pressure datapoints 1501-1512 can represent a postural state of the position 25

at each point in time during an acquisition period (e.g., step1520 from FIG. 15B). For example, pressure data points1501-1512 can represent pressure data points at times t_l,L2, t_3 ... t_12, where points of time t_1 through t_12 candefine a data acquisition period of time. If the postural state of 30

a person at a point in time is calculated in real time, pressuredata points 1501-1512 can be acquired and a postural state ofthe person at time t-13, etc. (e.g., a subsequent postural state),etc. can be calculated.

In some embodiments, pressure data points 1501-1512 can 35

be acquired over a period of time and later used to calculate apostural state of the person at a later point in time (e.g., 6months later). For example, as stated above, pressure datapoints 1501-1512 from FIG. 15A can represent pressure datapoints at times t_1, t_2, t_3 ... t_12. By way of example, 40

pressure data points 1501-1512 can laterbe used for diagnos-tic purposes or analyzed at a later point in time to project whatthe current postural state of the person is and what a subse-quent postural state of the person might be. For example,pressure data points 1501-1512 acquired during times t_l, 45

L2, t_3 ... t_12 can be used at a later point in time (e.g., timepoint t_50) to determine a current postural state (e.g., at timepoint t_50), a previous postural state (e.g., at time point t_49)or subsequent postural state (e.g., at time point t_51).

In some embodiments, a Hidden Markov Model can be 50

used to calculate a postural state of the person (e.g., step 1530from FIG. 15B) at a point in time (e.g., a past postural state,current postural state or subsequent postural state of the per-son). For example, pressure data points (e.g., data points1501-1512 from FIG. 15A) can be used in a Hidden Markov 55

Model to calculate a subsequent postural state of a person.Each pressure data point 1501-1512 can be identified as typeof postural state (e.g., static postural state or dynamic posturalstate) (e.g., step 1525 from FIG. 15B). Pressure data points1501-1512 can be acquired over a period of time (e.g., acqui- 60

sition period at time points t_l, t_2, t_3 ... t_12, for example,in step 1520 in FIG. 15B). The range of postural states (e.g.,the sequence of postural states generated from pressure datapoints 1501-1512), a probability of transitioning between thetypes of postural states (e.g., probability of transitioning 65

between static state or dynamic state) and a current posturalsate (e.g., postural state at time point t_12) can be used in a

22Hidden Markov Model to determine a subsequent posturalstate (e.g., a postural state at time point t_13). A HiddenMarkov Model can be used to determine a past or a presentpostural state (e.g., a postural state at time point t_ll or t_12).

In some embodiments, a Bayesian segmentation can beapplied to the plurality of pressure data points (e.g., corre-sponding to pressure data points 1501-1512 from FIG. 15Aand determined, for example, at step 1525 from FIG. 15B) todetermine a postural state of a person at a point in time.

In some embodiments, the postural state of the person at apoint in time (e.g., a subsequent postural state of the person)(step 1530) is determined or calculated based on at leastprobability of transitioning between the static postural stateand the dynamic postural state. The probability of transition-ing between states can be calculated based on the plurality ofpostural states of the person over the period of time (e.g., overan acquisition period). For example, the probability of tran-sitioning between states can be determined by looking at thepostural states that correspond to the pressure data points1501-1512 that were acquired from the sensor (e.g., by look-ing at how the sequence of the postural states varies overtime)in step 1530 of FIG. 15B. By way of example, if pressure datapoints 1501-1512 correspond to the following sequence ofpostural states: D, D, S, S, S, S, D, D, S, S, S, S where "D" isa dynamic state and "S" is a static state, then any portion or allof the sequence can be used to calculate/determine a prob-ability of transitioning between the static postural state andthe dynamic postural state.

In some embodiments, each of the plurality of pressuredata points 1501-1512 from FIG. 15A reflects a location of acenter of mass of the person at a point in time. For example,if the sensor is on a shoe worn by a person, as the person isshifting/moving/swaying, the center of mass/gravity/force ofthe person can move as well. In some embodiments, a changeof the location of the center of mass of the person over theperiod of time is determinedbased on theplurality of pressuredata points 1501-1512. A selected set of the plurality ofpressure data points can be grouped (e.g., groups 1513 or1514) as corresponding to a static or dynamic postural state ofthe person. The selected set of the plurality of pressure datapoints can be grouped based on the corresponding location ofcenter of mass/gravity/force. For example, pressure datapoints 1503-1506 and 1509-1912 can indicate that the per-son's center of mass/gravity/force has remained relativelystable (e.g., a static postural state) because the location of thecenter of mass has not changed relative to the other datapoints. However, pressure data points 1501, 1502, 1507 and1508 can indicate that the location of the person's center ofmass/gravity/force is changing (e.g., a dynamic posturalstate).

In some embodiments, a person's activity can be moni-tored in addition to acquiring pressure sensor information.For example, an acceleration of a person over time can beacquired (e.g., by use of a measurement device such as anaccelerometer). In some embodiments, a location of the per-son can also be acquired (e.g., by using a GPS device). Infor-mation regarding a person's activity canbe used, for example,for diagnostic purposes.

In some embodiments, the pressure data points 1501-1512(e.g., information acquired from pressure sensors on a loadbearing structure as described herein) can be analyzed using,for example, Fourier transform or other signal processingtechniques. In some embodiments, the pressure data pointscan be analyzed to determine if there is a periodicity to thedata (e.g., pressure data points 1501-1512 from FIG. 15A). Aperiodicity of the pressure data points can be used in connec-

US 8,011,229 B223

tion with biometric data (e.g., heart rate, breathing, etc.) andcan be used for diagnostic purposes.

The above-described embodiments, methods and systemscan also be used in connection with feedback (e.g., immediatefeedback). By way of example, feedback (e.g., real-time orimmediate) can be used to alert a person or a load bearingstructure as described herein (e.g., robot, vehicle, etc.) Thefeedback could be used in connection with the determinedpostural stability (e.g., to alert the person that they are or willbe posturally unstable) (e.g., step 1540 of FIG. 15B). By wayof example, feedback could be accomplished via vibration(e.g., stimulation or vibration at feet or elsewhere), via vision(e.g., presenting the pressure data to the person's visual field,either on a phone, special glasses, or modified contact lenses),via hearing (e.g., head-phones or bone-phones) via taste (us-ing some sort of cartridge in the person's mouth), via tem-perature (e.g., at feet or elsewhere), via other sensory mecha-nisms (e.g., a tactile mechanism), or any combination thereof.This feedback could be used to improve balance or in con-nection with balance training or physical therapy, etc.

FIG. 16 is an illustrative graph comparing the number ofequilibria 1600 from subjects. Data (e.g., such as datadescribed above in FIGS. 10-14 taken from input sensors)taken from subjects with their eyes open 1605 and subjectswith their eyes closed 1610. As shown in the graph, thenumber of equilibria (e.g., equilibria as described above inFIGS. 10-14) increased when the subjects had their eyesclosed (i.e. which could lead to imbalance/postural instabil-ity) as compared to when the subjects had their eyes opened.

The above-described systems and methods can be imple-mented in digital electronic circuitry, in computer hardware,firmware, and/or software. The implementation can be as acomputer program product. The implementation can, forexample, be in a machine-readable storage device, for execu-tion by, or to control the operation of, data processing appa-ratus. The implementation can, for example, be a program-mable processor, a computer, and/or multiple computers.

A computer program can be written in any form of pro-gramming language, including compiled and/or interpretedlanguages, and the computer program can be deployed in anyform, including as a stand-alone program or as a subroutine,element, and/or other unit suitable for use in a computingenvironment. A computer program can be deployed to beexecuted on one computer or on multiple computers at onesite.

Method steps can be performed by one or more program-mable processors executing a computer program to performfunctions of the invention by operating on input data andgenerating output. Method steps can also be performed byand an apparatus can be implemented as special purpose logiccircuitry. The circuitry can, for example, be a FPGA (fieldprogrammable gate array) and/or an ASIC (application spe-cific integrated circuit). Modules, subroutines, and softwareagents can refer to portions of the computer program, theprocessor, the special circuitry, software, and/or hardwarethat implements that functionality.

Processors suitable for the execution of a computer pro-gram include, by way of example, both general and specialpurpose microprocessors, and any one or more processors ofany kind of digital computer. Generally, a processor receivesinstructions and data from a read-only memory or a randomaccess memory or both. The essential elements of a computerare a processor for executing instructions and one or morememory devices for storing instructions and data. Generally,a computer can include, can be operatively coupled to receive

24data from and/or transfer data to one or more mass storagedevices for storing data (e.g., magnetic, magneto-opticaldisks, or optical disks).

Data transmission and instructions can also occur over a5 communications network. Information carriers suitable for

embodying computer program instructions and data includeall forms of non-volatile memory, including by way ofexample semiconductor memory devices. The informationcarriers can, for example, be EPROM, EEPROM, flash

10 memory devices, magnetic disks, internal hard disks, remov-able disks, magneto-optical disks, CD-ROM, and/or DVD-ROM disks. The processor and the memory can be supple-mented by, and/or incorporated in special purpose logic

15 circuitry.To provide for interaction with a user, the above described

techniques can be implemented on a computer having a dis-play device. The display device can, for example, be a cathoderay tube (CRT) and/or a liquid crystal display (LCD) monitor.

20 The interaction with a user can, for example, be a display ofinformation to the user and a keyboard and a pointing device(e.g., a mouse or a trackball) by which the user can provideinput to the computer (e.g., interact with a user interfaceelement). Other kinds of devices can be used to provide for

25 interaction with a user. Other devices can, for example, befeedback provided to the user in any form of sensory feedback(e.g., visual feedback, auditory feedback, or tactile feedback).Input from the user can, for example, be received in any form,including acoustic, speech, and/or tactile input.

30 The above described techniques can be implemented in adistributed computing system that includes a back-end com-ponent. The back-end component can, for example, be a dataserver, a middleware component, and/or an applicationserver. The above described techniques can be implemented

35 in a distributing computing system that includes a front-endcomponent. The front-end component can, for example, be aclient computer having a graphical user interface, a Webbrowser through which a user can interact with an exampleimplementation, and/or other graphical user interfaces for a

40 transmitting device. The components of the system can beinterconnected by any form or medium of digital data com-munication (e.g., a communication network). Examples ofcommunication networks include a local area network(LAN), a wide area network (WAN), the Internet, wired net-

45 works, wireless networks, a packet-based network, and/or acircuit-based network.

Packet-based networks can include, for example, the Inter-net, a carrier internet protocol (IP) network (e.g., local areanetwork (LAN), wide area network (WAN), campus area

5o network (CAN), metropolitan area network (MAN), homearea network (HAN)), a private IP network, an IP privatebranch exchange (IPBX), a wireless network (e.g., radioaccess network (RAN), 802.11 network, 802.16 network,general packet radio service (GPRS) network, HiperLAN),

55 and/or other packet-based networks. Circuit-based networkscan include, for example, the public switched telephone net-work (PSTN), a private branch exchange (PBX), a wirelessnetwork (e.g., RAN, bluetooth, code-division multiple access(CDMA) network, time division multiple access (TDMA)

6o network, global system for mobile communications (GSM)network), and/or other circuit-based networks.

The system can include clients and servers. A client and aserver are generally remote from each other and typicallyinteract through a communication network. The relationship

65 of client and server arises by virtue of computer programsrunning on the respective computers and having a client-server relationship to each other.

US 8,011,229 B225

The handheld device can include, for example, a computer,a computer with a browser device, a telephone, an IP phone,a mobile device (e.g., cellular phone, personal digital assis-tant (PDA) device, laptop computer, electronic mail device),and/or other communication devices. The browser deviceincludes, for example, a computer (e.g., desktop computer,laptop computer) with a world wide web browser (e.g.,Microsoft® Internet Explorer® available from MicrosoftCorporation, Mozilla® Firefox available from Mozilla Cor-poration). The mobile computing device includes, forexample, a personal digital assistant (PDA).

Comprise, include, and/or plural forms of each are openended and include the listed parts and can include additionalparts that are not listed. And/or is open ended and includesone or more of the listed parts and combinations of the listedparts.

While this invention has been particularly shown anddescribed with references to preferred embodiments thereof,it will be understood by those skilled in the art that variouschanges in form and details may be made therein withoutdeparting from the scope of the invention encompassed by theappended claims.

What is claimed is:1. A computer-implemented method for determining pos-

tural stability of a person, the method comprising:acquiring, by a processor, a plurality ofpressure data points

over a period of time from at least one pressure sensor;identifying, by the processor, a postural state for each pres-

sure data point to generate a plurality of postural states;and

determining, by the processor, a postural state of the personat a point in time based on at least the plurality of pos-tural states and a probability of transitioning between atleast one of the plurality of postural states and anotherpostural state, wherein the postural state is at least one ofa static postural state or a dynamic postural state.

2. The method of claim 1, wherein the person is in thedynamic postural state when moving from a first static pos-tural state to a second static postural state.

3. The method of claim 1, further comprising determiningthat the person is posturally stable or posturally unstablebased on a number of times the person is in the dynamicpostural state.

4. The method of claim 1, wherein the plurality of posturalstates follows a punctuated equilibrium, a continuous seriesof static postural states defining an equilibrium.

5. The method of claim 4, further comprising determiningthat the person is posturally stable or posturally unstablebased on a number of distinct equilibria.

6.The method of claim 1, wherein determining the posturalstate of the person at the point in time is based on at least aprobability of transitioning between the static postural stateand the dynamic postural state.

7. The method of claim 6, further comprising calculatingthe probability of transitioning between the static posturalstate and the dynamic postural state based on the plurality ofpostural states of the person over the period of time.

8. The method of claim 1, further comprising acquiring atleast one of an acceleration or a location of the person over theperiod of time.

9. The method of claim 1, further comprising determininga range of postural stability states based on the plurality ofpressure data points.

10. The method of claim 1, wherein determining comprisesusing a machine learning technique on the plurality of pres-sure data points to generate the plurality of postural states.

2611. The method of claim 10, wherein determining com-

prises using a Hidden Markov Model on the plurality ofpressure data points or applying Bayesian segmentation to theplurality of pressure data points to generate the plurality of

5 postural states.12. The method of claim 1, wherein acquiring comprises

acquiring a plurality of pressure data points from at least onepressure sensor on at least one of a shoe, a sock, a sole insert,a cane, a crutch, a walker, a walking aid, a prosthetic leg, or a

io robotic leg.13. The method of claim 1, wherein each of the plurality of

pressure data points reflects a location of a center of mass ofthe person at a point in time.

15 14. The method of claim 13, wherein identifying comprisesgrouping a selected set of the plurality of pressure data pointsas corresponding to a static postural state based on the loca-tion of the center of mass of the person of each of the pluralityof pressure data points.

20 15. The method of claim 1, wherein determining comprisesdetermining in real time, a subsequent postural state of theperson based on the plurality of pressure data points.

16. The method of claim 1, wherein determining comprisesdetermining a current postural state or a subsequent postural

25 state of the person based on the plurality of pressure datapoints, wherein the plurality of pressure data points wereacquired during a previous period in time.

17.A computer-implemented method for determining pos-tural stability, the method comprising:

so acquiring, by a processor, at least a first pressure data pointand a second pressure data point from at least one pres-sure sensor;

identifying, by the processor, a first postural state and asecond postural state based on the first and second pres-

35 sure data points, wherein identifying comprises identi-fying the first postural state as static or dynamic and thesecond postural state as static or dynamic; and

determining, by the processor, a postural state at a point intime based on at least the first postural state, the second

40 postural state, and aprobability of transitioning betweenat least one of the first or second postural states andanother postural state.

18. The method of claim 17, wherein determining com-prises determining a subsequent postural state of a person.

45 19. The method of claim 17, wherein determining com-prises determining a subsequent postural state of a personbased on a probability of transitioning between the staticpostural state and the dynamic postural state.

20. The method of claim 17, further comprising determin-50 ing that a person is posturally stable or posturally unstable

based on at least the identified first postural state and theidentified second postural state.

21. The method of claim 17, wherein acquiring comprisesacquiring a plurality of pressure data points from at least one

55 pressure sensor on at least one of a shoe, a sock, a sole insert,a cane, a crutch, a walker, a walking aid, a prosthetic leg, arobotic leg, a vehicle, or an axle connected to at least onewheel.

22. A system for determining postural stability of a person60 comprising:

at least one pressure sensor coupled to the person thatacquires a plurality of pressure data points over a periodof time;

means for identifying a postural state for eachpressure data65 point;

means for generating a plurality of postural states of theperson over the period of time; and

US 8,011,229 B227 28

means for determining a postural state of the person at a 23. The system of claim 22, wherein each pressure datapoint in time based on the plurality of postural states and point corresponds to a center of gravity of the person at a pointa probability of transitioning between at least one of the in time.plurality of postural states and another postural state,wherein the postural state of the person is one of a static spostural state or a dynamic postural state. * * * * *

united states patent patent us 8,011,229 b2 lieberman et al

Documents