Wearable Automatic Feedback Devicesfor Physical Activities

Daniel SpelmezanRWTH Aachen University52056 Aachen, Germanyspelmezan@cs.rwth-

aachen.de

Adalbert SchanowskiRWTH Aachen University52056 Aachen, Germany

adalbert.schanowski@rwth-aachen.de

Jan BorchersRWTH Aachen University52056 Aachen, Germanyborchers@cs.rwth-

aachen.de

ABSTRACTWith this work we want to illustrate new ways to assiststudents during sports training and to enhance their learn-ing experience. As an example, we present the design ofa wearable system intended for snowboard training on theslope. The hardware platform consists of a custom-builtsensor/actuator box and a mobile phone acting as host de-vice. These devices run algorithms for activity, context, andmistake recognition, and trigger feedback in response to clas-sification results. Instructors can use such a system to auto-matically supervise posture and motion of students, i.e., todetect common mistakes that are difficult to recognize whenobserving students from far away, and to provide immediateaudible or tactile feedback for corrections during courses.The presented approach can further be applied to super-vise posture and to alert users to potentially harmful bodymovements performed during daily physical activities.

KeywordsHCI, mobile computing, wearable computing, assistive tech-nologies, context awareness, tactile feedback, snowboarding.

1. INTRODUCTIONLearning new sports is often difficult and time consum-

ing. Students need to practice for a long time until theycan correctly perform the necessary techniques of a sportsdomain. In some sports, such as tennis or golf, instructorscan always talk to students and explain correct technique.Moreover, an instructor can physically guide, for example,the student’s arm to demonstrate correct strokes. Such fre-quent concurrent feedback during exercises can be beneficialfor acquiring new motor skills [13]. In sports such as snow-boarding or skiing, however, the instructor cannot be withthe students while going down the slope. Due to this spatialseparation, a snowboarding instructor cannot directly talkto students when they incorrectly perform exercises (Fig. 1).The instructor typically provides feedback after exercises.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.BodyNets ’09 Los Angeles, CA, USACopyright 2008 ICST 978-963-9799-41-7 ...$5.00.

Figure 1: Two beginners descend the slope. Theinstructor cannot talk to the students to give themfeedback on their mistakes.

We present our work on wearable automatic feedback de-vices that are intended to support instructors during coursesand to help students in learning correct technique. Our ap-proach is especially useful when the spatial separation be-tween the instructor and the student is too great to talk orwhen the instructor cannot focus all of his attention on anindividual student. To demonstrate the feasibility of suchsystems and their potential during training, we have itera-tively designed and tested a wearable system for snowboardtraining, in collaboration with snowboarding instructors andstudents. This system classifies basic context informationon the slope and can be used to detect and to respond tocommon snowboarding mistakes in realtime. Tiny sensors,unnoticed by humans when woven into clothes or attachedto sports equipment, measure body movements and postureduring training. Actuators, such as vibration motors, rendertactile patterns across the body that alert users to incorrectmovements and that communicate hints for corrections. Au-dio messages through headphones are an alternative way toprovide instructions, however, they are less appropriate dur-ing the ride. Audio feedback hides valuable environmentalcues, such as sound coming from approaching skiers or soundstemming from changing snow conditions.

Another promising application domain for the describedfeedback systems is daily physical activity. For example,to correctly pick up a heavy box from the floor, the upperbody should remain straight while bending and stretchingthe legs. Many people, however, keep their legs straight andbend only their upper body downwards from the waist to

lift the object. These movements can lead to serious backinjuries when executed over and over again. A wearablesystems that monitors body movements can alert people tosuch wrong habits and signal how to adjust posture.

2. CONTRIBUTIONSThis work outlines the rationale for wearable automatic

sports assistants and provides an example of such a systemin a real-world setting. We start by introducing our visionof future sports training and the benefits that automaticfeedback systems offer to both instructors and students. Wethen describe the hardware design of our system, which wasmotivated by interviews with snowboarding instructors. Ex-amples and results from a field study conducted on the slopeillustrate that our approach can be used to develop an inter-active system for supervising posture and for teaching cor-rect snowboarding technique. Results from this work canbe applied to other domains where physical activity and thecorrect execution of movements is essential.

3. RELATED WORKThe growing interest in wearable computing for recogniz-

ing and correcting physical movements manifests in a varietyof projects. Michahelles et al. [7] developed a system to col-lect data of a skier’s run for off-line analysis. Similar toSESAME 1, this system targets coaches and elite athleteswho aim at improving sports performance. Takahata et al.[12] presented a realtime learning environment that uses au-dio feedback to teach the correct timing of a single karatepunch. Kwon et al. [4] developed a motion training systemfor taekwondo. Kunze et al. [3] conducted an experiment torecognize tai chi movements. Chi et al. [1] built a wearablesensing system to support judges in scoring taekwondo spar-ring matches. Lindeman et al. [5] described a setup to assistpatients during physical therapeutical exercises. Other sys-tems were designed for leisure and fun. Muller et al. [8],for example, presented an exertion interface that supportssports over a distance and fosters social interaction.

Some of these systems, however, were not designed for re-altime data analysis and feedback [3, 7]. Existing interactivesystems recognize only simple movements, such as punches[1, 8, 12]. Other approaches require expensive hardwareto capture body motions, use powerful processors to ana-lyze sensor data, and rely on prior training on a large userdatabase to build appropriate models that recognize andclassify motions [3, 4]. Another disadvantage most systemsface is their inherent design, which limits their usage to thelab. Only few systems have been designed to work in mobileor outdoor settings [7].

4. THE VISION OF SPORTS TRAININGTo illustrate benefits that automatic feedback systems

could offer during training, we have chosen the domain ofwinter sports. The distance on the slope prohibits immediatecommunication between instructor and students. Thick skisuits conceal body movements and make it difficult to assessthe performance of a student who rides far away. Moreover,the trainer has to split his attention across several studentswho participate in the course, which further reduces the fre-quency of feedback that an individual student receives.

1Sensing for Sport And Managed Exercise, sesame.ucl.ac.uk

frontside edge

backside edge

nose tail

front foot back foot

Figure 2: Riding with too much weight on the backfoot can lead to falls (photo by Martin Schliephake[2]). Future training systems could use actuators onthe body (indicated by white dots) to provide tac-tile instructions for corrections. The upper sketchillustrates snowboarding terminology.

Snowboarding instructors could use automatic feedbackdevices as tools that supervise and correct students’ mis-takes during exercises. A trainer might want to focus on aparticular mistake that she noticed while observing one ofher course participants. For example, a student might in-correctly shift his weight too much towards the back footduring turns (Fig. 2). To increase the student’s awarenessof his wrong movements and to support him in correctingweight distribution during the ride, as opposed to providingsuggestions for improvement only after the ride, the wear-able assistant takes over the task of analyzing posture usingon-body sensors. The system further provides immediatefeedback to alert the rider to incorrect postures. If an incor-rect weight distribution during turns is detected, a uniquetactile pattern will be rendered at the student’s left thigh(the front foot) to indicate correct weight distribution onthe snowboard. This tactile feedback, made of a sequenceof short vibrations running downwards on the thigh, signalsshift your weight from the back foot towards the front foot.

Once the student has learned to better adjust his weightdistribution on the board, the instructor might decide tofocus on another mistake, such as counter-rotation duringturns. For this, the wearable training system measures theorientation of the upper body relative to the orientation ofthe snowboard. If an incorrect upper body rotation is de-tected, a unique tactile pattern will be rendered around thestudent’s right shoulder to signal that the upper body shouldrotate towards the right instead of towards the left (Fig. 3).

Automatic training systems also offer benefits to amateurand experienced students who ride alone. Even skilled ridersmay have difficulty maintaining proper technique on chal-lenging slopes. Moreover, a rider’s subjective perception ofhis body posture often deviates from his actual stance. Thewearable assistant continuously monitors the rider’s per-formance during descents and provides immediate feedbackhow to correct posture and fine-tune movements.

5. STUDY OVERVIEWWe started by conducting an exploratory formative study

with domain experts to better understand the problem andto inform our design of a wearable prototype. Following

Figure 3: The student rotates his upper body to theleft, which is against the current riding directiontowards the right. This counter-rotation makes itdifficult to introduce the next turn towards the left(photo by Martin Schliephake [2]). The white dotsillustrate the possible placement of actuators.

our interviews with instructors, an initial study in the fieldhelped us to validate the concept, to experiment with differ-ent sensors, and to iteratively test and improve our system.Finally, we conducted an extensive user study on the slopewith potential end-users. During this study, we collectedand analyzed sensor data in order to investigate whetherit was possible to classify basic context information that isrequired to further interpret body movements, to recognizemistakes, and to teach proper technique in realtime.

6. INFORMING THE DESIGN OF SNOW-BOARD TRAINING SYSTEMS

Interviews with InstructorsWe interviewed six snowboarding instructors to better un-derstand how they teach and to gain more insight into themost common mistakes that occur in this sport. The in-structors confirmed our initial assumptions that they can-not observe all students at the same time on the slope. Theyexplained that it is often impossible to give students instantfeedback during an exercise. A student receives immediatefeedback only in cases when the instructor slowly rides be-side a beginner during the first lessons.

All of the interviewed instructors stated that our idea toprovide instant feedback during training could be useful tosupport both instructors and students during courses. Theyimagined using a wearable assistant themselves to fine-tunetheir own movements as well as using the system in theirown courses, allowing advanced students to focus on a par-ticular mistake. The instructors also pointed out that manybeginners do not perceive or do not react to immediate feed-back while they perform an exercise. Beginners are usuallytoo focused on keeping their balance. This suggested thatour system might be more useful for students who alreadyhave some experience in snowboarding, such as advancedand experienced riders who want to improve their skills.

Snowboarding MistakesThree common snowboarding mistakes [2] were identifiedduring the interviews: Incorrect weight distribution duringturns, incorrect rotation of the upper body during turns,and insufficient knee bending. The neutral position denotes

the correct pose of a rider on the snowboard and can beused as reference posture to find potential mistakes duringthe ride. In neutral position, the weight is central over theboard and distributed equally between both feet. Legs andankles are flexed. This flexing acts as a natural suspensionto compensate uneven terrain, for example when riding overbumps in the slope. Shoulders and hips are in line withthe feet’s stance on the snowboard. The head is up andthe rider looks towards the riding direction. We will nowbriefly summarize the three identified mistakes and mentionpossible sensors that can be used for automatically analyzingposture and detecting erroneous body movements.

Incorrect weight distribution (Fig. 2): To begin a newturn, the weight should be shifted towards the front foot. Af-ter pivoting the board, the rider should distribute his weightequally between both feet. However, many riders tend toincorrectly lean their upper body towards the tail. Theresulting posture shifts the weight towards the back foot,which makes it difficult to initiate and to pivot the snow-board across the fall line. This mistake can be addressedwith force-sensitive resistors (FSR) inserted into the boots,which measure the weight distribution between both feet.

Incorrect upper body rotation (Fig. 3): The rider’s shoul-ders and the waist should remain in the same plane withthe feet’s stance during the ride. To introduce a turn, theupper body should be rotated towards the intended ridingdirection. Yet many riders tend to twist their upper bodycontrary to the turning direction, which makes it hard toinitiate and perform the next turn. This mistake can beaddressed with gyroscopes or with digital compasses thatmeasure the orientation of the upper body relative to theorientation of the snowboard. Stretch sensors woven intothe garment [6] or magnetic field sensors [9] might be alter-native approaches for identifying this particular mistake.

Insufficient knee bending: Bending the legs helps as anatural suspension to compensate for uneven terrain. Someriding techniques also require alternating from high to low(flexing and extending the legs) while pivoting the boardduring turns. Many riders, however, cannot correctly assessif they bend their legs sufficiently or not. They tend tostretch their legs and to bend their upper body downwardsfrom the waist. This mistake can be addressed with bendsensors or with stretch sensors attached to the rider’s joints,which measure the degree of flexion.

System SetupThe conducted interviews with domain experts helped us tobetter understand the requirements for a snowboard train-ing system that senses motion, detects mistakes, and pro-vides feedback in realtime. We were first looking at existinghardware solutions that we could use for building a wear-able prototype. In particular, we were interested in a systemthat allowed us to easily experiment and exchange differ-ent sensors and actuators on the fly and to log sensor datafor initial tests. Moreover, the system should allow us toimplement customized algorithms for context and mistakerecognition as well as higher application logic for comparingthe student’s current posture to the intended riding pos-ture required by particular snowboarding techniques. Al-though specialized sensor systems for motion capture exist(see xsense.com), these systems did not prove to be conve-nient for our tasks. Such systems are still limited in termsof customizability and programmability. We thus decided to

SensAct box (Arduino BT)- sensing- controlling actuators- basic signal processing (low-pass filters, ...)- classification algorithms (posture, activity)

Host device (Java ME)- control SensAct boxes- data logging- visualize sensor data- basic signal processing- classification algorithms- higher application logic

8 sensors

Bluetoothcommunication

6 actuators

Figure 4: The system architecture of our platform.

Figure 5: The SensAct box allows replacement ofsensors and actuators at runtime.

design our own sensor/actuator box that better meets therequirements detailed in our vision. Throughout the con-ducted user studies described in this work, we continued toiteratively improve and refine our platform for outdoor use.

Hardware PlatformOur system consists of two units: A standard mobile phoneand our custom-built SensAct sensor/actuator box. The mo-bile phone, a Nokia N70 in our case, acts as host device.Fig. 4 illustrates the system architecture and the tasks ofeach unit. The SensAct box (Fig. 5) contains a BluetoothArduino board2, an open-source electronics prototyping plat-form. A custom-built motor controller for the Arduino pro-vides vibrotactile feedback through actuators, such as smallvibration motors. Actuators are connected via TS, sensorsvia SUB-D connectors to avoid loose connections. The di-mensions of the SensAct box are 15x8x5 cm (6x3x2 in).

Software ArchitectureBluetooth allows the host device to concurrently control upto seven SensAct boxes. We developed software libraries forboth the SensAct box (programmable in a C-like language)and the host device (Java, Python). These libraries includealgorithms for basic signal processing, posture classification,and activity recognition (described below) used for imple-menting stand-alone programs. Programs can be freely dis-tributed between several SensAct boxes. All algorithms canalternatively run on the host, using raw sensor measure-ments streamed in realtime from multiple SensAct boxes.

2www.arduino.cc

The host sends control messages to the SensAct boxes.These messages define, for example, the sampling rate forsensors or trigger actuators for rendering vibrotactile feed-back. Other control messages start and stop the streamingof raw sensor data or of classification results. We found nonoticeable delay caused by the Bluetooth communicationbetween SensAct box and host device. For example, theaverage time to send a six bytes command that activatesactuators connected to the box is approximately 39 ms.

Initial Testing of the TechnologyUsing our sensor system, we conducted an exploratory pi-lot study in an indoor winter sport resort with three snow-boarders at advanced beginner level. This study aimed attesting our system under real-world conditions. We furthercollected a first set of raw sensor data for off-line analysis.

Insoles with TouchMicro-103 FSRs were inserted into theboots to measure the amount of force applied by the feet.We placed two FSRs under the ball of each foot (1st and 5thmetatarsal bones) and one FSR under the heel (calcaneus).These sensors had a diameter of 10 mm and measured forcesup to 667 N (68 kg). To measure the amount of knee flexion,BendShort3 bend sensors (87 mm long) were wrapped infoam to increase their robustness and attached to the back ofeach knee with kneepads. Though some participants statedthey noticed sensor cables under the insoles and the foam atthe knees, this equipment did not cause discomfort and didnot hinder them in their movements. However, adjusting allsensors properly was time consuming and lasted one hour.

All six sensors were connected to our custom-built sensorbox. We further used two Shake SK64 devices to measurerotation of the rider’s upper body relative to the rotation ofthe snowboard. The Shake SK6 is a matchbox-sized inertialmeasurement unit with built-in digital compass algorithmand Bluetooth communication. We attached one Shake de-vice to the lower front leg and another Shake device to theupper body of the rider with hook and loop fasteners.

Fig. 6 shows our prototype system with all sensors usedfor data collection during the field study. The host deviceconnected to all three sensor packs over the Bluetooth serialport profile and recorded raw sensor data at 20 Hz. We ad-ditionally captured the subjects on video while descending.The distance between the starting point on the slope andthe camera was about 60 meters. We chose this distanceto simulate a setting that was similar to what snowboardinstructors typically observe during courses. Each subject

3www.infusionsystems.com4www.samh-engineering.com

Figure 6: The first hardware prototype and sensorsetup to measure the rider’s posture and motions.

descended the same part of the slope two times. All partic-ipants wore the SensAct box in a small waist bag and thehost device in their pocket. We experienced no data loss orconnection problem and found no significant overhead in theBluetooth communication between the wireless devices.

Preliminary Results and Instructors’ OpinionTo inspect the raw sensor measurements off-line, we syn-chronized sensor and video recordings using custom-writtensoftware. The sensor recordings revealed that setting thresh-olds on sensor signals was sufficient to reliably detect theamount of knee bending and to estimate the weight distribu-tion between toes and heels on the snowboard. Accelerationduring the ride, however, influenced the compass measure-ments, thus making it difficult to reliably detect incorrectrotation of the upper body only with a digital compass.

Following our initial tests, we presented our wearable sys-tem to eleven snowboard instructors and 28 skiing instruc-tors who participated in an advanced training course forinstructors. One of the snowboarders and three of the skierswere professional instructors who led the course. We in-troduced the participants to our vision of a wearable snow-boarding assistant and explained how the sensing systemworked. Finally, we visualized sensor data collected duringinitial testing. Examples included amount of knee bendingand weight distribution between toes and heels, since thesesensors provided the best measurements during descents.

73% of the snowboard instructors and 68% of the ski in-structors considered our idea to sense the rider’s motion andto provide realtime feedback during snowboard and ski train-ing to be potentially very valuable. The other participantsquestioned whether an automatic system could correctly rec-ognize mistakes during the ride. They argued that withoutdetailed knowledge about the slope it is impossible to dis-tinguish between correct and incorrect movements. For ex-ample, in some cases, an instructor needs to see the gradientof the slope in order to assess if a student bends his legs suf-ficiently. Moreover, classifying certain body movements ascorrect or wrong also depends on the current context dur-ing the descent. For mistake recognition to work, the systemneeds to distinguish whether the rider performs a frontsideturn or a backside turn, which determines the set of validmovements. While the first objection is valid for a subsetof all possible mistakes, our prototype system might stillbe used to detect an incorrect posture or movements thatare independent of the slope’s characteristics, such as wrongweight distribution or upper body rotation during turns.

Figure 7: A user with the prototype system.

7. SYSTEM EVALUATION WITH USERSMotivated by the results of our initial test and the instruc-

tors’ opinion, we decided to conduct a formal user studyto further analyze sensor measurements that proved mostpromising for building a wearable snowboarding assistant atthe present time. The main goal of this study was to inves-tigate if it was possible to detect the instant when a riderbegan a new turn and to determine if the turn was per-formed on the frontside edge or on the backside edge. Dur-ing frontside turns the snowboarder faces uphill and rideson the frontside edge, during backside turns he faces down-hill and rides on the backside edge (see sketch in Fig. 2).As described by instructors, this context information duringdescents is essential to automate the task of recognizing mis-takes that do not conform to correct riding techniques. Weadditionally tested algorithms to recognize incorrect weightdistribution and insufficient knee bending. Finally, we eval-uated an algorithm for activity recognition that classifieswhether participants were pausing or riding. Algorithmshave been evaluated off-line on raw measurements recordedon the slope. We performed this step to optimize algorithmsfor the best recognition accuracy across all participants, be-fore the final implementation on our wearable platform.

Experimental SetupEight snowboarders (one female) aged 23-27 participated inthis study. On a scale ranging from level one (beginner) tolevel five (expert), one participant rated his skills as levelone, two as level two, three as level three, and two as levelfive. One of the expert snowboarders was a snowboard in-structor. Four participants stated that they had attendeda training course to improve their skills in snowboarding.On average the subjects snowboarded between one and twoweeks per year during holidays.

We used a similar hardware setup as in the pilot studyto record raw sensor data and to simultaneously captureour participants on video. Sensor measurements compriseddata from six force sensors inserted into the boots, two bendsensors attached to the knee, and one 2D accelerometer atthe upper arm (Fig. 7). The SensAct box sampled data at50 Hz and streamed measurements to the host device.

The distance between the starting point on the slope andthe camera was about 140 meters. By choosing this dis-tance, the subjects gained higher speed and performed moreconsecutive turns as compared to the pilot study. For eachsubject we recorded several runs. For some runs, partici-

pants were instructed to descend the slope as they alwaysdid. For other runs, participants were introduced to an al-ternative snowboarding technique that required stretchingand flexing the legs during turns. We asked participants toride according to this alternative technique in order to verifywhether they can perform the task as required and whethersensor measurements revealed knee flexion during turns.

Turn and Edge DetectionKnowing the riding edge and the instant when a new turnbegins is essential to further interpret if a rider’s movementsare correct or wrong. To determine whether the frontsideedge or the backside edge is subject to the highest pres-sure, we compare FSR measurements to reference valuesrecorded during an initial calibration step while standing onlevel ground. This comparison yields whether the rider per-forms a turn on the frontside edge or on the backside edge.Transitions from one edge to the other edge correspond tothe beginning of turns while pivoting the board.

The algorithm works as follows: Mean shifting first ad-justs raw sensor values by subtracting the corresponding ref-erence values. As a result, the new reference values become 0for all sensors. Simple exponential smoothing reduces sensornoise. The algorithm then sums the forces measured underthe balls of feet (SB), the forces measured under the heels(SH), and computes the difference D = SB − SH. Sim-ple moving average with window size w returns the meanweight distribution E = SMAw(D) on the snowboard. Eyields whether the weight is towards the balls or towardsthe heels and determines the riding edge. A threshold valueTE defines the tolerance range around the reference value 0,where we regard the weight distribution between the ballsand the heels as evenly distributed. The rider is riding onthe frontside edge if the weight distribution is towards theballs (E > TE). The rider is riding on the backside edgeif the weight distribution is towards the heels (E < −TE).The rider pivots the board from one edge to the other edgeif the weight is neither too much towards the balls nor toomuch towards the heels (−TE 6 E 6 TE).

To determine the best recognition rate depending on dif-ferent exponential smoothing factors α (0 6 α < 1), wecompared the output of our algorithm to the turns on thevideo footage. We considered true positives (recognizedturns), false negatives (missed turns), and false positives(false alarms). Experimental sessions using the recordingsfrom our pilot study allowed us to choose a threshold valueTE = 50 and a window size w = 25 (roughly 500 ms) to com-pute E. We used the first runs from our subjects as trainingset and the last runs as test set to evaluate the describedalgorithm for turn and edge detection.

ResultsFig. 8 shows an example output of the algorithm (blackcontinuous line) for turn and edge detection applied to theraw sensor measurements recorded from two FSR sensorsper foot (1st metatarsal bone, calcaneus). Classification re-sults for the training set were independent of the smooth-ing factor α. All 56 turns were correctly classified eitheras frontside turns or as backside turns. For the test set,we found an exponential smoothing factor of α = 0.9 torank highest. The algorithm correctly classified 59 out of 61turns (96.7%) and reported two temporary false positives.Lower smoothing factors ranked only slightly lower with at

-100

-50

0

50

100

150

200

250

300

350

7161 7764 8364 8964 9564 10165 10770

Time (ms)

Sen

so

r v

alu

es

D = SB - SH

E = SMA(D)

Algorithm output (edge)

Figure 8: Output of the algorithm for turn andedge detection (black continuous line): Backsideedge (Y = −50), frontside edge (Y = 50), pivoting(Y = 0). Transitions between −50 and 50 indicatethe beginning of new turns.

most five temporary false positives for the test set. Usingall three FSR sensors instead of two FSRs per foot did notsignificantly improve classification results.

DiscussionThe high recognition accuracy shows that turns and the rid-ing edge can be accurately classified with only two force sen-sors per foot. The slightly lower accuracy of the algorithmto recognize turns in the test set might stem from the sin-gle calibration step that we performed for each subject onlybefore the first descent. FSR sensors that were not appropri-ately placed under the feet can further skew measurementsand influence results; for some subjects we experienced dif-ficulties when trying to fit the insoles with the sensors intothe snowboard boots. Sensor cables under the soles of thefeet and displaced sensors after consecutive descents mightbe other reasons. Repeating the calibration step before eachdescent and using boots with built-in sensors might help toimprove the overall recognition accuracy in future.

The threshold TE corresponds to the sensitivity of thealgorithm to detect the riding edge; the algorithm interpretsa shift in weight towards the toes (heels) as riding on thefrontside (backside) edge. Low thresholds can be used todetect turns and to identify the riding edge for beginnerswho do not yet have the skills to increase the edge angle ofthe board and thus rather slide down the slope. For expertriders, who are skilled enough to carve on the edges, higherthresholds can assess the quality of their carving technique.

A similar algorithm as used for turn and edge detectioncan theoretically estimate the weight distribution betweenthe front foot and the back foot. Evaluation revealed thatestimating the weight between both feet on the snowboard,as opposed to between toes and heels, was not accuratelypossible even with three FSR sensors inserted into each boot.The front binding, which fastens the front part of the footto the board, influenced measurements and did not yield re-liable results. Future work should investigate whether moreFSR sensors per foot, sensors built directly into the board orinto the binding might help to detect whether riders incor-rectly shift their weight towards the back foot during turns.

-150

-100

-50

0

50

100

150

36689 39699 42721 45741 48748

Time (ms)

Sen

so

r v

alu

es

Left + right knee

Algorithm output (knees)

Algorithm output (edge)

Figure 9: Amount of knee flexion during the ride(red line): Flexed legs (Y = −50), stretched legs(Y = 50). Riding edge (black line): Backside edge(Y = −100), frontside edge (Y = 100).

Knee Flexion during the RideThe two bend sensors attached to the back of both kneesmeasured knee flexion and duration of flexion during theride. This information revealed whether participants rodewith straight knees for extended time periods. For one oftheir descents, the subjects were instructed to alternate be-tween extending the legs while pivoting the board and flex-ing the legs during traversal. To assess whether the partic-ipants performed this technique as required, we comparedbend sensor measurements to a threshold value TK thatwe recorded in neutral position before descending the slope.Fig. 9 depicts knee flexion in relation to the riding edge forone participant; the student stretched and bent her legs suffi-ciently while pivoting the snowboard. The temporal compo-nent of the classification results further helped to assess thequality of the student’s technique. Flexion almost coincidedwith pivoting from the backside edge to the frontside edgebut occurred slightly before pivoting from the frontside edgeto the backside edge. Though bend sensors worked reliablyduring most descents, the foam with the sensors occasionallyslipped out of the kneepad and had to be readjusted.

Activity Recognition: Stop and GoA wearable snowboarding assistant should analyze postureand provide feedback only if the student is riding. Duringbreaks the system should remain idle. To distinguish be-tween riding and standing, we measured acceleration in di-rection to the descent and vertical to the slope using one 2Daccelerometer, which was attached to the upper arm. Ouralgorithm summed the standard deviations of both acceler-ation components over the last 0.5 seconds and comparedthe result to a reference value measured while at rest. Thissimple activity recognition algorithm achieved an accuracyof 80.5% in classifying whether participants were riding orpausing. False positives occurred while riding at low speed,such as during the slow deceleration phase before rest orduring the slow acceleration phase after rest. The accu-racy of our approach to detect activity is only slightly lowercompared to [10], which extracted more features and used aclustering algorithm to differentiate between six activities.

8. DISCUSSION OF RESULTSThe described algorithms set thresholds on sensor signals

to classify posture and activity. While this straightforwardapproach does not require training of recognition models inadvance, our method does not scale to continuous motion.Machine learning techniques, such as hidden Markov models,better address recognition of such movements [3, 4]. Furtherwork has to be done until all snowboarding mistakes can beaccurately identified using body-worn sensors.

Despite that limitation, the obtained results are adequateto build interactive training systems that do not dependon tracking and differentiating fluent and continuous bodymotions. Classification results presented in this work revealtransitions between movements, such as changing weight dis-tribution or several degrees of joint flexion, and yield a pos-ture model of the body. This model allows to trigger real-time feedback intended to demonstrate correct execution ofbody movements that conform to specific sports techniques.

State machines can be used to implement such a behavior.In our case, feedback could be triggered at the appropriatetime during descents, reminding and guiding students dur-ing turns. For example, our system knows when transitionsbetween turns occur and whether the student currently per-forms a frontside turn or a backside turn. This informationcan be used to instruct correct technique for basic turns:While riding on the frontside edge, the wearable assistantnotifies the student to shift your weight to the front foot,followed by turn your upper body to the left to introduce thenext turn. While riding on the backside edge, instructionsare shift your weight to the front foot, followed by turn yourupper body to the right. To sensitize students for correct tim-ing to alternate from flexing and extending the legs duringconsecutive turns, the system provides instructions to bendyour legs as soon as a new turn has been detected, then tostretch your legs followed by turn your upper body to the left(right) in order to introduce the next turn. The describedapproach can be compared to an instructor who pushes orpulls the student in the correct directions to indicate correctposture, similar to tennis and golf instructors who guide thestudents’ arms to demonstrate correct strokes.

Tactile motion instructions [11] are vibration patterns trig-gered across the body, as introduced in our vision of futuresports training. These instructions represent specific bodymovements, such as shift your weight to the left foot, turnyour upper body to the right, or bend your legs. We proposeda set of ten such instructions and used our system to con-duct a study with ten snowboarders. The task was to iden-tify randomly triggered tactile instructions across the bodywhile descending the slope. Results revealed that our par-ticipants perceived very well tactile instructions (87% cor-rect) as compared to corresponding audio instructions (97%)played back over earplugs while snowboarding. Moreover,students responded on average one second faster to tactileinstructions than to audio instructions. This faster responsetime to tactile instructions over their audio counterparts isan important advantage for sports such as snowboarding;the rider has to react quickly to changing conditions andcontinuously adjusts posture to keep balance. A post-testquestionnaire revealed no significant difference between au-dio and tactile instructions regarding comfort of the system,intuitiveness of instructions, or distraction during the ride.

The results presented in this work further demonstratethat wearable computing applications intended for supervis-

ing and alerting users to wrong movements performed duringdaily physical activities can be implemented and executed inrealtime on standard low-power computing devices. Refer-ring back to the second introductory example, bend sensorswoven into trousers and a tilt sensor (or an accelerometermeasuring tilt) attached to the upper body are sufficient tomonitor whether one correctly bends the legs or whether oneincorrectly bends only the upper body to lift an object fromthe floor. Tactile motion instructions triggered immediatelyat the torso and at the upper thighs can signal the correctexecution of movements to prevent injuries.

9. SUMMARY AND CONCLUSIONSWe presented our work on wearable automatic feedback

devices for assisting people during daily physical activitiesand for supporting instructors and students during sportstraining. These systems are supposed to detect harmful andincorrect posture and movements using body-worn sensors,and to provide tactile feedback for corrections with actuatorsplaced at key positions across the body.

As an example of such systems, we have developed a firstinteractive assistant for snowboard training. Based on in-terviews with domain experts, we have designed a hardwareplatform that classifies basic context information and thatcan be used to demonstrate and to indicate appropriatetiming and correct execution of specific body movements.Context-recognition was evaluated during a study with end-users on the slope. The results show that simple featureextraction derived from sensors and setting thresholds onsensor signals is sufficient to reliably identify the riding edgeand to measure knee flexion. These classification results al-low to build a posture model of the body and to compare cur-rently performed movements to specific sports techniques.Apart from measuring reference values in neutral positionwhile at rest, no subject specific training for creating recog-nition models was required. The described algorithms runon standard low-power micro-controller devices in realtime.

According to interviewed snowboarding and skiing instruc-tors, the proposed approach of automatic performance anal-ysis with immediate tactile feedback during exercises hasthe potential to change current training methods and to en-hance the learning experience of winter sport practitioners.The system we described was tailored specifically for snow-board training but can be adapted to other sport domains aswell, such as skiing, surfing, freeboarding, riding, or dancing.

10. FUTURE WORKA wearable system for accurately tracking the relative po-

sition and orientation of body parts using magnetic fieldtechnology has been recently demonstrated [9]. This ap-proach can help to improve the recognition of incorrect up-per body rotation while descending the slope. Alternatively,stretch sensors and gyroscopes attached to the rider’s torsomight help to reliably detect this particular mistake. Sim-ilarly, analyzing incorrect weight distribution towards theback foot while descending the slope needs further investi-gation with different sensor setups and snowboard bindings.

Tactile motion instructions are a promising technique forproviding immediate feedback for corrections during sportstraining. We further plan to use our system to explore theinfluence of such realtime tactile feedback on the learningexperience of snowboarding students during real courses.

11. ACKNOWLEDGMENTSWe would like to thank SnowWorld.com for their kind

support provided during our user studies. This work wasfunded in part by the German B-IT Foundation.

12. REFERENCES[1] Chi, E. H., Song, J., and Corbin, G. “Killer App” of

Wearable Computing: Wireless Force Sensing BodyProtectors for Martial Arts. In 17th Symposium onUser Interface Software and Technology, pages277–285. ACM Press, October 2004.

[2] Dt. Verband f. d. Skilehrwesen, Interski Deutschland.Snowboard-Lehrplan. BLV Buchverlag, Munchen, 2003.

[3] Kunze, K., Barry, M., Heinz, E., Lukowicz, P., Majoe,D., and Gutknecht, J. Towards Recognizing Tai Chi -An Initial Experiment Using Wearable Sensors. In 3rdInt. Forum on Applied Wearable Computing. Springer,April 2006.

[4] Kwon, D. Y. and Gross, M. Combining Body Sensorsand Visual Sensors for Motion Training. In Int. Conf.on Advances in Computer Entertainment Technology,pages 94–101. ACM Press, New York, 2005.

[5] Lindeman, R.W., Yanagida, Y., Hosaka, K., and Abe,S. The TactaPack: A Wireless Sensor/ActuatorPackage for Physical Therapy Applications. In VirtualReality, pages 337–341. IEEE, Washington, March2006.

[6] Mattmann, C., Amft, O., Harms, H., Troster, G., andClemens, F. Recognizing Upper Body Postures usingTextile Strain Sensors. In 11th Int. Symposium onWearable Computers, pp. 1–8. IEEE, 2007.

[7] Michahelles, F., Crameri, E., and Schiele, B. Design,Implementation and Testing of a Wearable SensingSystem for Professional Downhill Skiing inCooperation with Trainers. In 2nd Int. Forum onApplied Wearable Computing. Springer, March 2005.

[8] Muller, F., Agamanolis, S., and Picard, R. ExertionInterfaces: Sports over a Distance for Social Bondingand Fun. In Int. Conference on Human Factors inComputing Systems, pages 561–568. ACM Press, NewYork, April 2003.

[9] Pirkl, G., Stockinger, K., Kunze, K., and Lukowicz, P.Adapting Magnetic Resonant Coupling Based RelativePositioning Technology for Wearable ActivitiyRecogniton. In 12th Int. Symposium on WearableComputers, pages 47–54. IEEE, Piscataway, Sept.2008.

[10] Randell, C. and Muller, H. Context Awareness byAnalysing Accelerometer Data. In 4th Int. Symposiumon Wearable Computers, pages 175–176. IEEE Press,Washington, 2004.

[11] Spelmezan, D., Jacobs, M., Hilgers, A., and Borchers,J. Tactile Motion Instructions for Physical Activities.In Int. Conference on Human Factors in ComputingSystems. ACM Press, New York, April 2009.

[12] Takahata, M., Shiraki, K., Sakane, Y., andTakebayashi, Y. Sound Feedback for Powerful KarateTraining. In Int. Conference on New Interfaces forMusical Expression, pages 13–18. National Universityof Singapore, Singapore, 2004.

[13] Wulf, G. Attention and Motor Skill Learning. HumanKinetics, 2007.