HUMAN GAIT AND POSTURE ANALYSIS FOR DIAGNOSING NEUROLOGICAL DISORDERS

H. Lee1, L. Guan1, J. A. Burne2

1 School of Electrical and Information Engineering, 2 Department of Biomedical ScienceUniversity of Sydney

hlee, ling@ee.usyd.edu.au j.burne@cchs.usyd.edu.au

ABSTRACT

This paper describes a number of new techniques to enhance the performance of a video analysis system, free from motion markers and complicated setup procedures, for the purpose of quantitatively identifying gait abnormalities in static human posture analysis. Visual features are determined from still frame images out of the entire walking sequence. The features are used as a guide to train a neural network, in an attempt to providing assistance to clinicians in diagnosing patients with neurological disorders.

Keywords: image analysis, neural networks, pattern classification, Parkinson’s disease, gait analysis.

1. INTRODUCTION

Current diagnosis of many neurological disorders, such as Parkinson’s disease (PD), involves human observation of the posture and movement of the gait. These observations tend to be subjective and depend greatly on the experience and judgement of the clinician, which can vary from person to person. Even though, there are image processing systems that attempt to automate this process, they usually require a highly constructed laboratory environment, which may not be suitable for some patients.

Disorders that affect movement and posture include stroke and head injuries. All of them have different visual symptoms such as muscle spasms, paralysis of limbs, lopsided stance, shuffling walk and a stooped gait. From these symptoms, an experienced clinician can diagnose the type and degree of severity of the disorder [1].

The aim of this work is to improve a video image analysis system developed at the University of Sydney [2] to intelligently and realistically incorporate visual assessment techniques used by the neurologists. In

particular, we propose three techniques to enhance the performance of the system. First, in data acquisition, a tracksuit is introduced to replace the uncomfortable tight outfit used before, in an attempt to provide a more realistic diagnostic environment. Second, this work investigates colour image processing techniques. Since humans see the world in the colour spectrum, processing colour images should provide additional information not available in the grayscale domain. Finally, a general regression neural network (GRNN) [3] is applied to implement the well known sequential feature evaluation methods [4] for feature selection. The most important features are selected and used in decision making. The system yielded a correct diagnosing rate of up to 85% using the test data. The result suggests that the techniques have the potential to assist the neurologists in providing a second opinion in diagnosing neurological disorder.

2. SYSTEM OVERIVEW

The video image analysis system consists of laboratory setup, a video/image capturing device and a software processing tool.

2.1. Laboratory setup

The human subjects are required to wear a specifically designed outfit, that has different colours for different parts of the body. Originally, a tight bodysuit was used in data collection [2] as shown in Figure 1(a). This approach is not practical in real life, since even a healthy person would feel uncomfortable wearing such a tight outfit, let alone patients with a neurological disorder. To alleviate this problem, a tracksuit is designed to replace the bodysuit ( Figure 1(b)). This has the advantage of relaxing the subjects during the experiment, and, therefore, more realistic gait patterns can be observed and analysed. In addition, the practical laboratory setting would be more beneficial in the context of medical research.

1(a) 1(b)Figure 1: Examples of the costume used in the experiment: tight bodysuit (a) and tracksuit (b)

2.2. Image Acquisition

The overall image acquisition subsystem consists of the following hardware components shown in Figure 2: a video camera, a video recorder, and a frame grabber. This subsystem also includes the necessary software for the interface between the PC and the hardware, as well this software can perform some preliminary image processing tasks such as noise filtering, and image enhancement.

Figure 2 Hardware settings

2.3. Image Analysis

The image analysis subsystem is software based. The software performs the required processing to extract important visual information given in Figure 3, such as joint angles and swing distances of the limbs from the image obtained in the previous stage. The contribution of this paper to the image processing subsystem is detailed in the following section.

3. IMAGE ANALYSIS

This subsystem performs all the major image processing tasks necessary for analysing neurological disorder, including color image segmentation, medial axial analysis, feature extraction and selection, and decision making. Among these tasks, the new contribution lies in segmentation and feature selection.

3.1 Colour Segmentation

A neural network based segmentation technique is used to segment out different parts of the body in the image. A back propagation algorithm is used to train the system to recognise the colours used. It uses the RGB values of any pixel in the image as the input to train the neural network to recognise different colours appearing in the image. Particularly, the different colours of the tracksuit correspond to different parts of the body. Once the network is trained, segmentation can be obtained much faster for massive data processing. This is in contrast to the previous work [2] where grayscale based segmentation algorithms were used, and substantial design effort had to be committed in order to obtain reasonable segmentation. Further more, this technique is applicable for other colour combinations, which might not provide significant contrast in the grayscale domain.

Figure 3 Locations of the visual features

3.2 Feature Extraction

PD symptoms such as tremor, rigidity, stiffness of the limbs and lack of co-ordination, must show some significant differences in the features of the gait, such as joint angles, swing distances, and swing trajectories of the limbs, velocity of the movement, etc. From the observations of the gait in normal people and that in the PD patients, it is evident that the stretchability and the

joint flexibility of the limbs are the key features to distinguish the gait patterns between the two groups. The stretchability and the joint flexibility best describe the rigidity and stiffness of the limbs. Apart from featuring the rigidity and the stiffness, the swing directions and swing distances of two arms relative to the median axis of the torso indicates the inability to execute simultaneous movements. In this research, processing is restricted to static images rather than dynamic image sequences. Based on the above reasons, the joint angles and the swing distances are used as the inputs to decision making for classification.

To extract visual features, the segmented limbs must first be reduced to its skeleton representation. To obtain the skeleton of the object, thinning techniques are required. Thinning can be defined heuristically as a set of successive erosions of the outermost layer of the shape, until a connected unit-width set of lines (skeleton) is obtained [5]. After thinning, ten features were extracted from the skeleton, namely:

F1 Front knee joint angleF2 Back knee joint angleF3 Front ankle angleF4 Back ankle angleF5 Front elbow angleF6 Back elbow angleF7 Stepping distanceF8 Front hand to median axis of torso distanceF9 Back hand to median axis of torso distanceF10 Front hand to back hand distance

These features are graphically illustrated in Figure 3.

3.3 Feature Selection

The importance of selecting the relevant subset from the original features is closely related to the “curse of dimensionality” problem in function approximation, where sample data points become increasingly sparse when the dimensionality of the function domain increases, such that the finite set of samples may not be adequate for characterising the original mapping. In this work, a general regression neural network (GRNN) [3] is applied to implement the well know sequential forward selection (SFS) and sequential backward selection (SBS) methods [4] to select the most effective features. The GRNN consists of four layers (Figure 4). The input layer simply passes the input vector variables X (the features, in this case) into the hidden layer. The hidden layer consists of all the training samples, X1, …, Xi. When an unknown pattern X is presented, the squared distance Di

2 between the unknown pattern and the training sample is calculated

and passed through the kernel function. The summation layer has two units A and B, the unit A computes the summation of exp[-Di

2/(22)] multiplied by Yi the classification result, associated with Xi. The B unit computes the summation of exp[-Di

2/(22)]. The output unit divides A by B to provide the prediction result[3]. The advantage of using such a network over histogram analysis is that GRNN needs only a single pass of learning to achieve optimal performance in classification. Also it provides a non-linear approach for feature selection.

Figure 4: GRNN architecture

The selected features, according to their discriminatory power by SFS and SBS, are summarized in Table 1.

Sequential Forward SelectionFront hand to median axis of torso distance

Front knee joint angleFront hand to back hand distance

Front elbow angleBack elbow angleStepping distanceBack ankle angle

Sequential Backward SelectionFront ankle angle

Back hand to median axis of torso distanceFront hand to back hand distance

Stepping distanceBack knee joint angle

Front elbow angleBack ankle angle

Front knee joint angleTable 1: Features selected by both approaches

3.4 Decision Making

The decision making part consists of a three-layered back propagation neural network. It was trained by the features selected from the previous stage, and the performance of the system was tested using new images. Separate neural networks were constructed for training SFS features and SBS features. In the case of SFS features there were a total of 14 neurons; 7 input neurons corresponding to 7 SFS features used, 5 hidden neurons and 2 output neurons corresponding to healthy (1,0) and patient (0,1) groups. The second network consists of 8 input neurons, 5 hidden neurons and 2 output neurons. Further more, another network was constructed to train with all 10 features, as to compare with the classification results obtained by the feature extraction strategies described earlier. Six hidden neurons and 2 output neurons are implemented in the third

network.

a) Sequential forward selection

b) Sequential backward selection Figure 5: Feature selection using GRNN

4. EXPERIMENTAL RESULTS

We implemented the proposed enhancement strategies into our video analysis system and tested the system on a database of both PD patients and healthy people. The patients were video taped at Sydney Westmead Hospital, and the data on healthy people was collected at the University of Sydney. In total, 80 images (30 from the patients group and 50 from the normal group) were extracted from the videos. From the 80 images collected both from normal and patient data, 40 of them were used to train the neural networks, and the other 40 were used to test the system performance. As Table 2 shows, 85% correct classification was achieved by using the features

selected by the SBS, and 82.5% by the SFS features. The performance of the neural network using all ten features in training and testing are also tabulated in Table 2. It provided a 77.5% correct diagnose only, due to curse of dimensionality.

Correct Normal

False Positive

Correct Patient

FalseNegative

Forward Selection

12 3 21 4

Backward Selection

13 2 21 4

All features

12 3 19 6

Table 2: Classification results from static analysis

5. CONCLUSIONS

In this paper, we propose several techniques to improve the performance of a video image analysis system for the diagnosis of neurological disorder. In data acquisition, tracksuit is used to replace the uncomfortable tight outfit in order to provide a more realistic diagnostic environment. We investigate colour image segmentation techniques. Since humanbeings see the world in the colour spectrum, processing colour images provided additional information not available in the greyscale domain. We also applies a general regression neural network in selecting the most effective features for decision making. The system yields a correct diagnosing rate of up to 85%. The result suggests that the techniques have the potential to assist neurologists in providing a second opinion on diagnosing neurological disorder.

REFERENCES

[1] F.H. McDowell, “The Diagnosis of Parkinsonism or Parkinson Syndrome,” Contemporary Neurology, 1990.

[2] R. Chang, L. Guan, J. Burne, “An automated form of video image analysis applied to classification of movement disorders,” J. of Disability and

Rehabilitation, vol. 22, no. 1/2, pp. 97-108, January/February 2000.

[3] D.F. Specht, “A general regression neural network,” IEEE Trans. Neural Networks, vol. 2, no. 6, pp. 568-576, Nov. 1991.

[4] J. Kittler, “Feature set search algorithm,’’ in Pattern Recognition and Signal Processing (C.H. Chen, ed.), Sjithoff & Noordhoff, 1978.

[5] R.C. Gonzalez, R.E. Woods, Digital Image Procesing, Addison-Wesley Publication Company, 1993.