literature review draft 4
TRANSCRIPT
-
7/31/2019 Literature Review Draft 4
1/24
Page 1 of24
Hwa Chong Institution
NTU-A*STAR-HCI H3 SCIENCE RESEARCH PROGRAMME
2012
Research Proposal
Title of project: Ambient and Visual Sensor Based Falling Prevention and Detection
System
Name of student: Lee Choon Kiat
CT group: 12S68
Acknowledged by (pls sign)
___________________________ ______________________________
A*STAR mentor HCI teacher-mentor
(Name: ) (Name: )
Date of submission: ________________
-
7/31/2019 Literature Review Draft 4
2/24
Page 2 of24
Literature Review + Proposal
1.1 Introduction
The progressive aging of the population, particularly the increase in the number of people living to a
very advanced age, has become a major issue in public health due to the societal challenges that this
demographic change creates. Due to most of the elderly living from health problems and living alone,
they require both support with daily activities and psychological support to feel safe because they are
alone. For the elderly, involuntary falls are frequent, and are likely to cause a loss in quality of life for the
victim. Furthermore, a fall in the elderlys house can be extra dangerous due to the fact that the victim
can easily lose consciousness and thus become unable to ask for help, which could be crucial to their
health if the accident is serious. Thus, in order to avoid this scenario, fall detection systems that are
capable of identifying falls and notifying caregivers when the elderly fall are very much required.
Firstly, a literature review has been conducted to gain an understanding into the background of the
project. With the background knowledge gathered during the literature review, aspects of the problem
that will be addressed in the project were then identified and a research proposal (section 3) was
drafted.
The literature review will be conducted in two portions. The first part of the literature review will be
conducted on the biomechanics behind the falling motion, as well as the reasons why the elderly fall.
The second part of the literature review will focus on existing solutions that possess fall detection
capabilities.
-
7/31/2019 Literature Review Draft 4
3/24
Page 3 of24
2.0 The Falling Motion
2.1.1 Fall scenarios what happens during a fall
(Yu,2008) describes 3 of the most important fall scenarios that detail what happens during a fall.
Fall from standing
o Fall from standing is defined here as a transition from the standing state to lying on the
floor
o Process lasts 1-2 seconds, consisting of several sub-actions
1. A person will usually fall in one direction. Thus, this will cause both the head and
the center of mass of the person to move in one plane during falling
2. The head reduces in height from standing height to lying height.
Head will travel in free-fall
3. Head will fall within a circle centered at the foot position of the last standingtime and with a radius equal to the height of the person.
4. Head lies on the floor at the end of the falling process. Head will lie on the floor
with little / no motion for a period of time
Fall from sitting (on a chair)
o Process lasts 1-3 seconds
1. Person is sitting in chair at beginning of fall
2. Head reduces in height from sitting height to lying height.
Head will travel in free-fall
3. Lying body will be near to chair
Fall from sleeping (on bed)o Process lasts 1-3 seconds
1. Person is lying on bed at beginning of fall
2. Body reduces in height from bed height to lying height
Body will travel in free fall
3. Lying body will be on floor near to the bed.
From the above descriptions of a fall, coupled with other definitions of falls found in current literature, I
propose that a general description of a fall can be described as follows:
A fall is defined as an unintentional transition in position of an individual, resulting in him
coming to rest at a lower level
The head will reduce in height from the original height to lying height
o During this period, the head will travel in free-fall
The head will lie in the vicinity of the place of occurrence of the fall
-
7/31/2019 Literature Review Draft 4
4/24
Page 4 of24
(Berg et al., 1997) conducted a survey on 96 male and female volunteers between the ages of 60 and 88
that aimed to determine the circumstance and consequences of falls in elderly patients. Causes of falls
were determined from the participants descriptions of their falls. The following graph details their
results.
Figure 1: Causes of falls in elderly (Berg et al. 1997)
From this graph, we can conclude that most of the falls (trip, slip, misplace step, legs giving way,
knocked over, loss of support surface) were falls from standing position, making this kind of fall the mostimportant kind of fall to detect.
However, I theorize that falls from the bed and falls from the chair may be particularly hard to detect by
ambient sensors as they happen over a long time frame and involve a relatively much smaller change in
vertical displacement of the head, thus causing the acceleration of the head to be much smaller than in
a fall from standing position. As ambient sensors typically work by applying algorithms to detect changes
in the vertical acceleration of the head to detect falls, falls from the bed and falls from the chair may be
a source of false negatives for ambient sensors that we could adapt the Kinect to detect. This would
allow a multi-modal system containing both the Kinect and ambient sensors to achieve a much higher
fall detection rate.
-
7/31/2019 Literature Review Draft 4
5/24
Page 5 of24
2.1.2 Causes of falls why did a fall occur?
Some research has been conducted into identifying why falls occur, with most employing the same
methodology as (Berg et al.,1997). (Zecevic et al., 2006) conducted an investigation with the aim of
finding the differences between the elderlys perceptions of the causes of their falls and that of the
research community. His results are presented below:
Table 1: The most frequent reasons for falling suggested by seniors
and health care providers compared to risk factors reported in research literature
As can be seen from table 1, certain factors seem to be key indicators of a fall. For example, balance is
listed as an important reason for falling by all three parties, suggesting that many falls are due to a loss
of balance.
By comparing the two sets of causes of falling provided by (Berg et al., 1997) and (Zecevic et al. 2008),
we will be able to pinpoint the more important causes of falling, as well as identify causes of falling that
we may be able to predict using our fall detection system in order to attempt to implement fall
prevention technologies.
Cause of fall identified by Berg
et al. 1997
Matching cause identified by
Zecevic et al. 2008
Ranked importance
Trip Slip, trip, stumble, vision Slip, trip, stumble
Slip Slip, trip, stumble Loss of balance
Misplace step Inattention Inattention
Loss of balance Balance Vision
Legs giving way Muscle weakness Muscle weakness
Knocked over NIL
Loss of support surface Inattention
-
7/31/2019 Literature Review Draft 4
6/24
Page 6 of24
Of the five main causes of falls that have been collated, I feel that loss of balance and inattention can be
causes of falls that can be prevented using the Kinect, as it may be possible to derive algorithms that can
predict balance given the center of mass and feet position, which are both obtainable from the Kinect.
-
7/31/2019 Literature Review Draft 4
7/24
Page 7 of24
2.2 Fall detection
Fall detection is a relatively developed field, with numerous solutions provided by researchers all around
the globe. However, many problems still exist, with the problem of false readings proving to be the
largest barrier to their widespread adoption. However, other problems such as the lack of a pervasive
solution that is non-invasive exist, and are major limitations that prevent fall detection systems from
emerging from the lab. Most fall detection systems can be classified into three types that are
differentiated by the kind of sensors that are employed.
1. Body-worn sensors
These fall detection sensors utilize body-worn sensors such as accelerometers and
gyroscopes to determine the motion of the human body. Abnormally fast motions are
usually taken to represent falls
2. Ambient sensors
A relatively unemployed solution due to the difficulty in gathering data that is easily
employed in fall detection. This solution utilizes ambient sensors such as ultrasound andinfrared to gather data to determine falls.
3. Cameras
A field that has not been explored too extensively due to the difficulty of employing the
complicated image processing algorithms that require large amount of computing
power. However, this solution has been regularly employed due to the large amounts of
data that it can easily capture.
2.2.1 Body-worn sensors
Many implementations of fall detection systems have been created that utilize body-worn sensors as
their main mode of data collection. These systems generally utilize at least one or more accelerometers
to measure the acceleration of different parts of the human body.
If a large acceleration exceeding a preset acceleration threshold value is obtained, then a fall is indicated.
This method has the advantage of only needing to process relatively small amounts of data, and so the
data processing can sometimes be performed by the microprocessor connected to the sensor itself. This
eliminates the need for bulky wireless transfer modules that can make the device overly bulky and thus
unappealing to potential users.
-
7/31/2019 Literature Review Draft 4
8/24
Page 8 of24
Figure 2: Acceleration observed when falling backwards
More advanced systems such as that by (Bourke & Lyons, 2006) combine gyroscopes together with the
accelerometer data. As the gyroscopes can obtain rotational data in 3 or even 6 degrees of freedom, by
combining it with the acceleration data obtained by the accelerometer, the specificity of the solutionthat is created can be improved significantly as posture detection and other data can be calculated more
accurately. However, this addition of data means that the fall detection algorithm that has to be
developed becomes much more complicated in order to fully capitalize on the extra capability provided
by the extra data. Thus, many such solutions utilize wireless data transfer technologies such as
Bluetooth, Zigbee to send the data back to a computer, where the data is then analysed. While this
allows much more complicated and computationally intensive algorithms to be employed to improve
the specificity of the system, it increases the bulkiness and complexity of the system
Figure 3: Fall detection using 2 axis gyroscope and accelerometer
Indicates a fall
-
7/31/2019 Literature Review Draft 4
9/24
Page 9 of24
2.2.2 Ambient Sensors
Various ambient sensors have been employed for fall detection, from microphone arrays, pyroelectric IR
sensors (Sixsmith et al., 2005) and ultrasound (Hou et al., 2007) These solutions have the advantage of
being extremely cheap, yet with the ability to be very pervasive while avoiding being invasive. This is due
to the sensors being relatively inexpensive and able to be mounted such that they will not invade the
users personal space. This makes these solutions very attractive for those looking to develop user-
friendly systems. However, these systems generally suffer from much reduced sensitivity and specificity
in fall detection due to the difficulty in gathering useful and accurate data from the limited number of
sensors and the constantly changing background. For example, the pyroelectric IR grid employed by
(Sixsmith et al., 2005) only achieved a 30% fall detection rate, a far cry from the 99% achieved by other
fall detection systems. Thus, I feel that these systems will be able to overcome this weakness by
employing multi-modal solutions that utilize different kinds of sensors working in tandem to provide
adequate useful data to allow for accurate fall detection. This has been employed by (Hou et al., 2007)
and shows promise for further research.
Figure 4: IR and visible images using a 16x16 array based system by (Sixsmith et al. 2005)
2.2.3 Cameras
Video based detection has a rich history in the field of fall detection due to its powerful data acquisition
potential. Furthermore, as it is non-invasive, it also possesses the advantage of ambient sensors while
combining it with the data acquisition prowess of the body worn sensor solutions. However, the
availability of data is a double-edged sword, as it becomes difficult to isolate the user from the
background data. This results in many complicated algorithms having to be applied to the camera datato generate useful conclusions that can be used for fall detection. Thus, many applications do not run in
real time due to the large amount of processing power needed, making the applications not very
feasible in the real world. The sheer amount of data also results in the accuracy of the fall detection
algorithms being compromised if the data is not processed accurately due to extraordinary ambient
conditions (eg: constantly changing background, multiple users).
-
7/31/2019 Literature Review Draft 4
10/24
Page 10 of24
For processing of data, many solutions utilize Matlab and (suspected) OpenCV. Matlab is used due to the
easy implementation of the complicated and highly mathematical image processing algorithms while
OpenCV is used due to its powerful inbuilt image processing functions.
Once the background is removed and the user is isolated, there are several general methods that are
used to determine if a person has fallen. The following paragraphs detail some of the methods used:
Aspect Ratio
The aspect ratio of a person is probably the simplest and easiest to implement feature that can help in
fall detection. However, it is quite effective due to the big difference in posture when a person has fallen
or not fallen. In this method, the aspect ratio of the user is first determined from the data. A sudden
increase in the persons width to height ratio is then used to determine if the person has fallen. The
main problem with this method is that it does not work when the person falls and ends up in a lying
position where he is occluded, resulting in unpredictable fall detection when numerous occlusions are
present.
Figure 5: Comparison of aspect ratio between fallen and standing positions
Standing width to
height ratio is small
Fallen width to height
ratio is large
-
7/31/2019 Literature Review Draft 4
11/24
Page 11 of24
3D head tracking
Advanced systems are able to perform 3D head tracking, which tracks the motion of the users head in
the video in 3D using multiple cameras. The 3D model obtained is then used for fall detection. For
example, in (Rougier & Meunier, 2007), the 3D head model obtained is used to determine the vertical
and horizontal velocity of the persons head. A fall is then indicated if certain thresholds for these twodata are breached. This system should be quite accurate based on the descriptions of falls obtained
earlier from (Yu, 2008) as it can directly obtain the vertical displacement of the head, which is a key
descriptor in the events of a fall.
Figure 6: Head Tracking by (Rougier & Meunier, 2007)
-
7/31/2019 Literature Review Draft 4
12/24
Page 12 of24
Fall angle
The fall angle is another parameter used for determining falls. The fall angle () is the angle that the
centre of mass of the persons body makes with respect to the ground. When there is a sharp decrease
in fall angle from close to 90o
to close to 0o, a fall is indicated. I feel that this method could be quite
accurate and suitable for determining falls due to its widespread use in body-sensor solutions thatpossess >90% fall detection rates. However, it may be difficult to implement this solution without
efficient and proven algorithms due to the difficulty in determining the fall angle.
Figure 7: Fall angle of object in different poses (Vishwakarma et al.)
-
7/31/2019 Literature Review Draft 4
13/24
Page 13 of24
3.0 Proposal
3.1 Overview
In this proposal, I will outline a research plan for a fall detection system based on the Microsoft Kinectthat can detect falls automatically.
3.2 Sensor employed
The main sensor that I propose to employ is the Microsoft Kinect, coupled with other ambient sensors
that are linked together to form a multi-modal system. The Kinect is a motion sensing input device by
Microsoft for the Xbox 360 video game console and Windows PCs. The Kinect sensor is a horizontal bar
connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or
below the video display. The device features an RGB camera, depth sensor and multi-array microphone,
which provides full-body 3D motion capture, facial recognition and voice recognition capabilities.
RGB CameraDepth Sensor
Multi-array
Microphone Motorized Tilt
-
7/31/2019 Literature Review Draft 4
14/24
Page 14 of24
3.2.1 Capability of the sensor what can the sensor detect?
RGB camera
o Horizontal field of view: 57 degrees
o Vertical field of view: 43 degrees
Depth Sensor
o Quoted by Microsoft 1.2m to 3.5m
o However, preliminary tests seem to show that its minimum range is about one
handspan (about 50-60cm), while some research papers seem to suggest that the depth
sensor can detect objects as far as 10-12m away.
o The resolution of the depth sensor is not constant and varies according to the distance
between the object and the Kinect. The greater the distance between the object and the
Kinect, the coarser the resolution. The resolution of the Kinect at 2m has been found to
be approximately 40mm (Gill et al., 2012)
o Inner workings of the depth sensor
The depth sensor consists of a IR projector and an IR detector. The IR projector
projects a speckled pattern of IR dots, and the IR detector detects the reflected
pattern. This allows the IR detector to determine the depth from the sensor to
the object.
o Issues with the depth sensor
Unreliable depth values
Depth values can be unreliable due to a few reasons
o Low resolution at far distances
With a resolution of 0.1cm at 9-10m, the Kinect may not
have the required sensitivity at long distances
o The IR sensor introduces noise to the edges of objects
The variation in depth values can vary by as much as
130mm at 2m, and increases as the distance between
the object and the sensor increases
Shadows
As the IR sensor requires the pattern projected by the IR projector to
determine the depth map, it is not possible for the IR sensor to estimate
distances outside the line of sight of the sensor. This results in the
shadow seen in the figure below. Due to the relative positions of the
laser and IR sensor, the shadows can only be observed to the left of the
object.
-
7/31/2019 Literature Review Draft 4
15/24
Page 15 of24
Figure 9: Sketch of why shadows occur in depth image
3.2.2 Capability of the Microsoft Kinect SDK API what can the API do?
Microsoft, the manufacturer of the Kinect, has developed the Microsoft Kinect SDK, which provides the
tools and APIs, both native and managed, that are needed to develop Kinect-enabled applications for
Microsoft Windows by providing support for the features of the Kinect, including color images, depth
images, audio input, and skeletal data. Some of the key functions that the SDK provides are as follows:
Recognize and track moving people using skeletal tracking.
Determine the distance between an object and the sensor camera using depth data.
Capture audio with noise and echo cancellation or find the location of the audio source.
Enable voice activated applications by programing a grammar for use with a speech recognition
engine.
In this project, the main functionalities of the SDK that will be employed are its ability to capture depth
and colour images from the Kinect sensor as well as the skeleton tracking functions of the SDK.
One of the key advantages of the Microsoft Kinect SDK has over the open-source OpenNI API, the otheralternative API for the Kinect, is that the SDK can perform skeleton tracking without requiring a
calibration pose, allowing more advanced and innovative fall-detection algorithms that require the use
of skeleton tracking to be implemented. As this feature is extremely new (it was first available in the
May 2012 release of the SDK), this will allow our project to pioneer fall-detection algorithms if the time
allows.
-
7/31/2019 Literature Review Draft 4
16/24
Page 16 of24
3.3 The Proposal What will we do?
In most camera based fall detection systems, it has been established that the ability of the data to
provide the 3D real world coordinates of the user is extremely important to allow for accurate fall
detection (Rougier et. al, 2011). Currently, 3D real world coordinates are obtained by using either a
multiple camera array system, which requires precise calibration or a Time-of-Flight camera, which can
be extremely expensive to purchase. Thus, I feel that the Kinect sensor can be employed instead due to
its low cost and ease of installation. This will enable the solution to be more user friendly. Furthermore,
the Kinect can obtain a depth image, which is invaluable in precisely tracking the motion of the user
around the room, as well as allowing the privacy of the person to be readily preserved. Furthermore,
due to the use of an IR sensor to obtain the depth image, this solution will be able to work both day and
night in zero-light conditions, which greatly increases the pervasiveness of the solution. In our project, I
propose that a fall detection system based on the Kinect sensor should be developed.
3.3.1 Hardware Gathering of data from the Kinect sensor and its placement
As explained earlier, in order to perform accurate fall detection, it is often necessary to obtain the 3D
real world coordinates of the user from the data gathered from the Kinect sensor. Thus, in order to
obtain this data accurately, I feel that it would be best to employ the depth sensor from the Kinect. By
utilizing the voxel map obtained from the depth sensor, together with ground plane detection
techniques, it should be possible to obtain the 3D real world coordinates of the user. The use of the
depth sensor also has the important advantage of being able to preserve the users privacy in a way that
is difficult to replicate using a RGB camera system, helping to alleviate some of the users potential
concerns.
The Kinect sensor comes with a unique set of limitations not present in other camera based fall
detection system, with the first and foremost being that it cannot detect the depth of objects that are
less than 80-100cm in front of the Kinect sensor. By taking into account this limitation, as well as other
key limitations of the Kinect sensor, I have come up with three rough guidelines that can be applied to
the placement of the Kinect sensor in the room
1. The Kinect sensor should be placed such that it will not be able to obtain the depth image for
the whole room. This is most easily accomplished by placing it in one of the four top corners of
the room, which would provide the Kinect sensor with the biggest field of view possible.
2. The Kinect sensor also has to be placed in such a way that it will be physically
impossible/unlikely for a fall to occur in the blind spot of the sensor that is due to the minimum
range of the Kinects depth sensor.
3. Lastly, due to the shadow effect, it may be wise to place the Kinect sensor to the left of the
room to minimize blind spots caused by the shadow effect. This should help to reduce the
effect of the shadow effect.
-
7/31/2019 Literature Review Draft 4
17/24
Page 17 of24
3.3.2 Software
The C++ programming language
C++ is a statically typed, free-form, multi-paradigm, compiled, general-purpose programming language.
It is regarded as an intermediate-level language, as it comprises a combination of both high-level andlow-level language features. Developed by Bjarne Stroustrup starting in 1979 at Bell Labs, it adds object
oriented features, such as classes, and other enhancements to the C programming language.
C++ is one of the most popular programming languages and is implemented on a wide variety of
hardware and operating system platforms mainly due to its high speed and efficiency, as well as the
wide support for C++ available.
Furthermore, as the Microsoft Kinect SDK API that was chosen to interface with the Kinect sensor
requires the use of either C++, C# or visual basic, the choice was made to use C++ as the programming
language of choice for the fall-detection program.
Microsoft Kinect SDK API
Microsoft, the manufacturer of the Kinect, has developed the Microsoft Kinect SDK, which provides the
tools and APIs, both native and managed, that are needed to develop Kinect-enabled applications for
Microsoft Windows by providing support for the features of the Kinect, including color images, depth
images, audio input, and skeletal data.
-
7/31/2019 Literature Review Draft 4
18/24
-
7/31/2019 Literature Review Draft 4
19/24
Page 19 of24
Activity Fall / Non-Fall
1 Forward collapse (fall on knees) Fall
2 Forward collapse (end lying down) Fall
3 Backward collapse (fall on knees) Fall
4 Backward collapse (end in sitting position) Fall
5 Backward collapse (end lying down) Fall
6 Sideways collapse (right) Fall
7 Sideways collapse (left) Fall
8 Fall from bed Fall
9 Sitting down on a chair Non-Fall
10 Standing up from a chair Non-Fall
11 Lying down on bed Non-Fall
12 Getting up from bed Non-Fall
13 Jumping Non-Fall
14 Pick up something from the floor Non-Fall
15 Bend forward and tie shoe laces Non-Fall
Table3: Fall and non fall scenarios developed by (Tolkiehn et al., 2011) Table 4: Fall and non fall scenarios adapted from (Tolkiehn et al., 2011)
-
7/31/2019 Literature Review Draft 4
20/24
Page 20 of24
An analysis of these above sets of fall scenarios that is similar to many others utilized by other research
projects allows us to generalize the fall scenarios and come up with our own more comprehensive set of
fall scenarios by integrating this table with the previous definitions of a fall provided by (Yu,2008).
(Yu, 2008) provided his definitions of a fall in terms of the starting and ending positions of the head,
while those provided by (Tolkiehn et al., 2011) employs roughly the same methodology. However,(Tolkiehn et al., 2011) further classifies falls by the direction of the fall (sideways, forward collapse,
backward collapse) and gives extra indications as to the ending position (ending lying down, sitting
down). Other fall scenarios developed by other research projects specifically to challenge the
specificity of their system have included falls that end up with the person occluded as well as falls that
occur in the presence of multiple users. Thus, I feel that for our fall scenarios to be comprehensive
enough to cover more types of falls to improve the specificity of our system, we should try to classify the
falls through different parameters and obtain the various fall scenarios by mixing and matching the
different parameters. The implementation of this system is shown below:
A fall is defined by its:
Starting position
o A fall can have either one of these three starting positions
Fall from standing
Fall from sitting
Fall from lying (on bed)
Direction of fall
o A fall is characterized in two dimensions
Y-direction
Forwards
Backwards
X-direction
Rightwards
Leftwards
Variations in the X and Y magnitudes allows the direction of the fall to be
accurately pinpointed
Ending position
o There are two main ending positions
Sitting down
Lying down Unconscious or face flat on floor
Hands supporting head off the floor
Time of fall
o Based on the definitions of a fall provided by Yu, 2008, the time of fall is characterized
into two main portions
Fast fall
-
7/31/2019 Literature Review Draft 4
21/24
Page 21 of24
1-1.5s
Slow fall
1.5-3s
Occluded?
o The person may be occluded during different periods of the fall
Start of the fall
During the fall
At the end of the fall
Next, in order to obtain the fall scenarios, it is only necessary to mix and match the different fall
classifications to obtain the different fall scenarios. For example, one fall scenario that could be obtained
is as follows: A fall that occurs when the person is in the standing position, falling backwards, and lands
on the floor sitting down after a short period of time (about 1-1.5s), and is not occluded during any
period of the fall.
This particular fall characterization allows us to cover different kinds of fall comprehensively and thus beable to identify gaps in current research that can be addressed by us. For example, many fall detection
solutions have a lot of trouble with occlusions that block the cameras view of the fall, resulting in
inaccurate fall detection. However, as mentioned earlier, during preliminary testing and
experimentation the fall scenarios shown in Table 4 will be used instead to allow us to focus on the few
fall scenarios that will occur most frequently and thus enable us to improve on the specificity and
sensitivity of our system with the greatest efficiency.
Fall detection algorithm
After identifying the different fall scenarios, different fall detection algorithms that had been employed
by other research projects were then evaluated, and then solutions with potential for use in the project
were selected and modified for use in this project. The algorithms will be developed sequentially and in
a modular fashion so that the sensitivity of the system can be tweaked by the end user in the Graphical
User Interface (GUI) described in the next section.
First filter: aspect ratio change
o The change in aspect ratio of the person from start to end positions of the fall is quite
stark, especially when the person falls from a standing position and ends up in a lying
position. While this solution is a little crude, it provides a relatively accurate way of
determining if a fall has occurred with little computational overhead, and thus I feel that
it can be employed in our fall detection solution to provide a sort of first pass filter. Thiswill allow real time computational overheads to be reduced. In order to improve this
algorithm and provide more accurate fall detection, the aspect ratio of the person could
be calculated in 3 dimensions instead of the current 2 dimensions, providing a more
accurate estimate of a fall. However, this method runs into problems when the user is
occluded at the end of the fall, resulting in other filters having to be applied to
guarantee accurate fall detection.
-
7/31/2019 Literature Review Draft 4
22/24
Page 22 of24
Second filter: Vertical velocity of the centre of mass of the user with respect to the ground
o The centre of mass of the user could be identified by modeling each pixel that makes up
the user as a particle of mass 1 unit. This would then allow us to obtain the centre of
mass of the user in real world 3 dimensional coordinates. By obtaining the slope of the
ground plane, we could then calculate the vertical velocity / acceleration of the centre
of mass of the user with respect to the ground, which could be sufficient for us to
conclude that a fall has fallen.
Last filter: Fall angle / angular velocity of the user
o The fall angle of the person could also be calculated. By differentiating the fall angle of
the person against the time taken, we could obtain the angular velocity of the user,
which when coupled with the second filter should enable us to accurately determine if
the user has fallen even if he is occluded at the end of the fall. However, the mainproblem with this solution is that this filter will be quite difficult to implement due to
the difficulty of the coding. This last filter is designated as future work, and will be
implemented only if there is extra time at the end of the project as the previous two
filters should be able to detect the majority of the falls. If the previous two filters are
unable to perform as well as expected, this last filter could also then be implemented to
improve the accuracy of the system even further.
In order for the fall detection algorithms to work, thresholds which will indicate falls will have to be
developed. For example, when the vertical velocity of the user is greater than the threshold value for
vertical velocity, a fall alarm will be triggered. In order to determine the proper threshold values for
each of the fall detection algorithms, experiments will be conducted in which the Kinect sensor is placed
in the target location, and a variety of falls that correspond to the fall scenarios provided in Table 4 will
be recorded. The resulting data will then be analysed to obtain an optimal threshold value.
For each filter implemented, a few threshold values will be collected that correspond to different
scenarios and sensitivity levels. This will allow the end user of the system to tweak the sensitivity
settings of the actual program based on the actual conditions, allowing the system to be more robust
and customizable.
Graphical User Interface (GUI)
In order for the final product to be appealing to the end user, a graphical user interface (GUI) will be
built to give a user-friendly experience. The GUI will mainly provide the end-user with a user-friendly
interface with which to operate the fall detection program, allowing the program to be more appealing
to the end user. The GUI will include:
-
7/31/2019 Literature Review Draft 4
23/24
Page 23 of24
A screen which will display the skeletons of the users as well as blurred out depth/color images
from the Kinects depth sensor and camera, which will help to preserve the privacy of the users
while allowing the end-user to evaluate and check the accuracy of the fall-detection program
Customisable sensitivity settings, which will allow the user to adjust the sensitivity of the fall-
detection program, as well as the sensitivity of the fall-detection program to certain specific falls
which might occur much more where the sensor is deployed.
An alarm that will alert the end user when a fall is detected by the fall-detection program
Optional settings that allow the end user to indicate if they want to receive sms or email
updates when falls occur in case they are not able to hear the alarm.
3.4 References
Y. Xinguo, (2008). Approaches and principles of fall detection for elderly and patient,. Proc. 10th IEEE
Int. Conf. e-Health Netw., Appl. Serv., 4247.
Berg, W. P., Alessio, H. M., Mills, E. M., & Tong, C. (1997). Circumstances and consequences of falls in
independent community dwelling older adults.Age and Ageing, 26, 261-268.
Zecevic, A., Salmoni, A. W., Speechley, M., & Vandervoort, A. A. (2006). Defining a Fall and Reasons for
Falling: Comparisons Among the Views of Seniors, Health Care Providers, and the Research Literature.
The Gerontologist, 46(3), 267-276.
Bourke, A., & Lyons, G. (2008). A threshold-based fall-detection algorithm using a bi-axial gyroscope
sensor. Medical Engineering & Physics, 20, 84-90.
A. Sixsmith, N. Johnson and R. Whatmore. (2005) Pyroelectric IR sensor arrays for fall detection in the
older population,J. Phys. IV France 128, 153-160
Jennifer C. Hou, Qixin Wang, Bedoor K. AlShebli, Linda Ball, Stanley Birge, Marco Caccamo, Chin-Fei
Cheah, Eric Gilbert, Carl A. Gunter, Elsa Gunter, Chang-Gun Lee, Karrie Karahalios, Min-Young Nam,
Narasimhan Nitya, Chaudhri Rohit, Lui Sha, Wook Shin, Sammy Yu, Yang Yu, Zheng Zeng, (2007) "PAS: A
Wireless-Enabled, Sensor-Integrated Personal Assistance System for Independent and Assisted Living,"
Joint Workshop on High Confidence Medical Devices, Software, and Systems and Medical Device Plug-
and-Play Interoperability, 64-75
C. Rougier, J. Meunier, A. St-Arnaud and J. Rousseau, (2007). Fall Detection from Human Shape and
Motion History using Video Surveillance, 21st IEEE International Conference on Advanced Information
Networking and Applications Workshops (AINAW'07),875-880.
-
7/31/2019 Literature Review Draft 4
24/24
Page 24 of 24
Vishwakarma, V., Mandal, C., and Sura, S. (2007) Automatic Detection of Human Fall in Video. Pattern
Recognition and Machine Intelligence:Automatic Detection of Human Fall in Video,616623, 2007.
Gill, T.; Keller, J. M.; Anderson, D. T. & III, R. H. L. (2011) A system for change detection and human
recognition in voxel space using the Microsoft Kinect sensor. IEEE, 1-8
C. Rougier, E. Auvinet, J. Rousseau, M. Mignotte, and J. Meunier. Fall detection from depth map video
sequences. In International Conference on Smart Homes and Health Telematics, 2011.