a wearable hybrid haptic feedback stimulation device for...

International Journal of Mechanical & Mechatronics Engineering IJMME-IJENS Vol:19 No:05 104

190205-3737-IJMME-IJENS © October 2019 IJENS I J E N S

Abstract— A sense of touch restored by upper limb prostheses

can greatly help patients with upper limb amputation to perform

activities of daily living. In this work, a novel hybrid haptic

feedback stimulation system for upper limb prostheses has been

designed, developed and evaluated. The wearable device consists

of one servomotor and two vibration motors, which function as a

pressure feedback display and a vibration feedback display,

respectively. The pressure display conveys the sensing of the

contact pressure and its strength level, while the vibration display

gives an indication about the continuity of the contact pressure. An

evaluation on sensation and response has been conducted with

healthy subjects. The results showed that the subjects can identify

the touch, the start of touch, and the end of touch accurately. In

particular, 96 %, 95 % and 88 % of the healthy subjects are able

to distinguish the grasp, the pressure level, and the slipping

objects, respectively. In addition, the usability tests demonstrated

that the system is easy to setup and comfortable to use. The device

is able to convey real-time haptic feedback to the amputees without

the risk of surgery, brain confusion and intensive pre-training.

This solution contributes to making the haptic feedback

stimulation a common functionality in upper limb prostheses in

the near future.

Index Term— Feeling recovering, Haptic feedback stimulation

system, Mechanotactile stimulation, Tactile glove, Upper limb

prostheses, Vibrotactile stimulation.

I. INTRODUCTION

The human hand can be regarded as a multi-degree of freedom

(DOF) system, with massive number of sensors, actuators,

tendons, and a complex control strategy which can perform

multi-tasks in daily life, such as grasping, manipulating objects,

sensing, interacting with the environment, and using gestures to

support speech and express emotions. The loss of human hand

causes severe physical and mental illness [1]. Prostheses have

emerged to help with such physical loss and partially restore the

lost functionality. The upper limb prostheses help the patients

with upper limb amputation to touch and grasp objects, but

compensating the loss of sensation during the use of prosthetic

hands is still in progress. There have been numerous efforts to

develop and equip the prostheses with the haptic feedback

stimulation system, so that the amputees can interact with the

surrounding and restore the sensation as natural as possible.

The haptic feedback stimulation system can be divided into two

main techniques: invasive and non-invasive, depending on the

methods used to stimulate the patient’s peripheral nerves in

order to restore the somatosensory perception. The invasive

technique depends on the surgical intervention and surgical

access to the nerves of the amputees [2, 3], while the non-

invasive technique relies on exciting the patient's nerves

externally by fixing haptic wearable devices on the amputees'

residual parts [4-6]. More specifically, the non-invasive

feedback stimulation system – the priority of this work, can be

further classified into six stimulation systems, depending on

how to stimulate the patient’s skin and provide the information

of the sensory system to the amputee’s brain. The five feedback

stimulation displays are: pressure feedback stimulation system

[7-9], vibration feedback stimulation system [10-13], skin

stretch feedback stimulation system [4, 6, 14-16], squeeze

feedback stimulation system [5, 17, 18], electro feedback

stimulation system [19-23], and thermal feedback stimulation

system [24-30].

Numerous works have been done in order to identify the

most suitable type of haptic feedback display. In general,

comparison between two types displays, like comparison the

rotating and the linear vibration actuators or comparison the

vibration and the skin-stretch wearable devices, were

investigated to convey the sensory information to the amputees'

brain with a high response and good accuracy [31, 32]. On other

hand, several studies revealed the benefits of combining two or

more haptic feedback displays as a hybrid stimulation system

in order to increase the performance of the haptic wearable

device [33-37].

Several issues related to equipping the myoelectric

prosthesis with the haptic feedback stimulation system were

investigated in literature. Firstly, issues related to the

comfortability when using the haptic prosthetic hand, like the

heavyweight [31], the high noise of actuators [38], and absence

of sensation through the prostheses [39] were studied. The

increasing in energy consumption due to using the auxiliary

equipment of the haptic feedback stimulation system was

diagnosed as well [11, 37]. Moreover, issues concerned with the

ability of patient’s brain to recognize multi information

delivered from the sensory system at the same time [12]. For

example, when the sensory system of the prosthetic hand has

pressure, vibration, and temperature sensors, accordingly the

patient’s brain has to distinguish the contact pressure, surface

texture, and the surface temperature at the same time, which

lead to reduce the brain’s recognition accuracy. Finally, issues

regarded to the prostheses design itself.

Mohammed Najeh Nemah1,2, Saif Salih Khaleel3, Omar Hammad Aldulaymi1, Cheng Yee Low1*, Pauline Ong1, Balasem Abdulameer Jabbar1,2

A Wearable Hybrid Haptic Feedback

Stimulation Device for Upper Limb Prostheses

1Faculty of Mechanical and Manufacturing Engineering, University Tun

Hussein Onn Malaysia, 86400, Parit Raja, Batu Pahat, Johor, Malaysia

([email protected]). 2Engineering Technical College-Najaf, Al-Furat Al-Awsat Technical

University, 54001, Najaf, Iraq ([email protected]). 3Biomedical Engineering Department, Al-Mustaqbal University College,

51001 Hillah, Babil, Iraq ([email protected] ).

mailto:[email protected]:[email protected]



It is extremely difficult to design and product haptic upper limb

prostheses that capable to perform whole the users' life

activities comparable to the human real hand. For instance, the

ability of the users to detect and avoid the painful stimulus [9].

In general, it can be concluded that the two main disadvantages

of haptic feedback stimulation system are: (i) patient’s brain

might confuse due to the massive amount of data provided from

the sensory system during the operating of hand prostheses, and

(ii) the need of long hours of pre-training on how to recognize

the sensory information [40].

The main objective of this study is to design a hybrid haptic

pressure-vibration feedback stimulation system (HHPVFSS),

which has the ability to assist the patients of the upper limb

amputation to perceive multi-information about their

surroundings, at the same time, without confusing or requiring

a pre-training. HHPVFSS includes two main parts: the tactile

sensory system and the haptic feedback stimulation system. The

tactile sensory system aims to gather the perceptual information

about the touch, grasp, and the slippage by utilizing the

piezoresistive pressure sensors distributing over the hand. The

feedback stimulation system (FSS) is responsible for the

human-machine interface, by conveying the perceptional

information to the patient's brain through the hybrid wearable

device, which consists of the pressure feedback display and the

vibration feedback display. To the best of our knowledge, this

is the first endeavor to combine haptic pressure and vibration

feedback displays for upper limb prostheses. Such combination

allows the stimulators to operate in different directions and

deliver the tactile sensory information to the users in a quick

manner.

II. DESIGN CONCEPT OF HAPTIC FEEDBACK STIMULATION SYSTEM

In general, the HHPVFSS of upper limb prostheses can be

divided into three main parts: tactile sensory system, computer

system and haptic feedback stimulator system. The tactile

sensory system is responsible for collecting the information

from the environment and converting the information into

measurable data by utilizing different types of sensors. The

computer system is accountable for processing and analyzing

the sensory data, and subsequently, manipulating the comments

signals to control the feedback stimulators to excite the

amputee's brain. Finally, the haptic feedback stimulator system

aims to stimulate the amputees’ nervous system by exciting the

skin of their residual parts, for instance, the forearm, upper

limb, or any other parts of the body.

A. Design Concept Of The Tactile Pressure Sensory System

The main goal of this study is to design a tactile pressure

glove with the ability to detect the contact pressure. The

developed haptic system should have the ability to detect the

contact pressure with six pressure sensors and provide the

measured information to the patient’s brain by utilizing a single

hybrid pressure-vibration feedback stimulation device.

Therefore, the computer system should be programmed in an

effective way in order to select the largest pressure signal

among the six signals of the pressure sensors and transfer it to

the single actuator feedback stimulation device, which can be

fixed on the residual limbs of the amputees.

In order to develop a functional tactile prosthetic hand, the

tactile sensor has to be flexible enough to coat the curving

surfaces of the prosthetic hand [41]. Also, the tactile

force/pressure sensor should be rigid and sensitive enough to

measure the static and the dynamic forces applied on the

prosthetic hand. The piezoresistive force is then utilized to

detect the movement of the slipping objects [42]. In general, the

piezoresistive force sensor is fabricated from a semiconductor

material, which has the ability to change its electrical resistance

when an effective force is applied directly over it.

Based on the above acquaintance, the quantum tunnelling

composites (QTC) with 10 mm diameter, piezoresistive

pressure sensor was chosen to develop a haptic feedback

stimulation system because it has the ability to detect the touch,

grasp, and the object slippage at the same time, in order to cover

the critical points of the touch on the prosthetic hand like the

palm zone and five fingertips. Therefore, five QTC sensors

were fixed on each fingertip and an extra sensor was mounted

on the hand’s palm to increase the chance of capturing the force

affecting the hand, as shown in Fig.1.

Fig. 1. Design conception of a pressure sensory system



The signals from the pressure sensors were transferred to the

computer system by utilizing the Arduino Mega 2560

microcontroller, as shown in Fig.2. After that, the six pressure

signals obtained from the QTC force sensors were compared.

The largest instant signal among all was selected as the output

signal of the sensory system. Based on the output sensory

signal, the computer system controlled the feedback stimulation

actuators by providing special excitation with different effects

for both the pressure and the vibration feedback stimulation

systems.

Fig. 2. Design concept of a HHPVFSS.

B. Design concept of hybrid pressure-vibration wearable feedback device

On the other hand, a pair of 10 mm diameter, circular

vibration motors and a single servo motor were used

simultaneously, in order to provide pressure and vibration

stimulation to the skin of the amputee’s upper arm by exciting

his nervous system.

The servomotor of the pressure feedback stimulation system

was rotated from 0 to 35 degrees when the sensory system

signal was increased from 0 to 20 N. This was to notify the

amputees that the contact pressure between the prosthetic hand

and the grasping object was increased constantly.

The vibration motors were excited with a special periodical

signal, varying from 0 to 3 V for every 0.2 sec, according to any

value of the applied force. This was to inform the amputees that

the contact pressure was still active and the touching or

grasping action was continued. The vibration motors were

activated when the sensory signal exceeded 0.5 V. This was to

prevent the activation of the vibration feedback display when

the phantom signals were generated by the pressure sensors,

contributed by the hysteresis behaviors at no load case. Finally,

the Arduino Mega 2560 microcontroller was used to transfer

the excitation signals to the feedback stimulation actuators, as

presented in Fig.2.

III. FABRICATION OF HAPTIC FEEDBACK STIMULATION SYSTEM

A. Fabrication of Tactile Pressure Sensory System

A plastic glove equipped with spot pressure sensors was

designed as the sensory system of the HHPVFSS. The design concept, fabrication, and the evaluation tests of the tactile

pressure glove have been studied in previous own work [43]. In

general, the tactile glove fabricated from a plastic glove was

shielded with rigid foundations located under the pressure

sensors, to provide sufficient space for installing the pressure

sensors and increase its performance.

The rigid shield was designed using the Solidwork 2018

program and printed by a 3D printer of type Raise-3D-N2-Plus

[44], using Acrylonitrile Butadiene Styrene (ABS) material

[45]. Six QTC pressure sensors of SP200-10 series with 10 mm

diameter and 0.1 N to 20 N operating force from Peratech [46] were distributed over the hand. Five sensors were mounted on

each fingertip and one sensor was fixed on the palm. Such

arrangement of sensors was to cover all critical areas of the

hand and increase the probability to detect the contact pressure

while touching the surfaces or grasping the objects.

Based on this type of fabrication, the benefits obtained are: (i)

the tactile pressure glove can be equipped easily with the

prosthetic hand without obstructing the movement of the

fingers’ joints because the glove is made from elastic material;

(ii) the rigid shields of fingertips and the palm enables the base

of the pressure sensors to be stiff enough to work with high

accuracy; and (iii) the glove is appropriate for various sizes and types of the prosthetic hands due to its extensibility behavior.

The designed HHPVFSS has only one feedback stimulator,

but this singular stimulator has to convey the data of six

pressure sensors to the patient’s brain. To solve the problem,

the tactile sensory system was programmed to output the largest

instant signal from all the pressure signals. If the time

dependent vectors P⃑⃑ (t), R⃑⃑ (t), M⃑⃑⃑ (t), I (t), T⃑⃑ (t), and Pa⃑⃑⃑⃑ (t) represent the pressure sensors signals of the pinky fingertip,

ring fingertip, middle fingertip, index fingertip, thumb

fingertip, and the palm, respectively, the largest signal output

from the tactile sensory system 𝑉𝑂𝑢𝑡⃑⃑ ⃑⃑ ⃑⃑⃑⃑ (𝑡) can be calculated as:

VOut⃑⃑⃑⃑ ⃑⃑⃑⃑ (t) = Max[P⃑⃑ (t), R⃑⃑ (t), M⃑⃑⃑ (t), I (t), T⃑⃑ (t), Pa⃑⃑⃑⃑ (t)] (1)

The InMoov prosthetic arm was used to evaluate the proposed

pressure sensory system, as shown in Fig. 3. The design and the

necessary files of the InMoov prosthetic arm are open sources [47]. The prosthetic arm was built using 3D printer,



servomotors, artificial tendon, and Arduino Mega 2560

microcontroller. After that, the tactile pressure glove was

integrated to the InMoov prosthetic arm.

When the pressure sensors are excited by an external load or

pressure; the pressure sensors will measure this excitation and

present it as an analog signal in voltage. Therefore, the calibration of the pressure sensors is highly recommended to

calculate the relationship between the applied load or pressure

and the output of the pressure sensors. In addition, the

calibration is performed to increase the measuring accuracy and

minimize the error. The output signal of the sensor is then

compared to an accurate static load, and the relationship

between the measured and the accurate value is determined.

For this reason, a load stand device was designed and printed

using a 3D printer, as described in Fig. 4. The device is mounted

on the tactile prosthetic hand and the load is applied gradually,

with 0.5 kg for each increment. Then, the sensor output voltage

is recorded against the increasing of the static load. The calibration process is repeated for each of the six pressure

sensors individually.

The result of the calibration process is described in Fig. 5. The

horizontal and the vertical axes are the applied load and the

output measuring voltage, respectively. In order to model the

relationship between the independent variable (applied load)

and the dependent variable (pressure sensor signal), the 6-th

order polynomial was chosen as the fitting model after

numerous attempts. The obtained fitting model is given as:

FL = P1 × l6 + P2 × l

5 + P3 × l4 + P4 × l

3 + 4.923 × l2 +P5 × l + P6 (2)

Where: l represents the applied load (kg) and FL is a load fitting function. The obtained R-squared of the fitting model is

0.9959, indicating that the data are well explained by the fitting

model. While the values P1−5 are the fitting equation constants, which equals to P1 = 0.01962, P2 = −0.2925, P3 = 1.67, P4 = −4.447, P5 = 4.923, and P6 = −0.01707.

Fig. 3. Equipping the InMoov prosthetic arm with the tactile pressure

sensory system.

Fig. 4. Calibrating the pressure sensory system.

B. Fabrication of hybrid pressure-vibration feedback wearable device

The main aim of the HHPVFSS is to stimulate the amputees’

residual upper limb and convey the sensation of the touch, start

of touch, end of touch, pressure level, grasp, and slippage to the patient’s brain. Accordingly, a hybrid haptic pressure-vibration

feedback stimulator was designed to augment the haptic

sensation and enable the amputees’ brain to recognize the

massive amount of data provided by the pressure sensory

system. The wearable device consists of one SG-90 servomotor

[48] for the pressure stimulation and two linear resonant

actuator (LRA) vibration motors [49] for the vibration

stimulation.

Fig. 5. The performance of the pressure sensors vs. the applied load.

The three actuators were fixed on a curvature wearable case

of 70 cm length and 8 mm thickness, as shown in Fig.6. The

curvature case was designed using Solidwork 2018 and printed



by 3D printer using ABS material [45]. Since the two vibration

motors can work independently and provide the vibration

stimulation directly to the arm, therefore, it was fixed at the

inner side of the curvature wearable case in direct contact with

the patient’s skin. On the other hand, a circular piston of 15 mm

diameter and piston’s arm of 33 mm length have been added to the servomotor to deliver the pressure stimulation.

IV. INTERFACING THE HAPTIC WEARABLE DEVICE WITH THE COMPUTER SYSTEM

The main function of the computer system is to manipulate

and control the actuators of HHPVFSS according to the output

signal obtained from the pressure sensory system. Two different

types of signals were generated by the pressure sensory system.

These signals would be utilized to excite the pressure servomotor and vibration motors of the wearable device. The

pressure actuator would apply a pressing action on the patient’s

upper arm, which is proportional to the largest signal value of

the pressure sensors. Therefore, the value generated by the

pressure sensor within the range of 0 to 5 V will be converted

to a rotational excitation signal varying from 0 to 35 degrees, in

order to control the servomotor and satisfy the pressure

stimulation.

On the other hand, QTC sensor has an unstable behavior at

the absence of applying load [43]. Hence, the vibration motors

only starts working if the pressure signal is equal or more than 0.5 V. Then, the vibration actuators are excited by a specific

generated signal varying from 0 to 3 V with a suitable frequency

to provide a sensible and comfortable stimulation to the

amputees.

Fig. 6. The fabrication of HHPVFSS: a) rear view, and b) inside view.

The Arduino Mega 2560 microcontroller was used to connect

the SG-90 servomotor and the two LRA vibration motors with

the computer system, as described in Fig. 7. The Arduino

microcontroller was equipped with an external DC power

supply to cover the electrical loads required for operating the

pressure and vibration motors.

Fig. 7. Connecting the actuators of the haptic wearable device with the

Arduino microcontroller.

V. THE EVALUATION AND FUNCTIONALITY TESTS

Two functionality tests, specifically, the increasing load test

and slippage detection test, were conducted to evaluate the

functionality of the developed HHPVFSS. Additionally, the

evaluation tests on the system’s response, analysis of the

wearable haptic device behavior and its actuators due to varying the environment’s parameter measured by the tactile pressure

sensory system, were performed as well.

The experimental evaluation of the increasing load test has

been set up by using the load stand as described in Fig. 4. A

static weight was applied directly to the pressure sensor

mounted on the fingertip. The applied mass was increased from

0 to 2 kg by adding 200 g for each increment. The purpose of

this evaluation test is to calibrate the response of the servomotor

and the vibration motors due to the increase of the pressure

sensor signal.

The experimental evaluation of the slippage detection test has been set up by utilizing a cylindrical pipe equipped with a steel

hook at its bottom side. The cylindrical pipe was grasped and

held by the tactile prosthetic hand. Subsequently, various

weights were hanged to the hook of the cylindrical pipe, as

shown in Fig. 8. The first aim of this test is to evaluate the

performance of the stimulators when the load is increased from

0 to 5.5 kg by adding 0.5 kg for each increment. The second

aim is to validate the functionality of the HHPVFSS to detect

the slipping objects.



Fig. 8. Evaluation of slippage detection experiment.

VI. EXPERIMENTAL SETUP AND PROCEDURE

A total of 25 healthy volunteers were involved in this study

(mean age (SD) 27.68 ± 3.92 years). The healthy volunteers were used instead of the amputees’ volunteers for two main

reasons. The first reason is, exciting the human’s nervous

system is the main goal of the experimental testing, and hence,

it is not very important to use amputees’ volunteers. The second

reason is the difficulty to get amputees at any time or place. The

participants were right-hand dominant and did not suffer from any cognitive impairment that could affect their performance

during the tests. Volunteers were seated comfortably against the

table. They wore the haptic wearable device on their upper right

arm, as shown in Fig. 9. The eyes mask was used to ensure that

the volunteers only rely on the haptic information provided by

the haptic wearable device, since the participants have no visual

information of their surrounding during the test. Additional

earmuffs were worn to prevent an auditive detection.

The measured outcome in all the experimental examinations

is the stimuli identification rate (SIR), i.e., the rate that the

volunteers could correctly identify the tactile information by only utilizing the HHPVFSS.

In the first test, the examiner applied random forces on the

pressure sensors of the tactile sensory system through touching

the fingertip of the tactile prosthetic hand, as shown in Fig. 10.

The volunteers were asked if they could discriminate the touch,

start of touch, and end of touch. This is to evaluate the

effectiveness of the haptic wearable device in delivering

sensory information to the patients’ brain by stimulating their

nervous system. Evaluating the ability of the volunteers to

recognize the grasping objects was conducted in the second test.

The volunteers were asked to detect the grasping forces when

the prosthetic hand grasped a small ball or made a handshake

with the examiner, as presented in Fig. 11. This was to examine the functionality of the HHPVFSS to detect the contact pressure

when grasping an object.

Fig. 9. Experimental setup: the volunteer wore HHPVFSS with his right

hand.

Fig. 10. Touch detection test.

In the third test, the loads applied on the fingertip of the tactile

prosthetic hand were increased slowly. The volunteers have to

identify whether they were able to detect the changing force.

This test attempted to measure the effectiveness of the haptic wearable device in delivering a sense of increasing pressure to

the patient's brain.

Finally, the slippage detecting test was performed by making

the prosthetic arm to grasp and hold the cylindrical pipe, while

the hanging load was increased gradually, until the pipe slipped

down from hand, as shown in Figure 4.10. The main goal of this

test was to identify the ability of the HHPVFSS to convey the

information about the slipping objects to the users of the haptic

upper limb prostheses.

The experimenter recorded the volunteers' answers for all the experimental tests as follows: (i) yes, if the volunteers were able

to recognize the type and the duration of the excitation during

the test, (ii) no, if the volunteers were confused and unable to

analyze the haptic information. The comfortability of wearing



and using the haptic wearable device evaluated by the

volunteers was taken into account as well.

Fig. 11. Gasping detection test when the prosthetic arm: a) grasps a ball,

and b) shake hands with the experimenter.

VII. EXPERIMENT RESULTS AND DISCUSSION

A. Experiment of The Hybrid Pressure-Vibration Wearable Feedback Device

For the increasing load test, Fig. 12.a presents the linear

relationship between the rotational movement of the

servomotor and the largest instant sensory signal. The rotational

movement of the servomotor was varied from 0 to 35 degrees

when the pressure signal of the sensory system was increased

from 0 to 5 V, i.e. the load was increased from 0 to 20 N. Based

on the servomotor’s maximum rotational movement (35

degree) and the length of servomotor’s piston arm, the vertical

displacement of the servomotor’s piston was calculated, where

15.6 mm was obtained. This implied that the pressure feedback

display of the HHPVFSS was capable to deform the skin of the user’s upper arm with 15.6 mm vertical displacement. This

value of displacement is suitable to excite patient’s nervous

system and inform the patient’s brain about the increasing load.

This is due to, in general, this value is the highest value that can

be applied to reach the arm bone.

Fig. 12. The relationship between the wearable device’s actuators and the

largest instant sensory signal.

For the vibration feedback display of the HHPVFSS, a

repeatable pulses signal with 3 V maximum amplitude, 0.2 sec

period time, and a pulse width equal to 50 % of the period time was designed to drive the two vibration motors, as shown in

Fig. 13. The special pulses vibration motor signal was designed

based on several stimulation trials, in order to obtain the best

feedback stimulation from the vibration feedback display

without any interference with the noise frequency of the

servomotor.

The special pulsing signal was applied to drive the two

vibration motors at the instant when the contact with the

surfaces or objects took place. Therefore, the vibration motors

were excited by only one type of stimulation signal in all

operation, i.e. the vibration motors never got affected with the

increasing load. In addition, the vibration motors were excited at the instant when the pressure sensors signal exceeded 0.5 V,

in order to avoid the undesirable hysteresis phenomenon of the

piezoresistive pressure sensors that were distributed over the

tactile glove, as described in Fig. 12.b and Fig. 12.c.

Fig. 13. The vibration motor input signal during 1 sec.



On the other hand, Fig. 14 shows the responses of the large

instant pressure sensors signal, the servomotor’s rotational

movement, the special periodical pulsing signal that input to the

vibration feedback display, and the vibration output signal

during increasing the hanging mass from 0 to 5.5 kg by adding

0.5 kg for each increment of every 10 sec. The results of the slippage detection test showed that the rotational movement of

the servomotor was increased gradually when the load was

increased systematically. In addition, the controlling signal of

the vibration motors was generated for all operating period, but

it was only being allowed to pass the signal to the vibration

motors when the signal of the tactile sensory system exceeded

0.5 V.

Furthermore, the test results corroborated the functionality of

the HHPVFSS to help the amputees to recognize the slipping

objects by utilizing the haptic upper limb prostheses. It was

found that after the hanging mass reached 5.5 kg, the volunteer

was unable to hold the pipe. It can be noted that, when the load was decreased at the operation time of 110 secs, the output

signals of the servomotor and the two vibration motors were

rapidly reduced to zero, indicating there was no contact

between the prosthetic hand and the slipped object.

Fig. 14. The responses of the sensors and stimulators when the hanging

weight increased gradually.

B. User Experience Evaluation for the Hybrid Pressure-Vibration Feedback Stimulation System

The SIR of 25 healthy volunteers, which examined the ability

of the HHPVFSS to convey the tactile information to the brain

is described in Fig. 15. The final results were completely based

on the 100 answers taken from all the volunteers for each

experiment test (25 volunteers and four repetitions for each

test). The results showed that all volunteers were able to

recognize the touch, start of touch, and end of touch correctly

with 100 % accuracy. These results gave a positive impression of the effectiveness of the haptic wearable device in transferring

the tactile data.

In particular, 96 %, 95 % and 88 % of the volunteers were

able to detect the grasping objects, the pressure level, and the

slippage, respectively. The relatively low recognition

percentage of the slippage examination could be related to the

design of the tactile glove itself. The tactile glove does not have

a real sliding sensor to detect the slippage. It only depends on

the combination between the pressure sensors and the responses

of the volunteers’ nervous system.

Fig. 15. Stimuli identification rate of the able-body volunteers.

A comparison study between the SIR results of the current

work and eight previous studies was made, as presented in Table 1. The SIR results during touch, grasp, pressure level, and

the slippage examinations were considered. The selected

previous studies were as similar as possible to this work, in

which a pressure sensory system, depending on spot pressure

sensors was developed in the respective works. In addition,

previous studies used different types of feedback stimulation

system, like pressure, vibration, electro, squeeze, and skin

stretch to compare the functionality of the designed wearable

device with all types of existing feedback stimulation systems.

On the other hand, Table 1 summarizes too the information

about the installing position of the previous haptic feedback

displays, type of subjects, and the determination about whether the previous experiments were performed with or without a pre-

training.

The SIR results indicated that the current work recorded the

highest percentage in all the experiment examinations.

Moreover, it should be noted that as compared with previous

studies, no initial exercises and pre-training were included in

this work in order to increase the recognition rate of the

volunteers during the experiments.



VIII. CONCLUSION A novel HHPVFSS of the upper limb prostheses was

designed, in order to help the patients of upper limb amputation

to restore the sensation of their surroundings. The wearable

device was designed to stimulate the user’s upper arm skin by

means of utilizing pressure and vibration feedback stimulation systems. A tactile pressure glove was designed from a plastic

material and equipped with five QTC pressure sensors mounted

on each fingertips and an extra sensor was placed on the palm.

The tactile glove was integrated with the InMoov prosthetic arm

to produce a completely tactile system. On the other hand, the

haptic wearable device was fabricated from one SG-90

servomotor and two LRA vibration motors, which work

together to stimulate the patients’ nervous system.

The design of the tactile glove is the first step for designing a

complete haptic feedback stimulation system to enable the

amputees to sense their surrounding while they wear their own

haptic upper limb prostheses. Therefore, the haptic feedback stimulator is designed in order to provide the useful information

about the contact pressure and the slippage to the amputees’

brain with high response, acceptable accuracy, low noise and

power consumption, and without any brain’s confusing.

However, some limitations of the study need to be considered.

1. It is so difficult to find the amputees volunteers with upper limb amputation to perform the evaluation and

the functionality tests.

2. The designed HHPVFSS only has a single haptic stimulator that is capable to work with one sensory

signal, hence, it is very difficult to install a haptic wearable device that has six stimulators on the

patient’s upper arm to work with each sensory signal

individually.

The experimental results of evaluation tests outcomes reveal

that HHPVFSS provided a suitable level of accuracy and an

acceptable response for detecting the contact pressure. In

general, the evaluation tests and the SIR examinations proved

the functionality and the effectiveness of the HHPVFSS to

provide multi-information to the amputee’s brain without

confusing or pre-training. The SIR examinations recorded 100

% percentage recognition accuracy for the touch, start of touch,

and end of touch experimental tests, while about 96 %, 95 %

and 88 % of the participants were able to perceive the grasping

objects, pressure level and the slippage, respectively.

To the best of our knowledge, this is the first design of a

complete non-invasive haptic feedback stimulator combining the pressure and the vibration feedback stimulation system, in

order to work together at the same operation time. The pressure

feedback was used to convey the information about the contact

pressure and its strength level, while the vibration feedback was

programmed to be an auxiliary stimulator to give the user an

impression of continued contact pressure. In future work, the

HHFSS will be improved further to detect the surface texture

and surfaces temperature in addition to the contact pressure

detection.

Acknowledgments

The authors would like to express their gratitude to Research Fund RMC [Vot E15501] from University of Tun Hussein Onn

Malaysia for funding the research work.

List of abbreviations and symbols:

HHFSS : Haptic hybrid feedback stimulation system.

DOF : Degree of freedom.

ABS : Acrylonitrile Butadiene Styrene.

QTC : Quantum tunnelling composites.

LRA : Linear Resonant Actuator.

SIR : Stimuli identification rate.



TABLE I

Comparison between the stimuli identification rate of the current study with the previous works.

REFERENCES [1] K. Ziegler-Graham, E. J. MacKenzie, P. L. Ephraim, T. G. Travison,

and R. Brookmeyer, "Estimating the prevalence of limb loss in the

United States: 2005 to 2050," Archives of physical medicine and

rehabilitation, vol. 89, pp. 422-429, 2008.

[2] E. L. Graczyk, M. A. Schiefer, H. P. Saal, B. P. Delhaye, S. J.

Bensmaia, and D. J. Tyler, "The neural basis of perceived intensity

in natural and artificial touch," Science translational medicine, vol.

8, pp. 362ra142-362ra142, 2016.

[3] M. Schiefer, D. Tan, S. M. Sidek, and D. J. Tyler, "Sensory feedback

by peripheral nerve stimulation improves task performance in

individuals with upper limb loss using a myoelectric prosthesis,"

Journal of neural engineering, vol. 13, p. 016001, 2015.

[4] M. Rossi, M. Bianchi, E. Battaglia, M. G. Catalano, and A. Bicchi,

"Hap-pro: a wearable haptic device for proprioceptive feedback,"

IEEE Transactions on Biomedical Engineering, 2018.

[5] L. Meli, I. Hussain, M. Aurilio, M. Malvezzi, M. O'Malley, and D.

Prattichizzo, "The hBracelet: a wearable haptic device for the

distributed mechanotactile stimulation of the upper limb," IEEE

Robotics and Automation Letters, vol. 3, pp. 2198 - 2205, 2018.

[6] J. P. Clark, S. Y. Kim, and M. K. O’Malley, "The rice haptic rocker:

Altering the perception of skin stretch through mapping and

geometric design," in IEEE Haptics Symposium (HAPTICS), San

Francisco, CA, USA, 2018.

[7] G. Savioz, V. Ruchet, and Y. Perriard, "Study of a miniature

magnetorheological fluid actuator for haptic devices," in Advanced

Intelligent Mechatronics (AIM), 2010 IEEE/ASME International

Conference on, Montreal, ON, Canada, 2010, pp. 1197-1202.

[8] M. O. Culjat, J. Son, R. E. Fan, C. Wottawa, J. W. Bisley, W. S.

Grundfest, et al., "Remote tactile sensing glove-based system," in

Engineering in Medicine and Biology Society (EMBC), 2010 Annual

International Conference of the IEEE, Buenos Aires, Argentina,

2010, pp. 1550-1554.

[9] S. Fukushima, H. Sekiguchi, Y. Saito, W. Iida, T. Nozaki, and K.

Ohnishi, "Artificial Replacement of Human Sensation Using Haptic

Transplant Technology," IEEE Transactions on Industrial

Electronics, vol. 65, pp. 3985-3994, 2018.

[10] M. Nabeel, K. Aqeel, M. N. Ashraf, M. I. Awan, and M. Khurram,

"Vibrotactile stimulation for 3D printed prosthetic hand," in

Robotics and Artificial Intelligence (ICRAI), 2016 2nd International

Conference on, Rawalpindi, Pakistan, 2016, pp. 202-207.

[11] M. D'Alonzo, F. Clemente, and C. Cipriani, "Vibrotactile

stimulation promotes embodiment of an alien hand in amputees with

phantom sensations," IEEE Transactions on Neural Systems and

Rehabilitation Engineering, vol. 23, pp. 450-457, 2015.

[12] M. R. Motamedi, J.-P. Roberge, and V. Duchaine, "The Use of

vibrotactile feedback to restore texture recognition capabilities, and

the effect of subject training," IEEE Transactions on Neural Systems

and Rehabilitation Engineering, vol. 25, pp. 1230-1239, 2017.

[13] J. M. Fontana, R. O’Brien, E. Laciar, L. S. Maglione, and L.

Molisani, "Vibrotactile Stimulation in the Upper-Arm for Restoring

Individual Finger Sensations in Hand Prosthesis," Journal of

Medical and Biological Engineering, vol. 38, pp. 782–789, 2018.

[14] K. Bark, J. Wheeler, G. Lee, J. Savall, and M. Cutkosky, "A

wearable skin stretch device for haptic feedback," in EuroHaptics

conference, 2009 and Symposium on Haptic Interfaces for Virtual

Environment and Teleoperator Systems. World Haptics 2009. Third

Joint, Salt Lake City, UT, USA, 2009, pp. 464-469.

[15] O. Kayhan, A. K. Nennioglu, and E. Samur, "A skin stretch tactor

for sensory substitution of wrist proprioception," in Haptics

Symposium (HAPTICS), 2018 IEEE, San Francisco, CA, USA,

2018, pp. 26-31.

[16] E. Battaglia, J. P. Clark, M. Bianchi, M. G. Catalano, A. Bicchi, and

M. K. O'Malley, "The Rice Haptic Rocker: skin stretch haptic

feedback with the Pisa/IIT SoftHand," in World Haptics Conference

(WHC), 2017 IEEE, Munich, Germany, 2017, pp. 7-12.

[17] D. D. Damian, M. Ludersdorfer, Y. Kim, A. H. Arieta, R. Pfeifer,

and A. M. Okamura, "Wearable haptic device for cutaneous force

and slip speed display," in Robotics and Automation (ICRA), 2012

IEEE International Conference on, Saint Paul, MN, USA, 2012, pp.

1038-1043.

[18] E. Treadway, B. Gillespie, D. Bolger, A. Blank, M. O'Malley, and

A. Davis, "The role of auxiliary and referred haptic feedback in

References

Type of

sensory

system

Type of

Feedback

display

Setup of

Feedback

display

Type of subjects

Experimental

with

pre-training

Stimuli identification rate (SIR)

Touch Grasp pressure

level Slippage

Current

Study Pressure

Hybrid Pressure

and Vibration Upper limb

Healthy

volunteers No 100% 96% 95% 88%

[8] Pressure Vibration Forearm Healthy

volunteers No 99.3% 94.7%

[50] Pressure Vibration Upper limb Healthy

volunteers No 94% 75%

[51] Pressure Vibration Upper limb Healthy

volunteers Yes 93.9% 85%

[34] Pressure Pressure and

Vibration Upper limb

Amputees

volunteers Yes 93% 90%

[19] Pressure Electro Forearm

Healthy and

amputees

volunteers

Yes 90%

[17] Pressure Squeeze Forearm Healthy

volunteers Yes 70.3% 62.2%

[16] Pressure Skin stretch Upper limb Healthy

volunteers Yes 73.3%

[10] Pressure Vibration Forearm

Healthy and

amputees

volunteers

No

Yes

72%

94%



myoelectric control," in World Haptics Conference (WHC), 2015

IEEE, Evanston, IL, USA, 2015, pp. 13-18.

[19] M. Štrbac, M. Belić, M. Isaković, V. Kojić, G. Bijelić, I. Popović,

et al., "Integrated and flexible multichannel interface for

electrotactile stimulation," Journal of neural engineering, vol. 13,

p. 046014, 2016.

[20] M. Franceschi, L. Seminara, S. Dosen, M. Strbac, M. Valle, and D.

Farina, "A system for electrotactile feedback using electronic skin

and flexible matrix electrodes: Experimental evaluation," IEEE

transactions on haptics, vol. 10, pp. 162-172, 2017.

[21] E. I. Germany, E. J. Pino, and P. E. Aqueveque, "Myoelectric

intuitive control and transcutaneous electrical stimulation of the

forearm for vibrotactile sensation feedback applied to a 3D printed

prosthetic hand," in Engineering in Medicine and Biology Society

(EMBC), 2016 IEEE 38th Annual International Conference of the,

Orlando, FL, USA, 2016, pp. 5046-5050.

[22] M. Štrbac, M. Isaković, M. Belić, I. Popović, I. Simanić, D. Farina,

et al., "Short-and Long-Term Learning of Feedforward Control of a

Myoelectric Prosthesis with Sensory Feedback by Amputees," IEEE

Transactions on Neural Systems and Rehabilitation Engineering,

vol. 25, pp. 2133-2145, 2017.

[23] D. Pamungkas and K. Ward, "Tactile sensing system using electro-

tactile feedback," in Automation, Robotics and Applications

(ICARA), 2015 6th International Conference on, Queenstown, New

Zealand, 2015, pp. 295-299.

[24] S. Gallo, C. Son, H. J. Lee, H. Bleuler, and I.-J. Cho, "A flexible

multimodal tactile display for delivering shape and material

information," Sensors and Actuators A: Physical, vol. 236, pp. 180-

189, 2015.

[25] M. Hojatmadani and K. Reed, "Asymmetric Cooling and Heating

Perception," in International Conference on Human Haptic Sensing

and Touch Enabled Computer Applications, 2018, pp. 221-233.

[26] M. Gabardi, D. Leonardis, M. Solazzi, and A. Frisoli, "Development

of a miniaturized thermal module designed for integration in a

wearable haptic device," in Haptics Symposium (HAPTICS), 2018

IEEE, 2018, pp. 100-105.

[27] M. Gabardi, D. Chiaradia, D. Leonardis, M. Solazzi, and A. Frisoli,

"A High Performance Thermal Control for Simulation of Different

Materials in a Fingertip Haptic Device," in International Conference

on Human Haptic Sensing and Touch Enabled Computer

Applications, 2018, pp. 313-325.

[28] M. Nakatani, K. Sato, K. Sato, Y. Kawana, D. Takai, K.

Minamizawa, et al., "A novel multimodal tactile module that can

provide vibro-thermal feedback," in International AsiaHaptics

conference, 2016, pp. 437-443.

[29] K. Sato and T. Maeno, "Presentation of sudden temperature change

using spatially divided warm and cool stimuli," in International

Conference on Human Haptic Sensing and Touch Enabled

Computer Applications, 2012, pp. 457-468.

[30] S. Gallo, L. Cucu, N. Thevenaz, A. Sengul, and H. Bleuler, "Design

and control of a novel thermo-tactile multimodal display," in

Haptics Symposium (HAPTICS), 2014 IEEE, 2014, pp. 75-81.

[31] H. Huang, T. Li, C. Antfolk, C. Enz, J. Justiz, and V. M. Koch,

"Experiment and investigation of two types of vibrotactile devices,"

in Biomedical Robotics and Biomechatronics (BioRob), 2016 6th

IEEE International Conference on, Singapore, Singapore, 2016, pp.

1266-1271.

[32] K. Bark, J. W. Wheeler, S. Premakumar, and M. R. Cutkosky,

"Comparison of skin stretch and vibrotactile stimulation for

feedback of proprioceptive information," in Symposium on Haptic

Interfaces for Virtual Environments and Teleoperator Systems

2008, Reno, Nevada, USA, 2008, pp. 71-78.

[33] C. Antfolk, M. D'Alonzo, M. Controzzi, G. Lundborg, B. Rosén, F.

Sebelius, et al., "Artificial redirection of sensation from prosthetic

fingers to the phantom hand map on transradial amputees:

vibrotactile versus mechanotactile sensory feedback," IEEE

transactions on neural systems and rehabilitation engineering, vol.

21, pp. 112-120, 2013.

[34] H. Huang, T. Li, C. Bruschini, C. Enz, J. Justiz, C. Antfolk, et al.,

"Multi-modal Sensory Feedback System for Upper Limb

Amputees," in 2017 New Generation of CAS (NGCAS), Genova,

Genoa, 2017, pp. 193-196.

[35] E. Romero and D. Elias, "Design of a non invasive haptic feedback

device for transradial myoelectric upper limb prosthesis," in

ANDESCON, 2016 IEEE, Arequipa, Peru, 2016, pp. 1-4.

[36] A. Ajoudani, S. B. Godfrey, M. Bianchi, M. G. Catalano, G. Grioli,

N. Tsagarakis, et al., "Exploring teleimpedance and tactile feedback

for intuitive control of the pisa/iit softhand," IEEE transactions on

haptics, vol. 7, pp. 203-215, 2014.

[37] M. C. Jimenez and J. A. Fishel, "Evaluation of force, vibration and

thermal tactile feedback in prosthetic limbs," in Haptics Symposium

(HAPTICS), 2014 IEEE, Houston, TX, USA, 2014, pp. 437-441.

[38] P. Kapur, M. Jensen, L. J. Buxbaum, S. A. Jax, and K. J.

Kuchenbecker, "Spatially distributed tactile feedback for kinesthetic

motion guidance," in Haptics Symposium 2010, Waltham,

Massachusetts, 2010, pp. 519-526.

[39] J. Wheeler, K. Bark, J. Savall, and M. Cutkosky, "Investigation of

rotational skin stretch for proprioceptive feedback with application

to myoelectric systems," IEEE Transactions on Neural Systems and

Rehabilitation Engineering, vol. 18, pp. 58-66, 2010.

[40] M. N. Nemah, C. Y. Low, O. H. Aldulaymi, P. Ong, and A. A.

Qasim, "A Review of Non-Invasive Haptic Feedback stimulation

Techniques for Upper Extremity Prostheses," International Journal

of Integrated Engineering, vol. 11, 2019.

[41] H. Ritter, R. Haschke, and J. J. Steil, "A dual interaction perspective

for robot cognition: grasping as a “rosetta stone”," in Perspectives

of neural-symbolic integration, ed: Springer, 2007, pp. 159-178.

[42] D. P. Cotton, P. H. Chappell, A. Cranny, N. M. White, and S. P.

Beeby, "A novel thick-film piezoelectric slip sensor for a prosthetic

hand," IEEE sensors journal, vol. 7, pp. 752-761, 2007.

[43] M. N. Nemah, C. Y. Low, P. Ong, and N. A. C. Zakaria,

"Development and evaluation of a spot sensor glove for the tactile

prosthetic hand," International Journal of Engineering and

Technology (UAE), vol. 7, pp. 63-69, 2018.

[44] D. P. PVT.LTD. RAISE 3D N2 PLUS. Available:

https://www.ithink3dp.com

[45] A. M. Donald and E. J. Kramer, "Plastic deformation mechanisms

in poly (acrylonitrile-butadiene styrene)[ABS]," Journal of

Materials Science, vol. 17, pp. 1765-1772, 1982.

[46] Peratech. (November 2015). QTC SP200 Series Datasheet SP200-

05 Series, Single Point Sensors. Available:

https://www.peratech.com

[47] C. Antfolk, A. Björkman, S.-O. Frank, F. Sebelius, G. Lundborg,

and B. Rosen, "Sensory feedback from a prosthetic hand based on

air-mediated pressure from the hand to the forearm skin," Journal

of rehabilitation medicine, vol. 44, pp. 702-707, 2012.

[48] OnlineShop. (2018). Micro Servo 5-10g / SG90 Analog. Available:

http://www.towerpro.com.tw

[49] PrecisionMicrodrives. (2018). 10mm Linear Resonant Actuator.

Available: https://www.precisionmicrodrives.com

[50] L. C. Y. Mohammed Najeh Nemah, Ong Puluy,, "A vibrotactile

prosthetic device for detection of contact pressure and surface

texture in upper extremity," In progress: International Journal of

Advanced Robotic Systems, 2019.

[51] J. M. Fontana, R. O’Brien, E. Laciar, L. S. Maglione, and L.

Molisani, "Vibrotactile Stimulation in the Upper-Arm for Restoring

Individual Finger Sensations in Hand Prosthesis," Journal of

Medical and Biological Engineering, pp. 1-8, 2018.
http://www.ithink3dp.com/http://www.peratech.com/http://www.towerpro.com.tw/http://www.precisionmicrodrives.com/

a wearable hybrid haptic feedback stimulation device for...

Documents