2010 IEEE EMBS Conference on Biomedical Engineering & Sciences (IECBES 2010), Kuala Lumpur, Malaysia, 30th November - 2nd December 2010.

Haptic Based Augmented Reality Simulator for

Training Clinical Breast Examination

Mitan SoIanki, Vinesh Raja

Informatics & Virtual Reality Group, WMG The University of Warwick

Coventry, U.K. m.solanki@warwick.ac.uk

Abstract- Palpation is a procedure that enables a practitioner to

obtain tactile information about the internal condition of tissue

that is not visible from the surface. However the methods

currently employed to train these clinical diagnosis skills are

insufficient in accurately rendering the sensations of

abnormalities to the fingertips. In this paper we propose an

augmented reality haptic palpation system that simulates a wide

range of cancer pathologies to be implemented in training breast

examination. A homogeneous silicone elastomer material

mimicking breast tissue is palpated with the finger connected to a

haptic interface, which generates the perception that a seemingly

empty medium contains embedded tumours. In order to

represent stages and severity of the disease, the parameters of

depth, diameter, stiffness and location of the virtual lumps are

considered to contribute a fully reconfigurable biomechanical

inhomogeneous model. Experiments are conducted with users to

observe how accurately these parameters can be identified on the

breast surface. The results demonstrate that subjects could

precisely ascertain the objects composition and also distinguish

virtual lumps of different material compliance. The system

provides a novel alternative to existing healthcare trainers and

moves closer towards an immersive virtual reality simulator.

1. INTRODUCTION

The early detection of breast cancer is of great significance in improving the survival rate from the disease. It is the most common variant of cancer amongst women in the UK and the chances of surviving it are directly related to the stage at which the cancer is diagnosed [I]. The primary screening method involves conducting a clinical breast examination (CBE). This involves a visual inspection of the breast to discern any noticeable symptoms, but also a thorough tactile exploration in an attempt to distinguish any abnormal internal structures from the surrounding tissue. Although it doesn't permit the practitioner to establish a malignant tumour from a benign one, it is useful to uncover suspicious lesions for further examination. Palpation of anatomical mediums requires honed psychomotor skills however deficiencies in training this has been highlighted [2], [3] especially in medical schools where many healthcare professionals are educated.

A. Static simulators

In order to increase competency in palpation, simulators are designed to maximise the acquisition of tactile knowledge, focusing primarily on the realism that can be achieved and the

978-1-4244-7600-8/101$26.00 ©201 0 IEEE 265

training time allocated to students. The methods adopted and standardised across many medical schools makes use of simple silicone models resembling a breast embedded with nodularities that imitate the presence of abnormal structures. Although this may suffice to elevate ones tangible knowledge of a range of breasts, dynamic simulators must be introduced as they permit a degree of variability in teaching common or rare cancer pathologies. To embed masses that feel stiffer than the surrounding material [4] inflates small balloons with fluid inside the silicone model, to regulate certain properties of the lump. However in this approach, the locations of where the balloons are positioned are physically fixed, inhibiting progression towards a model that can be reconfigured to simulate variations in the disease. Some manufactured breast prostheses take a step further in emulating the multiple layers of a real breast by creating a bi-Iayered silicone to mimic adipose (fat) and glandular tissue. An advanced study [5] involves utilising pneumatic technology to simulate the presence of a tumour where contact with the surface of the model is initialised. Stiffness properties and location of the lump are variable and can be controlled to a certain extent but there are difficulties manipulating the apparent size and depth of the lump. Again the dynamism and reconfigurability of the model is inadequate. In these formulations, the concept of being able to rearrange or redesign a generic model to portray other diseased conditions is essential. Present simulators provide a limited range of pathological scenarios with one model, where tumours are fixed in size and location. Therefore it is of great importance to provide a simulator that offers early healthcare practitioners the opportunity to enhance and broaden their skill sets.

B. Haptic augmented reality

Augmented reality is traditionally used to provide a user with an ancillary layer of visual information to supplement the real world that they are seeing around them. Head mounted displays illustrate this concept well, by monitoring the position and orientation of the head and, overlaying images of buildings, street signs or the cockpit for example, from the appropriate observer's perspective. This is also being extended to mobile applications utilising built in cameras and image recognition software.

Estimations of how deformable an object is, is heavily influenced by its graphical representation because of the visual

dominance within the human sensory system [6], [7]. In

particular, the perception of an object's compliance can be

altered by manipulating what is visually rendered in relation to the haptic response [8]. The emphasis of visual and haptic

stimuli in virtual simulation is applicable depending on the

specific task; for example in virtual surgery situations, the

visual sense takes priority in creating incisions and suturing

whilst haptics takes precedence when exploring hidden

regions of the anatomy that are not evident through the endoscopic camera.

To enhance the user experience and deliver further tactile

knowledge about the properties of an object, a similar notion

is employed in haptics. Additional force stimuli are transferred

via a haptic interface in conjunction with whatever force

response is being received from the real object, augmenting the overall perception of the feel of the object as shown in Fig.

1. Amalgamating the stimulus that occurs when in contact

with a real object with a modulated force generated through a

haptic device has been explored for homogeneous objects; that

is for a medium possessing the same material properties

distribution throughout the space it occupies. It is possible to modify the stiffness of an actual object by transmitting a pre­

defined force to the user via a haptic device, which the user

then explores the object with [9]. Similarly supplying an

additional force to provide the illusion that the real object

being interacted with possesses the desires properties is achievable [10]. Augmented feedback to novices training in

surgical procedures to assist in grasping laparoscopic tools has

also proved successful, demonstrating that the supplementary

tactile cues allowed them to remain within acceptable grasping

limits [11]. Minute amplified haptic signals to facilitate tasks

with the fingers have also been employed [12]. To improve the manipulation of operational sliders a synthetically generated

force can be applied to just the fmgertips [13]. Training for

palpatory diagnosis is explored on a homogeneous object

where differences in skin surface are sought with a modified

haptic stylus to permit additional force transmission to the user

[14]. However, so far little research has been conducted to investigate how to represent the presence of a virtual object,

within a real object through a haptic interface.

Fig. I How haptic stimuli are generated.

266

In reality human organs comprise of a multitude of

materials with differing properties that is defined as

inhomogeneous objects. Within the context of medical simulation, abnormalities in soft tissue can be distinguished

from surrounding tissue at the region of interaction on the skin.

Breasts, livers or lungs exemplify inhomogeneous models in

nature as they are all capable of developing very rigid tumours

sharply contrasting their surrounding softer tissue [15].

Experiments carried out to capture the surface behaviour of a spherical inclusion embedded within a homogeneous silicone

medium, utilise specific parameters to manipulate the setup.

Firstly the diameter of the inclusion permits the variation in its

size, which in a medical context is used to determine the stage

to which a cancer has developed. Secondly the depth is

measured from the objects contact surface, which affects higher amplitudes or peak force responses. This diminishes as

the inclusion sits deeper towards the chest wall and isn't often

seen as a detrimental. Thirdly, the position in which the

inclusion is situated also dictates the severity of the disease,

especially if a malignant lump is in close vicinity to axiallary

lymph nodes. Finally the inclusion is distinguished by its stiffness, which is at least 10 times higher than the

surrounding tissue. Multiple training scenarios allowing

repeated practice can be facilitated by utilising these

parameters for reconfiguration of the virtual lump inside a real

object. In this paper we explore how virtual inhomogeneities

embedded in a uniform medium are perceived on the surface.

To investigate this we augment an otherwise homogeneous

real object with additional haptic cues representing an

inclusion. The total force response is generated by combining

the real force of the homogenous breast model and the virtual force from the inclusion, transmitted through a haptic interface.

The feasibility of reconfiguring the model virtually is also

explored through the configuration parameters. Subjective

assessments corresponding to each parameter are conducted

with users who are presented with a range of scenarios. We

demonstrate that the integration of virtual reality offers a viable alternative to existing static training simulators.

II. EXPERIMENTAL METHODS

The experiments are conducted by reconfiguring the virtual

lump to generate different breast-tumour compositions. In all,

six scenarios are created to ascertain what the use perceives and whether the inclusion is represented correctly using the

configuration parameters. These are of interest because they

can be related directly to the medical condition of a breast.

These are summarised in Table 1. A breast model constructed

of medical grade RTV silicone elastomers forms our base

model. The model itself is homogeneous and contains no embedded abnormalities as it is employed as a reference to

depict healthy tissues. It is positioned on a flat surface and in a

permanent position relative to the haptic interface. The

SensAble PHANTOM Desktop is integrated to provide force

feedback because of its higher sensitivity in comparison to its

stylus based counterpart, the Omni. The index finger is affixed

Scenario

I 2 3 4 5 6

TABLE I EVALUATION CONFIGURA nONS

Diameter Depth Stiffness (cm) (cm) 1.5 0.5 Stiff I 1.5 Stiff 2 2 NonStiff I I Stiff

2.5 0.5 Stiff 3 2.5 NonStiff

Anterior location (x,y)

(0,2) (0,6) (2,-1 ) (-2, I) (2,4.5) (0,0.5)

to the stylus by elasticised fabric that ensures a conjoined

motion. The virtual lumps are software generated and are

positioned to remain within the domain space occupied by the

real silicone based breast. The physical setup is illustrated in

Fig. 2. Fifteen subjects were involved in this initial study, seven of

whom were female and eight of whom were male. This ratio is

approximately representative across both genders but more

importantly takes into account that both female and males can

develop breast cancer. Each subject was introduced to the

system and signed a form of consent to take part in the study. In order to become familiar with the virtual environment,

subjects were given some time to interact with virtual objects

in pre-experiment training. They also had the opportunity to

see the arrangement of the configuration sheet that they would

complete during palpation with their free hand to record what

they perceived. To distinguish stiff and nonstiff inclusions in the study, an example of each was felt prior to the experiment.

The haptic stylus was lightly attached to the index finger of

their choice.

The embedded virtual inclusions varied in diameter from

1 cm to 3cm and the depths ranged between O.5cm to 3cm.

Given that the average areolar depth is quoted as 4.6cm [16], inclusions were positioned so as not to interfere with the real

contact surface. Because the precise elasticity modulus is not

quantifiable, the stiffness of the sphere in two scenarios is

minimised to ascertain whether its compliance can just be

distinguished. The initial familiarisation with the solitary

virtual objects facilitates this. Breast cancers themselves are statistically more likely to develop in areas of high adipose

Fig. 2 Palpating for a virtual inclusion in the experimental setup.

267

tissue concentration and within lactiferous ducts. Clinicians

often visualize the breast in four quadrants centered on the

mammary papilla or nipple. We arrange the locations of the inclusions in the x-y plane viewed from the anterior

perspective.

The subjects explore a real silicone based homogeneous

material emulating the human breast with their index fmger

and coalesced haptic stylus. The stiffness of the real material

is supplemented with the apparent stiffness that of the virtual, nonexistent spheres, creating the illusion that a lump is

embedded. The tactile information gathered by the user is

recorded on the provided form, an example of which is

depicted in Fig. 3. We envisaged that the user would commit

their favoured hand to the exercise. Therefore the form

consists of a scale representing the perceived depth, diameter, apparent stiffness of the virtual lump and an anterior outline of

the breast; each of which can be selected by a cross drawn by

the less dominant hand. One sphere is embedded per

configuration to limit the cross dependencies between

parameters. This procedure of palpating and recording by the

user is repeated for six scenarios.

[]:�, O�·:·'; D :J' o�·:·-;; [] :�' o":'''v 2 ! )( 2 g 2 !

1 0 1 0 I 0 :;< o 0 0

, 1· .. ·'.. i e", .... ,-- 0 : :'1*'" 3 o 1 1 3 4 0 1 1 3 4

[]:�, o�':·';' [] :f' o�':·� �:t, O":'''v 1 � 2 i 2 :l':

1 X o 1 0 1 6-o 0 0

� )�l 0 ';"" 3 4 0 : o .� ... :. 3 4 i i C .... '�-4 Fig. 3 An example of a completed configuration sheet.

III. RESULTS AND ANALYSIS

In the first experiment we analysed how well the diameter parameter is embedded into the model and how it is perceived. Subjects went through each of the six configurations and marked on the scale how large they perceived the diameter to be in centimetres. It was observed that as the configurations progressed, an exploration strategy began to develop when palpating for the size of the lump. Small circular motions, as often advocated by expert practitioners, emerged when probing around the upper convex face of the spherical inclusion. This was visible on our system as we could monitor the direction, orientation and pattern of search of the haptic interface. The precision in the diameter of the detected inclusions is obtained by comparing the values concluded by the users to the actual size in set in the virtual environment and calculating the percentage accuracy, shown in Fig. 4.

� ", <.> to

:; <.> <.>

« '" '" to

C '" � '" <L

100

90

80

70

60

50

40

30

20

10

Config 1 Con fig 2 Con fig 3 Config 4 Con fig 5 Config 6

Fig. 4 Accuracy in identi fying the diameter.

The percentage accuracy is averaged across participants for

each configuration in the proposed setup which demonstrates

that a high detection rate can be achieved. The average

perceived diameter by participants deviated by OAcm in

configuration 3. The general trend was that the spherical

lumps were perceived to be slightly smaller than the actual

size set, generating a small error. This can be attributed to the real silicone breast model providing its own elastic resistance

inhibiting the subject's finger to probe the entire spherical

width.

The second experiment considered the depth in a similar

manner. This parameter is defined by the depth that is

traversed by the subject's finger from the contact position of on the surface of the silicone breast model, until the embedded

sphere is intercepted. Assessing this parameter is more

difficult than the diameter in real situations during breast

examination, which was captured quite well in our study in

Fig. 5.

A larger magnitude of force must be applied in order to detect some of the inclusions residing deeper in the breast

material. In configurations 5 and 6 the accuracy diminishes to

59% and 56% respectively. We identify that the two largest

inclusions of 2.5cm and 3cm diameters are set in these

configurations; however their relationship with the depth is challenging to interpret explicitly without further research.

;R 0

>-u '" :; u u « Q) 0> S c Q) u Q; 0..

100

90

80

70

60

50

40

30

20

10

0

Con fig 1 Config 2 Con fig 3 Con fig 4 Config 5 Con fig 6

Fig. 5 Accuracy in identifying the depth.

268

Fig. 6 Accuracy in identifying the stiffness.

The experiment to investigate whether response in the variance in stiffness of the embedded inclusion, is transmitted to the fmger during palpation is explored next. To make this distinction, subjects are informed that amongst the six scenarios that they are going to be inspecting, some of them wi II include stiff and nonstiff spheres as experienced in the pre­experiment training. Fig. 6 portrays the accuracy in distinguishing these.

The mean number of inclusions whose stiffness was correctly identified was 81 %. The incorrectly determined

lumps are obligated to the compliance of the breast material,

as it augments the perception of the tumour stiffness. Of those

that were incorrectly diagnosed, a further breakdown is

provided illustrating how many in each configuration were apprehended.

In palpation, the location of in which the tumours are

located is also important. A cross sectional plane oriented in

the anterior direction is envisioned, eliminating the

dependency on the depth parameter. Thus focusing on the x-y

coordinates; the mean deviation in centimetres between the actual positions and positions highlighted by the subjects is

calculated and plotted in Fig. 7. In order to do this, motion in

the virtual environment is calibrated to represent motion in

real world coordinates and in dimensions of centimetres,

creating a one to one mapping. The cross sectional

representations of the breast on the configuration sheet are also scaled in proportion to the dimensions of the real silicone

breast in centimetres. The locations are measured with

reference to the nipple, centred at (0,0).

E s c o

�

.� '0 C o

� u o �

Config 1 Config2 Co,,'lg :1 Con",,4 Conllg5 Contlg e

Fig. 7 Participant deviation from the set location.

The mean deviation in centimetres across each

configuration is given by the pink horizontal line located

above the median value. This represents on average how far in the projected anterior plane the lump was detected by subjects,

from its actual location. These values ranges between O.64cm

and 1.08cm demonstrating that our system configured with

virtual lumps provides a realistic level of accuracy and begins

to establish testing standards. Thus potential users of the

system will benefit from the capability of locating cancerous lumps with such a reliable degree of accuracy.

IV. CONCLUSIONS

In this paper we evaluate the use of virtual reality in

parameterising cancerous lumps and detecting them through a

system that incorporates real and haptic feedback to the user.

Although the current standard of teaching and training breast examination relies on the palpation of synthetic models with

static rubber nodules, they are insufficient in representing the

variety of cancerous pathologies that users can be exposed to

in real life. Thus for realising a realistic palpation simulator,

we propose an alternative method with a goal to enhance the

training of clinical diagnosis. This paper contributes a novel utilisation of augmented haptic reality to generate the

perception of a cancerous lump through a haptic interface, in

an otherwise uniform silicone based medium resembling a

human breast. Furthermore the developed inhomogeneous

model can be reconfigured to represent different biomechanical scenarios by placing emphasis on the depth,

diameter, stiffness and location of the virtual lumps. The

experimental results demonstrate that these parameters

defining the characteristics of the tumour can be accurately

detected on the surface of the simulated skin under palpation,

to enable users to make informed decisions about the stage and severity of a potential cancer. Future work will involve

developing a more intuitive means of interaction using multi­

touch haptics to incorporate three fmgers. Advancing to a

system that is situated entirely in virtual reality will permit the

system to emulate a large range of breasts in comparison to

one fixed model. This will integrate various visual symptoms of breast cancer, different ethnicities and ages of women, as

well as material properties. Additionally, further modelling of

the skin surface and tumour relationship can be facilitated.

269

REFERENCES

[I] A. Nover, S. Jagtap, w. Anjum, H. Yegingil, w. Shih, W. Shih, and A. Brooks, "Modern breast cancer detection: a technological review," Journal of Biomedical Imaging, vol. 2009, pp. 4, 2009.

[2] K. Lee, D. Dunlop, and N. Dolan, "Do clinical breast examination skills improve during medical school?," Academic Medicine, vol. 73, (no. 9), pp. 1013, 1998.

[3] S. Barrett, J. Zapka, K. Mazor, and R. Luckmann, "Assessing third­year medical students' breast cancer screening skills," Academic Medicine, vol. 77, (no. 9), pp. 905,2002.

[4] G. Gerling, M. Martin, and S. Rigsbee, "Physical simulators for training clinical palpation skills," 2007.

[5] M. Takaiwa and T. Noritsugu, "Development of palpation simulator Using Pneumatic Parallel Manipulator," Proceedings of the 6th JFPS

International, vol. I, pp. 3,2005. [6] E. Burns, S Razzaque, A. Panter, M. Whitton, M. McCallus, and F.

Brooks Jr, "The Hand is Slower than the Eye: A quantitative exploration of visual dominance over proprioception," 2005.

[7] W. Wu, C. Basdogan, and M. Srinivasan, "Visual, haptic, and bimodal perception of size and stiffness in virtual environments," Asme Dyn

Syst Control Div Publ Dsc., vol. 67, pp. 19-26, 1999. [8] B. Knorlein, M. Di Luca, and M. Harders, "Influence of visual and

haptic delays on stiffness perception in augmented reality," 2009. [9] S Jeon and S Choi, "Modulating real object stiffhess for haptic

augmented reality," Haptics: Perception, Devices and Scenarios, pp. 609-618, 2008.

[10] Y. Kurita, A. Ikeda, T. Tamaki, T. Ogasawara, and K. Nagata, "Haptic augmented reality interface using the real force response of an object," Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology, pp. 83-86,2009.

[II] E. Westebring-van der Putten, 1. van den Dobbelsteen, R. Goossens, 1.

Jakimowicz, and 1. Dankelman, "The effect of augmented feedback on grasp force in laparoscopic grasp control," IEEE Transactions on

Haptics, 20 I O.

[12] T. Nojima, D. Sekiguchi, M. Inami, and S. Tachi, "The smarttool: a system for augmented reality of hap tics," vr, pp. 67,2002.

[13] C. Borst and R. Volz, "Evaluation of a haptic mixed reality system for interactions with a virtual control panel," Presence: Teleoperators &

Virtual Environments, vol. 14, (no. 6), pp. 677-696,2005. [14] E. Karadogan, R. Williams, J. I-lowell, and R. Conatser Jr, "A stiffness

discrimination experiment including analysis of palpation forces and velocities," Simulation in Healthcare, 2010.

[15] S. Lin, Y. Lee, and R. Narayan, "Heterogeneous deformable modeling of bio-tissues and haptic force rendering for bio-object modeling," Printed Biomaterials, pp. 19-37,2010.

[16] 1. Ladeji-Osias and N. Langrana, "Analytical evaluation of tumors surrounded by soft tissue," Proceedings of the 22nd Annual EMBS International Conference, vol. 3, pp. 2114-2117, 2000.