user experience design portfolio, harry brenton
TRANSCRIPT
Sample case studies
Case study #2 digital anatomy
Case study #1 social music annotation
Case study #4 emergency medicine
Case study #5 embodied dance
Case study #3 virtual stroke patient
!Client Goldsmiths University, Embodied Audio and Visual Interaction Group (EAVI)
Brief To allow musicians to collaboratively annotate audio & video recordings. !Team The project was managed a software architect who built the backend infrastructure. There were two senior software engineers, one junior developer and two music teachers. !
My role I led the user centred design process and managed a junior developer who implemented GUIs from my wireframes
I also managed the two music teachers who created user journeys and evaluated software prototypes.
Deliverables • A feedback support system called The Social Timeline that integrated into a social network. • Papers & reports describing the system. !!
Case study #1 social music annotation
User needs Musicians get insufficient feedback about ‘work in progress’. Feedback is often too vague to act upon. !Existing tools like SoundCloud are imprecise because they don’t allow comments on a region of a performance. !
Approach • Ethnographic studies to understand
user needs in context. • Users tests of existing UI design
patterns. • User centred development of a new
design pattern for social annotation. • Three user studies (each six months). !
Challenges • Creating an interface that imposes
beneficial constraints on information. • Growing a ‘pop up’ social network
from a cold start.
Interviews & observations User journeys
Card sorting study to map existing design patterns
research | concept development | prototyping | solutionCase study#1 social music annotation 4 of 25
User journeys (technology enhanced) Wireframes of social user interfaces
Multi-layered timeline for overlapping commentsRegional cue-points to make feedback specific
research | concept development | prototyping | solutionCase study#1 social music annotation 5 of 25
Username Song title
Post
Network graph
Waveform selection becomes cue point
Click cue point to loop playback
Comment pops up on MouseOver
Cue points stack up on timeline
Cue points can contain a threaded discussion
Refine interactions & workflow Test prototypes in action with real data
User screen recordings Remote user observations
research | concept development | prototyping | solutionCase study#1 social music annotation 6 of 25
Solution A new interaction design pattern called The Socia l Timel ine , for group annotation of audio & video recordings. !We embedded The Social Timeline in a ‘pop-up social network’ and ran 3 user s t u d i e s i n c l a s s i c a l , j a z z a n d contemporary music.
Outcomes • The interface focused feedback
directly on the ‘performance rather the performer’.
• This raised the quality of feedback by allowing users to give constructive criticism that was backed up by evidence.
!Next steps
• Hand over to a graphic designer to polish the interface.
• Explore commercialisation. !
Audio
User studies
Video
Digital ink annotation
research | concept development | prototyping | solutionCase study#1 social music annotation 7 of 25
!Client Imperial College London
Brief To explore new ways of teaching anatomy to medical students. !Team This project was part of my PhD thesis which I completed in Lord Darzi’s Department of Surgical Technology. I worked closely with anatomy teachers, surgeons and medical students. !My role I was a research fellow who led the project under the guidance of my PhD supervisors and senior academic staff. This project taught me fundamental principles of user centred software development such as iteration & rapid prototyping.
!Deliverables • 3D anatomy tutorial using a new interaction design pattern. • PhD thesis and academic papers.
Case study #2 digital anatomy
User needs Medical students find it hard to mentally translate between diagrams in textbooks & real humans. !Static media is weak at demonstrating functional relationships between connected systems in the body. !
Approach • Ethnographic studies to understand
user needs in context. • User tests of existing interaction
design patterns. • User centred development of a new
design pattern. !
Challenges • Creat ing an in te rac t ion tha t
demonstrates complex information about form & function.
• Ensuring the medical accuracy of content.
!
Interview doctors & students Observe teaching
Map existing interaction patterns
rotate strip zoom
research | concept development | prototyping | solutionCase study#2 digital anatomy 9 of 25
Visual explanations of injury by doctors Wireframes
Interaction tests Mock-ups
research | concept development | prototyping | solutionCase study#2 digital anatomy 10 of 25
3D prototypes Linked 2D / 3D prototypes
Eye tracking study Haptic interaction study
research | concept development | prototyping | solutionCase study#2 digital anatomy 11 of 25
Solution A new interaction design pattern that uses ‘dynamic l ink ing ’ to show connections between nerves & muscles. !This design was used in an anatomy tutorial and a haptic simulator. !
Outcomes • A randomised trial (n=50), two focus
groups (n=14) & a survey (n=50). • Dynamic linking had a very high
perceived value, • But it ‘over-automated’ learning by
providing too much help. !!
Next steps • Developer currently converting
framework to Unity / mobile. • Investigate dynamic linking further.
Randomised trial & focus group
Anatomy tutorial
research | concept development | prototyping | solutionCase study#2 digital anatomy 12 of 25
!Client National Health Service
Brief To test the feasibility of using consumer Virtual Reality to train stroke clinicians.
!Team I led the project under the guidance of a stroke consultant. A senior developer created a behavioural model for a virtual patient. A researcher had previously created user personas and user journeys for a stroke patient being admitted into hospital. We built upon this work. !My role I worked with the stroke consultant to translate a neurological stroke assessment into user interactions & virtual patient responses. A senior developer designed the behavioural architecture that controlled the user input / virtual patient responses. !Deliverables • A proof of concept demo. • A paper presented at The 15th International Conference on Intelligent Virtual Agents, Delft 2015
Case study #3 virtual stroke patient
User needs Stroke t ra inees lack ‘hands- on’ experience diagnosing strokes. !It is unsafe to practise upon real patients & videos don’t allow active exploration.
Approach
• Participatory design with a stroke doctor.
• Map a neurological test onto user interactions & patient responses.
• User tests with low & high fidelity prototypes.
Challenges • Visually explaining the effects of brain
damage upon social behaviour. • Ensuring medical accuracy. • Implementing complex interactions:
speech, eye contact, gestures. !
Study existing virtual patients
Plan virtual patient behaviour
Analyse neurological test
Map test onto user interactions
research | concept development | prototyping | solutionCase study#3 virtual stroke patient 14 of 25
Visual explanations of brain damage Speech interaction tests
Interactive storyboards
research | concept development | prototyping | solutionCase study#3 virtual stroke patient 15 of 25
Static prototypes Animated prototypes
User study Navigation prototypes
research | concept development | prototyping | solutionCase study#3 virtual stroke patient 16 of 25
Solution A virtual patient that responds to a user’s speech, hand gestures & gaze. !Demonstrates the feasibility of using consumer VR devices for stroke training, Although hand tracking is currently unreliable. !
Technology Gesture: Leap VR Speech: Google’s speech API Gaze: Oculus Rift DK2 Engine: Unity3D Mocap: Faceshift, OptiTrack
Next steps Validation tests: • Can an expert assess a user ’s
performance using the software ? • D o e s t h e s o f t w a r e i m p r o v e
performance on a real procedure ?
Speech interaction Linked internal / external
Hand gesture interaction
research | concept development | prototyping | solutionCase study#3 virtual stroke patient 17 of 25
!Client Ministry of Defence
Brief To develop a proof of concept VR simulator to train medics on the battlefield.
!Team My company (BespokeVR) won a contract from the Ministry of Defence to test the feasibility of using consumer grade VR for team training. I worked in collaboration with Plextek Ltd who were responsible for project management and military liaison. !My role I developed the software under the guidance of trauma consultants & surgeons.
!Deliverables • A proof of concept multiplayer demo. • Technical report describing & evaluating the system. !
Case study #4 emergency medicine
User needs Medics need to learn teamwork and decision-making skills in stressful, under-fire scenarios. !Tra inee med ic s l ack hands on experience in hostile environments.
Approach
• Participatory design with a trauma consultant.
• Build multi-player immersive VR simulation.
• User-tests with low & high fidelity prototypes.
Challenges • Recreating a hostile battlefield
environment. • Getting multiplayer interaction and a
remote control patient working together.
!
Study advanced trauma courses
Model virtual patient behaviour
Research existing simulations
Model treatment of leg injuries
research | concept development | prototyping | solutionCase study#4 emergency medicine 19 of 25
Reactive eyes
Realistic airways !Voice
Trachea with anatomical landmarks !Carotid pulse
Pneumothorax decompression !Independent chest excursion !Breath Sounds !Heart sounds
Chest tube
ECG !Chest compress !Pacing
IV arm
Multiplayer interaction tests Static 3D mockups
Model injured anatomyEnvironment tests
medic
patrol leader
simulation controller
research | concept development | prototyping | solutionCase study#4 emergency medicine 20 of 25
Navigation prototypes Low fidelity wounded soldier
User-test (Minister of State for Defence) Low fidelity controller
research | concept development | prototyping | solutionCase study#4 emergency medicine 21 of 25
Solution Medics use VR to practise treating a wounded soldier on a battlefield. !Multiplayer simulation with ‘Wizard of Oz’ control of a virtual patient. !!
Technology Network: 2 clients, 1 server Interaction: Razor Hydra, Rift Dk1 Engine: Unity3D !
Outcomes The system was evaluated by three experts who confirmed the face validity & content validity of the system.
Next steps E x p l o r i n g o p t i o n s f o r f u r t h e r development. !
Virtual patient
Usability test
Remote control interface
Content validity study
research | concept development | prototyping | solutionCase study#4 emergency medicine 22 of 25
!Client This is a ‘sandbox’ project to research & develop new techniques for my company (BespokeVR). !Brief To create interactive visuals that learn to respond to the free-form movements of a dancer. !!Team Myself and Dr Marco Gillies (senior lecturer at Goldsmith University). !My role I did the interaction design and visuals. Dr Gillies implemented the Interactive Machine Learning algorithms that make the
visuals learn to interact with a dancer.
Deliverables • Kinect version demo’d at the International Workshop on Movement & Computing Paris, 2014 • Prototypes with Oculus Rift DK2 & MS Kinect
Case study #5 embodied dance
Approach To make music visuals learn to respond to the free-form movements of a dancer. !This is an ongoing side project. It acts as a sandbox to test new techniques & technologies.
Challenge To create a sense of embodiment: the feeling that you are the music. !
Technology Body tracking Kinect Hand tracking Leap VR VR headset Oculus Rift DK2 We used Interactive Machine Learning (IML) to train the visuals.
Interim results
• Nearly there - but tech needs to be more mobile with better tracking.
• Presented a paper at MOCO 2014 !! Map visuals onto a dancer
Research sound visualisation: ‘Seeing Sound’ conference
concept development | prototypingCase study#4 embodied dance 24 of 25