dr. brian duffy [email protected]
DESCRIPTION
Dr. Brian Duffy [email protected]. A Flexible Framework for All. Socially engage each other on an equal footing by encouraging and facilitating minority groups to seamlessly integrate with everyone through universal design and a modular peripherals approach. The Multimodal Approach - PowerPoint PPT PresentationTRANSCRIPT
![Page 2: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/2.jpg)
© 2003 Gina Joue & Brian Duffy
A Flexible Framework for All
Socially engage each other on an equal footing by encouraging and facilitating minority groups to seamlessly integrate with everyone through universal design and a modular peripherals approach
The Multimodal Approach
Flexible integration between sensor modalities:
Fusion
Flexible integration of actuator modalities:
Fission
![Page 3: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/3.jpg)
© 2003 Gina Joue & Brian Duffy
Multimodal Fusion
TypingHandwriting
Pushing and clickingGloved hand gestures
Speaking
Body movementHead movement
Free hand gesturesFacial expression
Eye movement
Hand pressure
Muscle activity
Body stress levels
Brain activity
Position / Motion
Audio
Video
Tactile / Force
Skin Conductance
Neural
Voltage
keyboardtabletmouseglove
microphone
camera
haptic device
EEG
EMG
GSR
![Page 4: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/4.jpg)
© 2003 Gina Joue & Brian Duffy
Plug′n PLAY
Flexible Hardware ::
Social Interfaces ::
Play with Reality ::
Active Chair
Robotics
Virtual &Augmented
Reality
![Page 5: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/5.jpg)
© 2003 Gina Joue & Brian Duffy
The Active Chair :: Design
Air System3 pneumatic cylinders Variable air speed controllersPressure controllersSolenoid valves
Control SystemUSB system using a PIC controller (PIC16F87x) with Serial Communications Interface in full duplex mode (asynchronous) & a USB interface circuit
DesignHolds the occupant in a minimal-stress posture between the neutral body posture of zero gravitation and a normal posture of relaxation on a flat, level surface (based on NASA experimentation).
![Page 6: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/6.jpg)
© 2003 Gina Joue & Brian Duffy
The Active Chair :: Control
![Page 7: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/7.jpg)
© 2003 Gina Joue & Brian Duffy
The Active Chair :: Vicarious Adrenaline
Can you really know how I feel?
… where the OBSERVER
becomes the PARTICIPANT
OBJECTIVE: the transference of adrenaline-based motor sport experiences to an observer through a complex inter-modal interaction.
![Page 8: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/8.jpg)
© 2003 Gina Joue & Brian Duffy
The Active Chair :: Vicarious Adrenaline
“I used the term ‘rodeo-chair’. It is embedded in therapeutic connotations that are attached to the medical viewpoint of disability. However, the haptic
chair uses a reality that involves a playful atmosphere. This maze allows the mind to be in tune with an exterior force. It can be set in a virtual
environment or ultimately in a social context. If the body is paralysed, it does not mean it has to curl up. It needs exercise.
The mind is in the driving seat.” JAMES BROSNAN
![Page 9: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/9.jpg)
© 2003 Gina Joue & Brian Duffy
Immersive Play for All :: TRUST
Installed at the KK Women & Children’s Hospital in Singapore in 2005
![Page 10: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/10.jpg)
© 2003 Gina Joue & Brian Duffy
Social Interfaces :: JoeRobot
OBJECTIVE: To develop a robust modular robot for explicit social interaction between a robot and one or more other robots, virtual avatars or humans
The ideal is a balance between developing a system that can realistically meet the expectations of the people it interacts with while still embracing its machine-like qualities that make it ultimately useful
![Page 11: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/11.jpg)
© 2003 Gina Joue & Brian Duffy
Social Interfaces :: JoeRobot
Articulation When it moves it comes “alive”
Perception Vision: Colour tracking, feature recognitionLocalisationBiosensors
Communication Speech recognition & synthesisGestures
ControlModular code structureBehaviour librariesVision algorithmsCommunication protocolsSoftware interfaces (presentation) Embedded systems (e.g. speech)
![Page 12: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/12.jpg)
© 2003 Gina Joue & Brian Duffy
Play with Reality :: Seamless Worlds
Seamless integration of realities
Judicious synthesis between the real & the virtual
Manage how Virtual & Augmented Reality can become a powerful medium in our social play
fuse the REALGAME
![Page 13: Dr. Brian Duffy brd@media.mit](https://reader035.vdocument.in/reader035/viewer/2022062516/56812b1f550346895d8f1a96/html5/thumbnails/13.jpg)
© 2003 Gina Joue & Brian Duffy
Make it real …
A strong need to develop flexible modular designs to promote real interaction regardless of abilities, age, or situation.
SENSE – FEEL –EXPERIENCE
The integration of multi-sensory stimulation creates a highly immersive environment that helps facilitate motor control rehabilitation.
“… allows the mind to be in tune with an exterior force.”
James Brosnan