![Page 1: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/1.jpg)
Panagiotis Ritsos [email protected]
Project IVY Meeting – VMG Seminars – September 2011
The IVY project has been funded with support from the European Commission. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.
![Page 2: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/2.jpg)
THE PROJECT IVY GROUP
University of Surrey (UK)
Uniwersystet im. Adama Mickiewicza (Poland)
University of Cyprus (Cyprus)
Steinbeis GmbH & Co. KG für Tech-transfer (Germany)
University of Bangor (UK - Wales)
Eberhard Karls Universität Tübingen (Germany)
Bar Ilan University (Israel)
![Page 3: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/3.jpg)
The rise of migration and multilingualism in Europe requires professional interpreters in business, legal, medical and many other settings.
Future interpreters need to master an ever broadening range of interpreting skills and scenarios – training for which is often difficult to achieve with traditional teaching methods.
Project IVY employs 3D virtual environment technology to create an virtual educational space that supports the acquisition and application of skills required in interpreter-mediated communication.
Project IVY Partners provide user interaction and interpreter resources – audio and video material from previous video conferencing research – ‘BACKBONE’.
IVY Virtual Environment – Scope
![Page 4: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/4.jpg)
A dedicated 3D virtual environment for interpreting students and future clients of interpreters
A range of virtual interpreting scenarios (e.g. ‘business meeting’) that can be run in different working modes: simulation, activity/exercise mode, exploration and live interaction mode;
Multilingual video-/audio-based content for interpreting scenarios, by adapting and supplementing the corpora from the LLP project BACKBONE (in
EN, DE, ES, FR, PL, TR) and adding new corpora (GR and AR, EL or RU).
Two sets of pedagogical material for interpreter students and (future) ‘clients’, e.g. awareness-raising and interpreting exercises, and explanations.
Project IVY in a nutshell
![Page 5: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/5.jpg)
To present the technical requirements that have to be met from the IVY Virtual Environment (IVY-VE)
To present the strategic decisions, resulting design and implementation progress to date, towards the creation of a prototype
To provide an overview of the main features of our prototype
To allow for discussion on design aspects which require partner feedback
Presentation Outline
![Page 6: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/6.jpg)
To provide an intuitive, easy to use interface to a Virtual World for interpreting training and simulation
To allow access through that Virtual World to existing audio material of interpreting exercises and immerse participants in various scenarios
To allow easy dialogue management – addition, modification, deletion of existing dialogue scripts
To enable limited dialogue synthesis, resulting in different language combinations of existing dialogue scripts.
IVY Virtual Environment – Requirements
![Page 7: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/7.jpg)
Dialogues are assembled from mp3 files, corresponding to the scene participants (denoted A & B) speech
They do not have to be of a particular order – i.e. dialogues can follow a sequence such as `ABBAAABAB’
Audio sequences form a dialogue script…
…which is accompanied from textual information – title, keywords (domains), description and scene
In the future we will be able to replace the audio files corresponding to a language in a script with other ones, compliant with the latter…
…allowing limited dialogue synthesis
IVY Virtual Environment – Technical Aspects - I
![Page 8: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/8.jpg)
VE Visitors need to be able to roam freely into the areas of Project IVY, without any obtrusive GUI elements or VW noticeboards
Ideally users need to be able to `jump’ from scenario to scenario without needing to return to a `reception’ area
Audio controls need to include stop, rewind, fast-forward etc.
Main challenge is to `match’ audio events to environment events and/or avatar expressions and gestures
Dialogue management does not need to be performed from within the VE
IVY Virtual Environment – Technical Aspects - II
![Page 9: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/9.jpg)
![Page 10: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/10.jpg)
Second Life was chosen as the VW for our first prototype
Exploration of alternatives, such as OpenSim may follow in the future
Second Life compared to alternatives (OpenSim, ActiveWorls etc) offers:
Large community, offering various add-ons, plugins and examples of customisations
A platform for social interaction and education, used by numerous institutions, colleges, universities – thus increasing chances of exposure
Existing scenes build in Bangor Island, allowing faster scene development
Does not require that you run the VW yourself, but can access public servers
IVY Virtual Environment – Second Life
![Page 11: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/11.jpg)
Various Scenarios - Classroom, meeting room, shops, outdoor, community centre etc.
IVY Virtual Environment – Scenarios
![Page 12: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/12.jpg)
Various Scenarios - Classroom, meeting room, shops, outdoor, community centre etc.
IVY Virtual Environment – Scenarios
![Page 13: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/13.jpg)
Various Scenarios - Classroom, meeting room, shops, outdoor, community centre etc.
IVY Virtual Environment – Scenarios
![Page 14: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/14.jpg)
Interpreter students at Surrey University have already trialled basic meetings.
IVY Virtual Environment – Scenarios
![Page 15: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/15.jpg)
Courtroom settings are also being developed – multi purpose training scenarios – interpreter and user of interpreters.
IVY Virtual Environment – Scenarios
![Page 16: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/16.jpg)
![Page 17: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/17.jpg)
IVY Virtual Environment – Architecture
![Page 18: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/18.jpg)
Menu is a Heads-Up display (HUD) with same aesthetics to the SL user interface
IVY Virtual Environment – Menu
![Page 19: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/19.jpg)
Simple, Intuitive menu for selecting form, language pair and dialogue by title
Information Pane and Launcher
Essentially a Heads-up display object
Individual per user
Allows teleportation from scenario to scenario without requiring returning to a reception area
IVY Virtual Environment – Dialogue HUD
![Page 20: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/20.jpg)
Once a dialogue is selected a player and a teleportation confirmation window appear
IVY Virtual Environment – Player - Teleportation
![Page 21: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/21.jpg)
Simple audio player with dialogue specific information
Returns user to dialogue selection menu
IVY Virtual Environment – Player Detail
![Page 22: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/22.jpg)
Integrate IVY HUD functionality with the virtual world by:
Controlling participant-avatars when speaking through a telnet-based `bot’ server
Each bot performs basic gestures for as long as `he’ or `she’ talks
Explore the possibility of using directional sound
Integrate tighter with SL GUI elements such as flashing voice indicators
IVY Virtual Environment – Next Steps
![Page 23: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/23.jpg)
Where is the research element? Is it merely a development project, mixing and matching existing techniques?
The research element comes from the `service’ and not the technology…
Performance comparison to traditional methods used by interpreters
Investigation on how the sense of immersion enhances the user experience of IVY-VE participants
Definition of metrics for the above
IVY Virtual Environment – Research Element
![Page 24: Project IVY - Virtual Environment Development](https://reader031.vdocument.in/reader031/viewer/2022020122/547dc67db4af9fb4158b5494/html5/thumbnails/24.jpg)
Thank you!