dissemination and evaluation
DESCRIPTION
Strategy for evaluating and communicate the results of the AtGentive projectTRANSCRIPT
![Page 1: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/1.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Dissemination and Evaluation
Claudia Roda, Damien Clauzel, Joona Laukkanen
![Page 2: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/2.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Dissemination: Atgentive book
• Latest version has been distributed– Need feedback on proposal (5 pages to
read!)– Participation:
• I am listing as committed: Cantoche, OBU, Ontdeknet, and AUP
• Informed but not committed: INSEAD, UTA (I have no description of contribution)
• I had no feedback from: CELN, STC • Could you please check your bios
![Page 3: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/3.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Dissemination: Atgentive Book
• Timeline:– First version of each chapter: 15 February
2008– Feedback from review: 15 March 2008– Final formatted version of each chapter 15
April 2008– Publication: ??
![Page 4: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/4.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Evaluation
• Objective: enable the assessment of the value of the conceptual framework and reasoning module in contexts that are not represented in the pilots
• Should this go under strategic evaluation?
![Page 5: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/5.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Evaluation
• Conceptual Framework - Demonstrate that the conceptual framework may hold in the context of very different implementations and applications.
• Reasoning module - New version of the test centre: – Visualize the reasoning of the reasoning module in
applications such as Ontdeknet– Show how the reasoning module can be used also in less
structured, non-task-based applications, and in the context of several (rather than one) applications.
![Page 6: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/6.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Evaluation: Conceptual Framework
• We look at the context restoration issue. • Virtual multi-desktop interface as an implementation of
context-restoration facility • We hope to demonstrate that context restore will
enhance both performance and user satisfaction• Hypotheses (Virtual multi-desktop interface vs Control):
– Increased perception of productivity and pleasure– Tasks are completed faster and better– Tasks are resumed faster and more easily– Fewer interface management actions and fewer mistakes on
interface management actions
![Page 7: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/7.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Evaluation: Conceptual Framework
• Experiment:– Subjects work for 20 minutes on three fully described tasks
(spreadsheet based, translation, video). – They are interrupted regularly by requests to work at a
different task, they are required to satisfy these requests– Control group (standard window interface), experimental
group (virtual multi-desktop)• Measures:
– Task completion level– Task quality level (# of errors)– Task resumption times– Reported evaluation
![Page 8: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/8.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Task 1: Spreadsheet
![Page 9: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/9.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Task 2: Translation
![Page 10: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/10.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Task 3: Video
![Page 11: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/11.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Interruption
![Page 12: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/12.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Resume
![Page 13: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/13.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Evaluation: Reasoning Module
• Reasoning module - New version of the test centre: – Visualize the reasoning of the reasoning
module in applications such as Ontdeknet– Show how the reasoning module can be
used also in less structured, non-task-based applications, and in the context of several (rather than one) applications.
![Page 14: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/14.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
The test center
• Tool for experimenting with the RM• The test center was not very intuitive• Desire to experiment with the future
vision of the RM (still working on)
![Page 15: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/15.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
![Page 16: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/16.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Desktop
![Page 17: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/17.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Desktop
• Minimal applications on a desktop– application provides task
model, sends events– interact with the application
instead of creating events
![Page 18: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/18.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Desktop
• Interventions
![Page 19: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/19.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
User and task properties editable
![Page 20: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/20.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Reasoning module activity
![Page 21: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/21.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Utilities for creating environment events: New information available, idle input
![Page 22: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/22.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
EXAMPLE
![Page 23: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/23.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
user has finished intro and is notified about next task (learning goals)..
![Page 24: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/24.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
user starts learning goals (interventions disappear)..
![Page 25: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/25.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
user starts the wrong task, gets interventions suggesting correct subtask. Math task is also suggested as possible alternative task ..
![Page 26: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/26.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
The applications
• Pretty much like real applications– Started from desktop icons..
• Known task model• For now just one application
(AtGentSchool)• Displays interventions allowing..
– to start suggested task– In the future to provide feedback on the
suggestions
![Page 27: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/27.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
The applications
• Pretty much like real applications– Started from desktop icons..
• Known task model• For now just one application
(AtGentSchool)• Displays interventions allowing..
– to start suggested task– In the future to provide feedback on the
suggestions
![Page 28: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/28.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
The future vision
• Several applications– Some applications provide their task model– Applications may be very different from
what we have seen so far• Task management side bar
![Page 29: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/29.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Future test center
![Page 30: Dissemination and Evaluation](https://reader035.vdocument.in/reader035/viewer/2022062319/55646601d8b42a951e8b52ff/html5/thumbnails/30.jpg)
AtGentive General Meeting, Amsterdam 22-23 Oct 2007
Future test center