![Page 1: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/1.jpg)
(1)
Makahiki: An Extensible Open-Source
Platform for Energy Challenges
George LeeCollaborative Software Development
LaboratoryInformation and Computer Sciences
University of Hawaii
![Page 2: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/2.jpg)
(2)
Energy Competitions
What is an energy competition?• Groups try to reduce energy usage as much as possible• Can involve participation in events
Why have competitions?• Save money on energy costs• Opportunity to educate people about saving energy
Colleges have done this for over 20 years
![Page 3: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/3.jpg)
(3)
Dorm Energy CompetitionsHodge, Chelsea, Dorm Energy Competitions, Presented at the Behavior, Energy, and Climate Change Conference, November 2010
Over 160 colleges in the US planned to have one in 2010-2011 academic year
Reduce energy use during the competition by 9% on average
![Page 4: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/4.jpg)
(4)
Harvard Green Cup
![Page 5: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/5.jpg)
(5)
Bowdoin
![Page 6: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/6.jpg)
(6)
Energy Challenge Competitions focus on energy reduction
Energy literacy and sustained behavior change
Platform for creating energy challenges and researching behavior change
![Page 7: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/7.jpg)
(7)
MakahikiEnergy competitions• Supports near-real time energy feedback• Event and activity tracking• Personalized feedback
Energy challenge• Pedagogical approach• Social network integration• A serious game
![Page 8: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/8.jpg)
(8)
Serious Games Zyda, Michael, From Visual Stimulation to Virtual Reality to Games, IEEE Computer, September 2005
Definition: “a mental contest, played with a computer in accordance with specific rules that uses entertainment to further government or corporate training, education, health, public policy, and strategic communication objectives.”
![Page 9: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/9.jpg)
(9)
Serious Games as MotivationMcGonigal, Jane, Reality is Broken, Penguin Press, January, 2011
Created several “Augmented Reality Games” that involve real world tasks
Games provide a sense of accomplishment that rewards effort
![Page 10: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/10.jpg)
(10)
Serious Games and Productivity
Reeves, Byron, Total Engagement, Harvard Business School Press, 2009
People spend hours on online games
How can we keep people in the workplace engaged?
Harnessing techniques from games can make businesses more competitive
![Page 11: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/11.jpg)
(11)
Serious Games and Learning Murphy, Curtiss, Why Games Work and the Science of Learning, http://www. goodgamesbydesign.com/Files/WhyGamesWork_TheScienceOfLearning_ CMurphy_2011.pdf
Examined how the Navy used a game to train recruits
Saw a 50-80% increase in individual performance after 1 hour of playing
![Page 12: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/12.jpg)
(12)
“Smart Grid” GameWant to educate users on how to be “green”• Present tasks that help them be more aware• Declare commitments• Hold events like movie nights
Reward users in the form of points
A “Serious Game” based around energy literacy
![Page 13: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/13.jpg)
(13)
Integration with WattDepot
Dorm MetersWattDepotSensors
WattDepotServer Makahiki
WattDepotCloud Cache
![Page 14: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/14.jpg)
(14)
Near Real-Time Feedback Peterson et al, Dormitory residents reduce electricity consumption when exposed to real-time visual feedback and incentives, International Journal of Sustainability in Higher Education, 8(1):16–33, 2007.
Experiment at Oberlin College in 2005
Users with near real-time feedback outperformed those with “low-resolution” feedback
![Page 15: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/15.jpg)
(15)
Go Low
![Page 16: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/16.jpg)
(16)
Evaluating Makahiki Evaluation of the alpha version
Three phases of evaluation:• Mockup Evaluation• In-lab Evaluation• Beta evaluation
Actual competition in October 2011
![Page 17: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/17.jpg)
(17)
Goals of the evaluation How can we provide a system that supports researchers interested in users participating in an energy competition?
How can we effectively test the website and the overall design of the competition?
![Page 18: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/18.jpg)
(18)
Mockup Evaluation Created mockups using Balsamiq Mockups
Brought in friends and family to go through scenarios
Scenarios:• Initial set up• Getting points• Viewing energy use• Viewing prizes
![Page 19: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/19.jpg)
(19)
Mockup Landing Page
![Page 20: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/20.jpg)
(20)
Landing Page
![Page 21: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/21.jpg)
(21)
Mockup Get Nutz Page
![Page 22: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/22.jpg)
(22)
Get Nutz Page
![Page 23: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/23.jpg)
(23)
Mockup Evaluation Results Major changes:• Energy Goal Voting removed• Simplification of the “Get Nutz” page• Altruistic raffle prizes
Points were a significant motivator
On the right track
![Page 24: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/24.jpg)
(24)
Discount Usability Engineering
Nielsen, Jakob, Guerrilla HCI: Using discount usability engineering to penetrate the intimidation barrier. Cost-justifying Usability, pages 245–272, 1994.
Think aloud protocol, no need to capture video
Scenarios?
![Page 25: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/25.jpg)
(25)
Hutchworld Cheng et al, Hutchworld: Lessons learned. Virtual Worlds, 1834/2000:12–23, 2000.
Developed by Microsoft to help cancer patients
Three dimensional interface
5 minute test period before the evaluation starts
![Page 26: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/26.jpg)
(26)
Onboarding Users are unlikely to know how the system works beforehand
System has to guide the user and expose them to the various mechanics
The quest bar:• List of things users could do on the site• If completed, the quest does not appear• Not worth any points
![Page 27: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/27.jpg)
(27)
Onboarding Evaluation Representative users
Procedure:• Think aloud protocol• Captured computer screen using software• Let the subject use the site for 45 minutes• Post-evaluation questions• $10 bonus and Gift Certificate quests
Two rounds of onboarding evaluations
![Page 28: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/28.jpg)
(28)
April 2011 Results 5 first-year students from the Hale Lehua tower
Issues found:• Many users missed the quest bar• Rejected admin notifications were missed• Individuals frequently guessed at answers• Initial confusion on Go Low and Get Nutz
Positive feedback:• Found it addictive and would recommend it• One wished they could participate
![Page 29: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/29.jpg)
(29)
July 2011 Results 5 first-year students from the Hale Lehua tower
Many improvements:• Time through setup improved• More questions answered correctly• More quests attempted and completed
Issues:• Quests were not set up properly• Timing of rejection notification
![Page 30: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/30.jpg)
(30)
Beta Evaluation Evaluate the game over a period of time
Evaluation of logging system
Beta evaluation:• Brought in friends and family• Held a 1 week mini competition• Surveyed individuals after the competition• Also analyzed database and logs.
![Page 31: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/31.jpg)
(31)
Beta Evaluation Results 4 teams with 5 subjects each
18 of the subjects logged in at least once
Results:• Averaged 915 points• Completed 16.2 activities
Issues:• Must detect Internet Explorer• Restrict the Daily Energy Goal Game’s points
![Page 32: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/32.jpg)
(32)
2011 Kukui Cup 418 people logged in over the course of the competition
850 hours were spent on the site
42 individuals completed the round 3 survey• 25 found the system easy to use•Understood how to play the game
![Page 33: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/33.jpg)
(33)
2011 Kukui Cup Survey Results
![Page 34: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/34.jpg)
(34)
2011 Kukui Cup Survey Results
![Page 35: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/35.jpg)
(35)
2011 Kukui Cup Survey Results
![Page 36: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/36.jpg)
(36)
ContributionsMakahiki is an open source framework for energy competitions
Makahiki acts as a platform for researchers who are interested in user behavior during energy competitions
To evaluate the system, we created a methodology for testing Serious Games.
![Page 37: (1) Makahiki: An Extensible Open-Source Platform for Energy Challenges George Lee Collaborative Software Development Laboratory Information and Computer](https://reader036.vdocument.in/reader036/viewer/2022070412/5697bf841a28abf838c86c04/html5/thumbnails/37.jpg)
(37)
Questions? Kukui Cup Website: http://sites.google.com/site/kukuicup
Makahiki project website: http://github.com/csdl/makahiki/