job training cirriculum: january 2013 update

Learning Systems at Quest Alliance January 14, 2013 Eva Miller, Design Fellow Design Impact

© Design Impact 2012

Quest Alliance

Creates programs and services to prepare youth for a better future: Quest is a non-governmental organization that designs job training solutions for young people who have been failed by India’s educational system. English language and vocational training matter, but the true heart of Quest’s approach is life skills and work readiness, with a growing emphasis on active learning methods. Students understand themselves better, grow in confidence, and see work and education as a path to greater goals for their lives. Works with partners to deliver programs and services: Quest Alliance does not want to be in the service delivery business and creates partnerships with industry, education, and vocational organizations to deliver their programs. They also train facilitators to assure Quest’s educational philosophy becomes reality in the classroom.

Quest Alliance offices in Bangalore, India © Design Impact 2013

Create a system for learning that scales: Design structured information that works across all programs, from a single piece of active learning to an online environment that supports learners, facilitators, partners, and Quest staff

The project challenge © Design Impact 2013

ONE: Diary/Field Study to better understand the daily lives of our initial audience for a learning management system: facilitators

Milestones from November 3 to January 15

TWO: Create a content framework that encompasses Quest’s educational philosophy and clarifies active learning

THREE: Begin to define the features of the learning management system we’ll need


Identified…

Facilitators who match the desired audience from earlier work with staff: “eager/overwhelmed teachers”

Three from the Equip Youth vocational education program and two from Skills to Succeed program

Screened them and collected background information

Created a paper diary for participants

Asked them to keep the diary for 7 days and make 3 entries per day

ONE: Diary/Field Study © Design Impact 2013

Waited…

We began in late October, but many participants had delays because of festivals, holidays, or planned breaks between program batches

Some do not teach every day, so it took 2-3 weeks to get enough entries from them

Facilitators found it hard to schedule a time for me to visit their classroom, review the diaries, and walk through a typical day for them (field study).

This took an extra 6-7 weeks to do


Emerging themes…

Time. Some neglect home life, many come home late, too tired to prepare for the next day. Travel is often an issue and can take as much as 3-4 extra hours every day. Most report preparing over 2 hours a day.

Product opportunities: structured materials follow a pattern and take less time to “decode.” In active learning, most of the “content” is provided by the students, which lessens the need to commit a lesson to memory


Emerging themes…

Keeping students engaged. Students don’t come or come late, and many can’t focus long on the lessons

Product opportunities: connect curriculum more firmly to work preparedness, use more active learning methods, and run student design sessions to improve curriculum and see what is most relevant to their lives.


Emerging themes…

Student-Facilitator bond. Students crave the facilitator’s attention and the relaxed classroom atmosphere.

Product opportunities: create continuous feedback through the system to give students attention that promotes performance. Support communication through social tools that can use mobile technology, like SMS messaging, but collect conversations in the system.


Emerging themes…

Support. Some wonder if their organizations know or care what they are doing. Co-trainers lean heavily on each other.

Product opportunities: create online community that gives facilitators more colleagues with whom to share issues and ideas. Allow peers to review each other’s work. Make key leaders at the partner organization part of this community or create more continuous reporting to put success stories in front of them


Emerging themes…

Perfectionism and passion. All spend several hours every day to be well-prepared and take it hard when class goes poorly. All expressed their love for this work and how rewarding it can be.

Product opportunities: we can reward passion by calling out extra effort and letting other facilitators see this appreciation publically. We can help reduce the pressure to be perfect by minimizing process-heavy curriculum and using more active learning, instead.


Most of the field study interviews with facilitators are transcribed. All need to be analyzed for patterns and insights that can be converted into a “Day in the Life” facilitator journey. This is an experience map that calls out pain points and opportunities for our product.

There is still one more diary and field study to collect—the last facilitator finally started to make entries. I also feel we should have another male facilitator complete a diary and field study to balance out this group. But there may not be time to do this.

© Design Impact 2013 ONE: Diary/Field Study

• Festival season, holidays, weddings, and end-of-year vacations often meant I could not make progress because people were away for extended periods. I did not foresee this and could not plan accordingly. I had to move ahead with other design activities while waiting for diaries and appointments for the field study visits

• The loss of a month means trimming the project plan. This will remove the iterations of internal and external workshops to perfect our content model. These can continue, but the design work will have to move forward with initial findings only.

• Facilitators seem ripe for burnout. Most in our study are new to this field, but already seem over-burdened, though some report they are getting more comfortable in this role. All have other responsibilities at their organizations besides delivering Quest classes.

Concerns at this point © Design Impact 2013

Content Framework Design Sessions Giving shape to Quest’s educational philosophy

TWO: Content framework and active learning © Design Impact 2013

Researching…

How competitors and related organizations and experts define life skills and active learning

Quest’s own life skills, work readiness, English, and retail materials to see what they reveal about its educational goals

This research helped us create a first draft of our content framework: Quest’s educational goals



Defining…

Before our workshop, we revisited the user profiles for whom we are creating our content framework and learning system. We decided to add more detail to our facilitator profiles.

To do this, we used some of the early findings from our diary/field study. We’ll be able to create something richer once this study is complete.

Ritu is a more “traditional” teacher and not our emphasis for this system


Defining…

“Rashmi” is the eager/overwhelmed facilitator who comes to the work with fresh eyes and some idealism. She is our primary focus as a facilitator using the system.


Defining…

We added more detail to our student user profiles, too.

We revisited earlier work from Quicksand for this. When we do a content workshop with students, we’ll refine and enrich these.

“Sahir” is more entrepreneurial and impatient to get started on a business venture of his own. He is not our primary focus as a student using the system


Defining…

“Soma” is a vocational student who has a goal to get a job to help her family but discovers there’s a bigger world out there for her.

She is not our main emphasis as a student using the system.


Defining…

“Santosh” is the main student audience for our system. He arose from our earlier stakeholder workshop to align key Quest staff around what they want from a learning management system.

Staff most want to support the “aimless student” like Santosh, who wants to earn money but doesn’t have a clear goal in life.

Testing…

How staff view the content framework: categories and educational goals

How they would rank various educational goals in importance for our main user

How they would rate these goals in complexity for our main user

How they would categorize these goals themselves


Refining…

We changed the content framework slightly to reflect staff feedback

But we discovered our proposed categories are largely the same as staff. We didn’t reveal our category names and use only colors and images to indicate categories.


Prototyping…

We created a workshop using our new content framework to brainstorm classroom activities

Staff began by experiencing active learning through the “Marshmallow Challenge:” teams have 18 minutes to build the tallest freestanding structure they can from spaghetti, tape, and string


Prototyping…

We demonstrated that active learning has a structure we can follow: plan, do, give feedback, reflect, and apply to the future

Then asked staff to pluck a goal from each category in the content framework and brainstorm as many “active learning” ideas as they could.

Still analyzing this session, but the framework is mostly solid

Staff will need more direction on active learning


• Staff are responsible for various programs and see them as distinctive from others. The idea of a content framework as a single foundation for Quest’s curriculum design and active learning as the structure for delivering it means programs do not matter, except as wrappers for content. This is a big change from the way Quest has developed curriculum in the past.

• Doing only one iteration of activity brainstorming may not be enough. But there is not time for more.

Concerns at this stage © Design Impact 2013

Defining system features Beginning to see how the system will work

THREE: Defining the learning system © Design Impact 2013

Discovering pain points…

Staff wanted to respond to Accenture, a partner who is willing to help launch a learning management system to support Quest and other organizations they work with

To discover the features we wanted most, we worked through pain points with the current Moodle learning management system and talked about what features might help.


Finding solutions…

We wrote a feature list for Accenture that listed everything that addressed our pain points

We included some features from earlier discussions with Accenture

To help make our issues more clear, we included benefits to the business and to our users


She#pulls#up#her#course#page#to#have#a#look.#Selvi’s#class#is#doing#well#in#some#areas,#but#not#others.#

MyQuest

Your+Course+page

Class+progress

Find+ac6vi6es

Goals+and+styles

TOP+3BOTTOM+3

Hi,#Selvi:##Your#account#

Filter#by:

1C10#of#213#ac7

vi7es

See#all...Your+st

udents

##All#unreview

ed#(28)

####Quick#List#(1

0)

######Students#no

t

#########recently#e

valuated

Your+peers

###Not#recent

ly

####evaluated

#(14)

#####By#reque

st#(1)

#

EVALUATE

DISCUSS

Does#anyo

ne#have#

7ps#about

#using#

music#i#class

?#I#

would#like

#to...

I#made#a#few#

energizer

s#that#

really#work

#great...

Add#new

...

On#a#Saturday#a+ernoon,#Selvi#comes#home#and#decides#to#catch#up#on#her#student#evalua7ons

It’s#a#song#about#anger#from#Harita#and#Raj.#She#remembers#this#one!#Selvi#laughs#again#at#how#clever#it#was.

MyQuest

Ac6vity:++ Sing#an#angry#song

Created+by:+ Harita#V,#Raj#KAdded:+ Oct#30,#2012

Comments:# 8#comments#|#Add#comment

Quick+List:++++<+Last+item++|++Next+item+> Hi,#Selvi:##Your#account#

Adding clarity…

We responded to various questions from Accenture with a second version of the feature list

We completed and added some scenarios from earlier work to show how users will interact with the learning system we want


Adding clarity…

We added a few sketches, too.

We wanted to show that we want to include a visual indicator of progress to make it easy to see how well a student, facilitator, class, or center is doing over time. Seeing that story through data is persuasive to partners and motivates students and facilitators


DESIGN ACTIVITIES AND TIMELINE: Learning © Design Impact 2013

Design Activity Roles Time estimate

Review background materials and programs User experience lead: Eva Sept 3-7

Plan and give stakeholder workshop on the learning management product

User experience lead: Eva

Stakeholders: Aakash, Amitav, Abhijeet, Ayeesha, Augustine, Ali, Ashutosh, Jayashree, Nikita

Sept 10-21

Establish main audiences for the product as a user profile for now

User experience lead: Eva Sept 10-21

DONE

DONE

DONE


Write and share “Voice of the Business” findings User experience lead: Eva Sept 24-Oct 1

Select design team User experience lead: Eva Business owner: Aakash

Oct 2-12

Diary/Field Study to understand facilitators’ daily lives User experience lead: Eva Oct 12-Jan 30

Create “day in the life” snapshots of facilitators to guide product design

User experience lead: Eva Jan 10-Jan 30

Establish active learning framework: draft version, based on competitive analysis and audit of existing content

User experience lead: Eva Curriculum designer: Lara Content lead: Ali

Nov 14-Dec 11

Internal workshop to get staff feedback on active learning framework: rank by importance, complexity, and free sort for feedback on our categories. Analyze results and adjust.

User experience lead: Eva Product owners: Nikita and Abhijeet Curriculum designer: Lara Content lead: Ali

Dec 11-20

Internal workshop to generate draft activities: use framework. Create staff workshop activities to clarify active learning and brainstorm activities in art, music, drama, film, games, etc. Analyze results and adjust.

User experience lead: Eva Product owners: Nikita and Abhijeet Content leads: Ali, Leena

Curriculum designer: Lara

Dec 21-Jan 15

Write draft activities User experience lead: Eva Content leads: Ali, Leena


Jan 22-30

External workshop using draft activities with students: lead 4-5 students through activity, then explore what might work better. Explore student challenges and needs. (May need interpreter)

User experience lead: Eva Product owner: Nikita and Abhijeet Content leads: Ali, Leena


Jan 22-30

DONE

DONE

STARTED

STARTED

DESIGN ACTIVITIES AND TIMELINE: Defining © Design Impact 2013

DONE

DONE

STARTED

DESIGN ACTIVITIES AND TIMELINE: Model, test, and build © Design Impact 2013


Create student personas User experience lead: Eva Feb 18-19

Write pilot activities using the best ideas from student workshops


Content leads: Ali, Leena


Feb 18-22

Use pilot activities at selected training centers and observe


Product owner: Nikita

Content leads: Ali, Leena


Technology lead: Venky

Feb 25-Mar 8

Participant survey to get student and facilitator feedback on pilot activities



Mar 1-8

Analyze pilot results User experience lead: Eva Mar 4-11

Adjust activities and finalize content plan: hand off to content team



Content lead: Ali

Mar 12-15

Create initial learning system wireframes, review with team and finalize

User experience lead: Eva Mar 4-22

Create remaining wireframes and interaction design, review with team and finalize


Visual/Interaction designer: Kaleidoscope (TBD)

Mar 25-Apr 1

DESIGN ACTIVITIES AND TIMELINE: Model, test, and build © Design Impact 2013


Paper-based usability testing with facilitators and Quest staff



Product owner: Abhijeet

Content lead: Ali


Apr 2-12

Create clickable prototype User experience lead: Eva


Back-end developer: Azri Solutions

Content lead: Ali

Apr 8-19

Prototype usability testing with facilitators and staff User experience lead: Eva

Product owner: Abhijeet

Content lead: Ali




Apr 22-26

Begin development iterations and create final visual design




Apr 29-May 31

Present development iterations at demos to team User experience lead: Eva



May 6-May 31

Release (alpha) June 3

• Functional specifications are not in the spirit of agile development and any list of necessary features should come from actual designs. But Accenture’s timeframe may differ from ours, and they may not be able to accommodate our needs, once we have the product sketches in more detail.

• If timelines keep sliding, the next place to cut will be product development. We will end with tested system designs but won’t have time to write code. Quest will manage this work without a Design Fellow, so it will be important to make design documents clear for the next team to pick up.

Concerns at this stage © Design Impact 2013

Thanks.

Contact: Eva Miller [email protected] www.d-impact.org


job training cirriculum: january 2013 update

Documents

diaryfield study design

india design impact

design structured information

learning management

learning systems

active learning methods

quest staff

quest alliance offices