Expanding Community: Student Involvement in Quality

Assurance Testing Josh Baron

Director, Academic Technology and eLearning

MARIST COLLEGE• We are NOT a large research university! • Founded 1929 – small complex liberal

arts college• Located in Poughkeepsie, New York, USA• Approximately 5700 students (FTE)• 200 full-time faculty, 500 part-time• Strategic focus on distance learning

QA Overview from 10,000 ft

• QA = Quality Assurance• Process involves:

1. Testing using scripts to identify bugs

2. Fixing those bugs and verifying they were fixed

3. Release new version (

4. Conducting regression testing to test if bug fixes

More at: http://confluence.sakaiproject.org/confluence/x/jQ

Overview of Marist’s Model

• Dedicated three part-time (20-hours per week) CS graduate students to QA for 2.4– Focus was on OSP and Messages & Forums

• Students are paid $8.50 - $9.50 per hour • Students also involved in supporting faculty

who are using Sakai

Preparing for QA

• About 2-3 month before QA began (Jan.):– Students began exploring confluence and Sakai– Meetings with students and project coordinator– Got involved in QA and OSP conference calls

• About 1-2 months before QA began (Feb.):– Conference call with Megan May– Got involved in “preliminary test” – IMPORTANT– Got informal mentoring from OSP leads

Engaging in QA

• Student were dedicated “full-time” (20-hours per week) during formal QA process

• Students engaged with the community often– The were not initially comfortable doing so– Worked to identify bugs vs. requirements– Sometimes helped ID technical issues

• Close coordination was required

Challenges

• Latter test scripts often relied on completing earlier ones

• Students struggled to ID bugs vs. requirements

• Students need coordination and mentoring• Each “tag” release required dropping the

database, focusing us to start over

Benefits to Marist

• Great learning experience for students– Worked within open/community source project– Had to work in as a virtual team

• Focused on areas that were important to us• We contributed!!!• Kept us close in the loop on tool issues• Student were able to train me and my staff

Benefits to Community

• Inexpensive model for adding QA resources• Students may be able to dedicate more

focused attention to QA• Scalable and sustainable model• Creates a pipe for potential developer

resources

Lessons Learned & Recommendations

• Having a “community mentor” is important• Having a local coordinator is important• More automated testing is critical• Encourage students to reach out for help• Could develop global “pass around” model• Great way to learn about system and

prepare for future development work

Sakai.EDU Concept

• Create opportunities for undergraduate and graduate students to engage in work related to the Sakai project for which they receive academic credit.

• Focus on a range of academic and non-academic topics

• Possible “Community Award” for student teams

THANKS!• Megan May – for coordinating • Dawn Eckert – for helping us understand OSP and for

walking us though the test • scripts and how to execute them.• Lynn Ward, Chris Maurer, and the Entire Collab Sakai

Community – for helping resolve • out doubts during the testing process• Chris Maurer, Jim Eng – for quick follow to bugs reported

on JIRA• Tony Camilli, Sheeba Gandhi, Matt Rubins and others – for

helping set up the test • Environment.