motivation and satisfaction for vocational education...

30th

ascilite Conference 2013 Proceedings 67

Motivation and satisfaction for vocational education students using a video annotation tool

Meg Colasante College of Science, Engineering and Health Academic Development Group

RMIT University

Michael Leedham School of Engineering (TAFE)

RMIT University

This paper examines the use of a specific contemporary technology in tertiary education that of a

video annotation tool, MAT, in four vocational learning cohorts. These students, enrolled in

property services and audiovisual technology courses, analysed representations of workplace

issues in video. These videos included industry interviews, acted examples, and student-performed

role-plays. Student analysis was evidenced—and shared with peers and/or teachers—via

electronic annotations anchored to key points within the video media. The findings in this paper

focus on the motivation and satisfaction of these vocational students in their video annotation

activities using Bekele’s (2010) conceptual framework of factors attributing to success in online

learning. Overall, students’ perceptions of this electronic learning method tended to indicate

satisfaction across a range of factors, with clues for improvements in tool and/or learning design

support, and that the innovation is worthy of ongoing trial and refining from lessons learnt.

Keywords: video annotation, vocational education, property services, audiovisual technology

Introduction

A multiple-case study within an Australian university saw a media (video) annotation tool (MAT) introduced

across a range of disciplines and tertiary sectors, including four cases from the vocational sector. The four

vocational cases comprised three different property services cohorts and one audiovisual technology cohort.

MAT is a learning tool that allows upload and granular annotation of video. As could be expected, the videos

analysed in MAT had vocational focus, such as interviews with industry representatives, acted workplace case

examples, or student role-play of work roles. See Table 1 for a summary.

Table 1: The four vocational MAT cohorts

Code Level Case cohort Subject theme Video/s for analysis in MAT

AV Diploma Audiovisual Technology Quality service 2 x commercially acted workplace customer experiences (examples)

PD Diploma Property Services (Asset and Facilities Management)

Customer service and leadership

1 x interview; senior industry representative (large company)

PT Certificate IV Property Services - Traineeship (Operations)

Customer service 3 x interviews; industry representatives (various

companies)

PO Certificate IV Property Services – Owners’ Corporation (Specialised)

Conducting meetings

4 x student team role-plays;

industry-styled meeting

30th

ascilite Conference 2013 Proceedings Page 168

Annotations were created by selecting and categorising segments of video content, and then adding notes.

Where the learning design accorded, this then built into collaborative threaded discussions anchored to the video

segments. The audiovisual technology cohort utilised individual student - teacher annotations, while each of the

property services cohorts involved small group learning design involving student - student - teacher annotations.

All four cohorts analysed their videos in both the physical classroom and the online classroom concurrently (an

increasingly accustomed environment), along with opportunities to continue presence in the online classroom

beyond the boundaries of the tertiary timetable.

The data that informed the findings involved a triangulation of student pre- and post-surveys, student and

teacher interactive process interviews (observation/demonstration and semi-structured interviews), and artefact

analysis. The literature framing the vocational cases focused on online learning engagement with content, and

content suitable for presentation in video for learning interaction and workforce preparation, such as

professional case studies and role play/simulated performance. Synthesis of data presents the findings themed in

recognised motivation/success factors (as aligned to Bekele, 2010).

Learning engagement with online content, including video content

Learning generally involves making meaning of learning content, from passive transmission methods through to

active consumption such as analysis and discovery learning. Collaborative analysis of learning content has been

used successfully in traditional classrooms, and increasingly elaborately in technology-supported learning. An

example of classroom collaborative analysis as used by Black (1993) involved students writing their own ideas

of a chemical topic, redistributing, then volunteers read out ‘quite good’ examples. This approach gathered

momentum with the students “and after pulling three or four answers together with some discussion of the

merits of each, the class had developed a very complete understanding of the [chemical] concept … [and] the

answers also allowed us to clarify some misconceptions” (Black, 1993, p.143). Educational technology extends

the possibilities of content analysis and collaboration:

Collaborative learning can enhance knowledge acquisition, and, when coupled with the use of

digital technology, it can aid in the generation of creative thought processes through the provision

of a shared electronic space within which learners are encouraged to take risks, make mistakes and

think critically as they work together (Wheeler, Waite and Broomfield, 2002, in John & Wheeler,

2008, p.38).

As others before him, Bekele (2010) recognised that a single factor (such as the educational technology

employed, e.g. Kirkwood, 2009) does not alone cause success, but “[p]resumably, technology, course, and

support factors mutually affect success measures” (Bekele, 2010, p.118). A meta-analysis study by Bekele

(2010) examined 30 published studies for factors of success with online learning environments (with or without

a face-to-face learning component). These 30 studies each sought to measure motivation and/or satisfaction.

From his findings, Bekele (2010) developed a conceptual framework based on a range of factors he found

affected success (see Table 2). Bekele grouped the last four factors together under 'motivation'. Motivation has

already been highlighted as a factor for engagement with MAT activities in four undergraduate case integrations

(see Colasante & Lang, 2012).

Table 2: Bekele (2010) conceptual framework of factors affecting internet-based learning success

Bekele (2010)

factors

Detail

Technology

factors

technology attributes; student ICT skills, experiences, or views, e.g. technology is

easy/friendly; perceived or actual use/function of technologies, e.g. dependable access

Course factors quality elements in course design, e.g. course relevance, organisation, goal clarity,

flexibility; the ‘how’ of learning, e.g. problem based, process oriented

Support factors technology leadership and support provided by faculty/tutors, administrators, and

peers

Mo

tiv

atio

n

Task choice student choice of task(s); availability of ample activities, resources, and technologies

to freely choose from; student choice of time and place of learning

Effort constant challenge and/or effort; need to expend a reasonable amount of effort

Persistence time spent on-task; continue working despite any obstacles encountered, e.g.

technicalities, support systems, group dynamics, and thinking skills obstacles

30th


Achievement level of achievement; achievement demonstrated via task choice, effort and persistence

Two decades ago educational video was considered expensive, raising the question “could video material

provide worthwhile material that could not be provided more cheaply using other media?” (Rowntree 1990,

p.256). Early use of video to connect learner and teacher included a “stiff, unemotional ‘talking head’ of a

professor or tutor” (McGreal & Elliott 2008, p.147). Now video is easier to procure and can represent learning

content of infinite topics ready for interaction via modern methods, such as individual or collaborative textual

annotation (e.g. Rich & Hannafin, 2009). Representation of vocational case examples for analysis, video

recordings of student role-play/performance, and granular analysis of video content are discussed below.

Vocational case examples in guided video-case analysis

Authentic case representations – in written, video, or other format – can help to develop and/or apply work-

ready skills, such as social interactions, negotiation, problem solving, and critical thinking knowledge and skills

(Bennett, et al., 2002). Bennett, Harper, and Hedberg (2002) contrast case-based learning to problem-based

learning (which asks students to establish solutions) to that of opportunities to learn from past case experiences

of people in their discipline fields. While this may involve experts from the field, Bennett et al (2002) note

‘exemplars’ or excellent examples are not necessary; rather cases that illustrate “the complexities and

contradictions inherent in realistic situations” (Spiro & Jehng, 1990, in Bennett et al., 2002).

Video recordings of student role-play

Facilitating learning via role-play is an established teaching strategy for interactive skill development of

interpersonal skills (e.g. for human services professionals, Johnson & Douglas, 2010). It also promotes team

decision-making, professional communication, and can help “students develop abilities in problem solving by

requiring them to assume different roles and confront unstructured problems in scenarios involving the

professional domain of the given role” (Hou, 2012, p.211). Role-play remains important in learning as it offers

“a deeper kind of learning … the ability to see the world from different points of view” (Dalziel, 2010, p.56).

This deeper learning arises mainly from post role-play reflection (Dalziel, 2010), and video can aid this

reflection on role-play (e.g. Walter & Thanasiu, 2011; Robinson, 2007).

Granular analysis of content

Analysis of content, in fine or course granularity, is supported by segmentation of content data into discrete

chunks (e.g. Medina & Suthers, 2008). A text-based segmentation example involved postgraduate education

students using a wiki to ‘sketch-thread-theorise’ (Davies, Pantzopoulos & Gray, 2011), where students were

asked to note their own professional accounts, highlight key segments and draw out keywords, and then

annotate with their reflections and associated theories. This activity was combined with peer contributions and

formative feedback, and was found to create “a rich learning environment where professional outcomes were

enhanced” (Davies, et al., 2011, p.810). The MAT annotation system allows for segmentation of video. For

example, undergraduate chiropractic students analysed segments of a videoed chiropractic clinical case by

selecting, categorising and adding their reflections and theoretical knowledge to each selected segment (to build

clinical notes and a working diagnosis) with largely positive findings (Colasante, Kimpton & Hallam, in press).

Methodology

The methodological approach was via a multiple-case study, with mixed-method data collection. The project

sought to examine the use of MAT across different industry disciplines and tertiary sectors. Nine class cohorts

who identified as using MAT for work-relevant themes were invited to participate, four of which were from the

vocational sector and form the focus of this paper. Therefore, case selection was purposive as they comprised

teachers and students who were (a) early adopters using MAT for (b) work-relevant and/or industry partnered

themes. Purposively selected cases are recognised particularly in qualitative studies to deliberately select cases

or units that can help answer specific research questions (Teddlie & Yu, 2007).

The mixed data collection methods harnessed both qualitative and quantitative data from students and

qualitative feedback from teachers. The methodology was trialled in a pilot-case study in preparation for the

multiple-case study (Colasante, 2011), therefore, this project benefited from pre-tested instruments following

minor design adaptation. Methods included pre- and post-survey, observation/demonstration, interviews, and

artefact analysis. University ethics approval was granted to conduct the research.

30th


Data collection

The vocational students using MAT were invited to complete a survey in two parts; a questionnaire before using

the new tool, and another after. Both questionnaires included primarily quantitative questions (mostly Likert-

scaled), and a minority of qualitative questions. The pre-survey established demographic detail and attitudes to

online learning, while the post-survey harnessed student opinions of their experiences of learning with MAT.

Further to this, both students and teachers were invited to participate in individual ‘interactive process

interviews’ or IPIs (Colasante, 2011). These involved half-hour observation/demonstration and interview

sessions, involving 10-15 minutes of observation while using MAT and think-aloud protocol, followed

immediately by 10-15 minutes of semi-guided discussion on their experiences using MAT. In all vocational

cases, the MAT activities had concluded by the time of interview, therefore participants were asked to

demonstrate and talk-through their activities in the first part of the IPI session. Additionally, student and teacher

participants were invited to allow specific MAT related learning and assessment artefacts to be used for purposes

of the study, to compliment general MAT learning analytics (general analytics are used in this paper).

The classes ranged in size from 20 to 39 (sum of 110 students). Student research participation rates (Table 3)

ranged from 23 to 69 per cent for the surveys (59 pre-surveys and 37 post-surveys completed across the four

cases). Student participation numbers in IPIs were low, however, formed a useful source for clarification.

Table 3: Participation levels in the study

Class size ^ Pre-surveys completed

Post-surveys completed

IPI participation Students

IPI participation Teachers and Teacher assistant

AV 39 18 (46%) 13 (33%) 1 student 1 teacher 1 assistant**

PD 22 13 (59%) 5 (23%) 1 student 1 teacher* 1 assistant**

PT 20 8 (40%) 10 (50%) 2 students 1 teacher*

PO 29 20 (69%) 9 (31%) 2 students 1 teacher* 1 assistant**

Sum 110 59 37 6 3

^ Class size represents number of students enrolled, not the number of students who actively attended classes.

* Same property services teacher, therefore IPIs conducted in 1 sitting.

** Same teacher assistant, therefore IPIs conducted in 1 sitting.

Limitations

For ease of comparison between cases, survey data is presented in ‘SSPS valid percentages’, despite some cases

having low participation numbers, and each case with at least some non-responses between pre- and post-

surveys. With established “evidence that nonresponse can affect survey estimates” (Fowler, 2009, p.54), the

percentages represent potentially biased samples of each cohort. The participation rates are given above.

Findings of motivation and satisfaction across the four vocational cases

Data analysis findings from the four vocational cases, one audiovisual technology and three property services

cohorts, present here under the Bekele (2010) factors for success with ‘Internet-Supported Learning

Environments’ of: technology, course and support factors, and then the four motivation factors of task choice,

effort, persistence, and achievement. These factors theme findings from across the data range of student pre- and

post-surveys, student and teacher IPIs and general learning artefact analysis harnessed from within MAT.

Demographically, the gender mix of respondents across the four cases was predominantly male (approximately

90%) in all but the PO case, which had an almost even mix (53% male). The age range demonstrated a typical

post-secondary age range for AV with a minority of mature-aged students, compared to the three property

services cohorts, which each represented a mature-aged student base. There was a dominance of EFL (English

first language) respondents across the four cases.

Technology factors

Pre-survey data illustrated ICT access to technology and skills and attitudes towards learning with technology

across the cases. A majority reported daily access to computers and the Internet while a minority reported access

most of the time (5 to 12%) and none reported less frequent access. Self-perceived ICT skill levels were mainly

medium to moderately-high.

30th


On whether students liked learning online, the AV cohort gave the strongest positive responses, while most of

the property services students liked online learning at least some of the time and a minority (12-15%) not liking

online learning (Figure 1). When questioned on willingness to use video in learning (i.e. asked in case specific

questions, e.g. AV: “I would like to view customer service techniques via video footage”), there was majority

agreement (Figure 2).

Figure 1: Pre-survey attitude to online learning:

like online learning

Figure 2: Pre-survey attitude to online learning:

willingness to use video in learning

Two related questions (in different sections of the post-survey) directly sought negative reaction to MAT.

Figures 3 and 4 illustrate that more appreciated the technology of MAT than not, indicating not necessarily

finding difficulty in use and the technology not interfering with learning.

Figure 3: Post-survey attitude to ease of MAT use:

difficult to use

Figure 4: Post-survey attitude to ease of MAT use:

interfered with learning

However, to find out where some of the technological issues lay for the minority who didn’t indicate

satisfaction, the post-survey open responses provided clues such as access, creation of markers and general

glitches costing time. Examples of comments included:

• “there was a difficulty in accessing into MAT [sic.]. There should be a convenient link to be set up on the

website for easier access” (AV)

• “Too many glitches” (AV)

• “if it is only employed once, it would be the learning of a new process, otherwise it is of benefit” (PD)

• “unable to access MAT at work or home, frustrated by system” (PT)

• “[difficulty] getting markers to span correct time duration” (PO).

Although these comments represent a small minority, they do provide alerts for future improvements to the tool

(some of which have since been implemented), or potential improvements in support mechanisms for students.

Course factors

Outside the vocational cases, analysis of four concurrent higher education (undergraduate) cases of MAT use

found that course design factors effected student satisfaction. In particular it was noted:

Higher satisfaction responses by students were presented in MAT cases that had some or all of: 1.

30th


[T]eacher presentation and upload of videos in MAT (compared to student generation and upload

of videos) 2. [T]eacher feedback 3. [L]earner-learner interaction to achieve meaningful goals 4.

[F]ormal assessment requirement (Colasante & Lang, 2012, p.462).

All four vocational course designs met these four conditions excepting that while the AV case aimed for

meaningful goals, it did not incorporate learner - learner interactions. Note: while each cohort had short time

spans to conduct MAT activities, most over two to three weeks only, the PD group experienced intensive

interaction at their subject’s end. Course design features across the four cases are represented in Table 4.

Table 4: Course design factors involving MAT

Learning objective(s) Individual or group analysis

Teacher feedback

Industry involvement

Assessment

AV

Provide quality service

to customers

Individual

work

Each student

received

feedback

within MAT

Not directly Formative

assessment

(summative:

role play)

PD

Coordinate customer

service activities in the

property industry

Manage relationships /

networks

5 groups

Division of

labour promoted

Spot checks in

MAT; whole

class feedback

and debrief

Video: teacher

interviewed industry

rep from large

facilities management

business

Summative

assessment (plus

reflective

journal)

PT

Implement customer

service strategies in the

property industry

Establish networks

Manage conflict and

disputes

6 groups

Division of

labour promoted

Spot checks in

MAT; whole

class feedback

and debrief

Videos: teacher

interviewed three

industry reps from

various facilities

management

businesses

Summative

assessment (plus

reflective

journal)

PO

Facilitate meetings in

the property industry

4 groups

Division of

labour promoted

Spot checks in

MAT; whole

class feedback

and debrief

Not directly Summative

assessment (plus

reflective

journal)

Respondent satisfaction of course design features involving MAT is illustrated in Figures 5 and 6. AV was the

only cohort to report less than 50% satisfaction with access to expert opinion in video. Responses to an open

post-survey question “What was it about MAT that was least helpful to your learning?” indicated that the AV

video content was not as realistic/relevant/accurate as could be.

Figure 5: Post-survey satisfaction with course

features: access to expert opinion

Figure 6: Post-survey satisfaction with course

features: activities relevant to workplace practice

PT was the only cohort to report less than 50% satisfaction with activities relevant to workplace practice (fig.6).

Neither the open post-survey responses nor the IPIs indicated a major contributing factor for this; there was

generally satisfaction apart from technical issues. A high neutral response from the PT cohort may be indicative

of the wordiness of the question (or inclusion of the word ‘eventual’ when already employed in the field).

Despite the high neutral PT response, in interview one PT student articulated that he appreciated the video-

AV - Diploma

PD - Diploma

PT - C4 T/ship

PO - C4 Spec

AV - Diploma

PD - Diploma

PT - C4 T/ship

PO - C4 Spec

30th


centred learning compared to traditional learning, including access to the current and relevant practicing expert’s

input, and learning by discovery:

[The] process I had to follow was listen to the… [expert in the video who] told you where he

worked, how many people were under him, what’s critical to… his work environment, what had

to be done, what not had to be done [sic.] and how he went about those things… We had to

comment on set main headings and then break it down to putting it into our words, instead of the

teachers coming up with the handout and saying here this is what it is, listen to this. Which I don’t

think it really does much because the teachers telling you where all the good points are but… in

this you’re finding it. (PT Student 2 IPI)

Support factors

The teacher, teaching assistant and student interviews confirmed that support was structured and delivered into

each cohort, offering both technical and teaching support for at least the first class in which MAT was used.

However, in the wider student opinion of the survey, not all explicitly agreed that ‘MAT allowed me to receive

encouraging support’. While 62% of the AV respondents agreed, the remaining 38% were neutral; all remaining

cohorts had majority neutral responses. The high neutral response may indicate they were unsure what the

question meant (i.e. whether it meant technical support or learning support from peers and/or teacher). Upon

seeking further articulation on this in the open post-survey questions, only a few AV and PD students provided

clues to the ambivalence of the responses. These comments related to class time to support the learning in MAT

(PD: MAT introduced late in subject), and limited collaboration and feedback from others (AV:individual

activities). For example, students wrote:

• “no communication” (AV)

• “No feedback as it was last lesson” (PD)

• “Introduction into MAT was brief” (PD).

The teacher assistant interviewed noted that his classroom technical support was most needed in cohorts where

students were less technologically able, including the PD students:

I think I made a difference. Probably most in the… [PD] property services group where the people

were least technology savvy and I probably had the least impact in the audiovisual group, where

they seemed to be quite fluent. They were a younger demographic in the class and quite tech

savvy… They [PD] were an older demographic, I think most of them worked during the day. … A

few… didn’t even know how to log into the system… I had to be very explicit in all of the

directions in… [that] class. (Teacher Assistant IPI)

Task choice

Three themes of student choice emerged within the four vocational cohorts. The first was choice of video for the

AV students, where the teacher provided “two videos… [as] it gave students an option to annotate either one”

(AV Teacher IPI). The second was the choice of student groups to organise their own division of labour across

each of the property services cohorts. For example, for the PT cohort, the teacher “gave them the option, they

could either do it [annotate the video] themselves or they could work together in their groups and they could

divide up the video and mark a section each or share the markers to mark up… and share the workload”, and

similarly for the PD cohort. One PD student noted in interview the interrelationships between concepts across

the task division:

my two focuses were ‘customer service’ and ‘relationship building’… [while others had]

‘communications’ and ‘negotiations’. They would find their block [or marker] overlapping with

my block because, for instance, [the industry representative] talks about the relationship between

the contractor and the consultant, and there is a lot of negotiation and communications involved in

working with your consultants and your contractors (PD Student 1 IPI).

The third choice to emerge was the flexibility in annotation approach, e.g. recording the PO meeting minutes:

They could either do it as they viewed the video in dot points… [finding] a section where

something was being discussed and then they could take a minute of it with a dot point

highlighted in the video and they progressed throughout the video in that format. Or some of the

30th


students… viewed the whole video and then just stopped it here and there and made some

notations of the minutes [external to MAT] and then they went back and entered all the minutes in

at the end (PO Teacher IPI).

MAT was set up to support both processes, with yellow marker categories, ‘Note for minutes’, for progressive

in-video annotation, and green marker categories, ‘Minutes’, for summative end-of-video annotation; students

could select either option.

Effort

The students who actively participated in MAT tended to meet their teacher’s expectations of video annotations

(according to the teacher IPIs). Table 5 illustrates general MAT learning analytics across the cohorts. In

particular, it shows the number of students who participated in MAT activities, and of those, how many markers

were created on average for each student and how many per cohort. The learning design should also be taken

into account as regards to the number of markers created, including that the three property services cohorts

worked in small groups and were encouraged to task delegate while the audiovisual technology students worked

independently.

Table 5: Evidence of student interactions in MAT (harnessed by MAT learning analytics)

Case Students active in MAT Markers created: average (range)/student per cohort

AV 22/39=56%

17 (1-20)

384

PD 16/22=73%

7 (2-14)

116

PT 17/20=85% 9 (1-16)

149

PO 21/29=74% 7 (3-20) 149

During analysis of the general MAT artefact analytics, qualitative observations were made of the students’

annotations to help contextualise the figures. For the AV cohort, the annotations were mostly short, direct points

of observation, although a minority provided more complex single sentences (e.g. drawing cause and effect). For

the PD cohort, the total markers per student were not entirely indicative of the amount of work completed, as

initial annotations sometimes sparked lengthy discussions in the threaded comment panels. While some students

kept annotations to a one-sentence entry, the majority completed detailed and lengthy annotations. For the PT

cohort, the majority of annotations and comments appear to be of a reasonable length (in most cases 1-3

sentences). The ‘comments’ panel featured frequently in this group, with many students commenting on their

own or peer’s markers. And finally, for the PO cohort, annotations within some markers were dramatically

longer than others (e.g. markers under the marker type ‘minutes’ were mostly several paragraphs in length

whereas ‘notes for minutes’ were generally 1-2 sentences). Consequently, the range quoted above may present a

distorted view of and underestimate the amount of work some students achieved.

Persistence

On student persistence, two post-survey open questions yielded examples. One question asked “How did you

overcome any challenges that you faced while using MAT?”. A range of responses to this illustrated that when

faced with challenges the AV students solved them by re-attempting the task or restarting the web browser,

together with asking their teacher for help or applying to their real-life experiences. The PD students asked

others including support staff for help or utilised trial and error as ways to solve challenges. While PT students

similarly asked others for help, and employed perseverance to solve challenges, some gave up, e.g. “unable to

overcome challenges due to lack of off site access” and “turning the computer off”. The PO students asked

others for help, or observed what others did, or re-did steps as ways to solve challenges.

Another open question, “From your experience of using MAT, what advice would you give to other students

who might be about to use it?”, also harnessed a range of responses, as summarised here. AV student advice

included: follow instructions carefully, think laterally, use good video content, and work through some of the

glitches. Some AV students gave MAT praise in their responses but one student felt it was pointless. PD, PT and

PO respondents encouraged others to give MAT a go and to explore its features. Additional encouragement

included: it is not hard to use and will help in your learning (PD), and to explore how it can be used in the

30th


subject discipline (PT).

Achievement

All IPI participants were asked via semi-structured questioning whether the MAT activities helped them to

achieve their class’ specific intended learning outcome/s. All responded positivity to this achievement; some

despite issues (see Table 6).

Table 6: Did MAT help students to achieve the specific intended learning outcome/s

IPI Student (S) and Teacher (T) quotes on achieving intended learning outcome/s

AV

S The video I thought was sometimes a bit vague… in what they were asking you to find and what was

actually in [the video] … In terms of the program itself it was very helpful

AV

T …it was a more effective assessment… [and] it’s an indicator for me and an indicator for them about how

much they know about the subject. So for me to be able to tell really quickly early on in the unit how they

are with their customer service knowledge was a really great opportunity to have

PD

S This is a tricky one, ‘cause I come from the industry and I’m studying something that I’m working in, a lot

of this is now almost innate to me. I regularly work on these principles, so I suppose you could say it was

reinforcing that at an academic level

PD

T They had to compare and reflect on it [video interview], I thought that was quite effective and I thought …

a lot of learning went into that and hopefully my idea was—and I think it worked—was to work out a best

practice strategy, so to use MAT very much as a learning tool, from a holistic perspective

PT

S-1 You may have whatever knowledge you may have, but if you have someone else’s experience it assists in

building up that sort of a knowledge, or reinforcing it

PT

S-2 It’s you who’s got to finish that final step and that’s the video step, that you’re doing the actual thing …

this is putting the practical and the theory together … to put it into your own sense or see how other people

do it and actually … putting it together, is a good way to know if you’re understanding the subject

PT

T Yes they did [achieve] and the idea was that they would reflect on their own knowledge, which I believe

they did and compare and contrast if you like what these industry experts were saying

PO

S-1 Having a look at the way other people do things, and certainly making comments on other people’s

groups, makes you take that step back… You don’t often get to critique your performance… having a look

at how other professional managers do it… we’re learning from our peers

PO

S-2 You either achieve that or you didn’t depending on the situation and the group… the dynamics in the

group and whether some people role played more effectively than others

PO

T …with this particular group of students [MAT was used] to really provide evidence to me of their already

existing skills. Although by viewing another person’s meeting and not their own, it really forced a lot of

learning and hopefully they all did pick up something that they didn’t already know from the other

students and so that peer-to-peer learning by viewing other student’s video I thought was very positive

Conclusion

Findings from study participants across the four vocational cohorts (audiovisual: AV; property services: PD, PT,

PO)—themed to Bekele (2010) conceptual framework factors for success with online learning—showed that for

technology factors, a majority found the tool not prohibitive to their learning, although a minority had a few

general issues, such as access and delay issues. For course and support factors, there was largely satisfaction,

although some dissatisfaction was noted for PD participants regarding MAT activities not presented until near

the end of their course, and AV respondents regarding quality of videos on offer plus indicators toward the

individual approach being a factor. While IT and learning support was offered in all four vocational MAT

classes, it is unclear whether students overall where satisfied with the learning support offered.

Motivation was encouraged by task choice where students were afforded flexibility via small group task

delegation (PD, PT, PO) and choice between two videos to analyse (AV). Student effort was demonstrated via

the general learning analytics within MAT, illustrating number of markers created supported by descriptors to

gain a sense of effort per marker. The students found ways to continue working despite obstacles encountered

by using methods such as asking for help, restarting web browser, repeating steps, or persisting in general.

Advice respondents would give other students included giving MAT a go, explore its features, and follow

instructions carefully, and also to think laterally, use good video content, and work through glitches.

Achievement towards the various intended learning outcomes—as indicated by student and teacher interviews—

tended to be effective, with a couple of qualifiers such as sometimes being more of a reinforcement of

30th


knowledge than gaining new knowledge, and despite video quality in AV. Others were enthusiastic in their

satisfaction with learning achievements.

Overall, students’ perceptions of MAT from the four vocational cases indicated majority satisfaction across a

range of factors, albeit with clues for improvements in technology and learning design support. This suggests

that further trial and examination of MAT should occur in the vocational education sector, along with tool

improvements and refinement of learning design and support. Detailed models for each of the four cases are

currently under construction, in readiness to share with those interested in how learning design might be

structured with the use of video annotation tools in vocational education.

References

Bekele, T. A. (2010). Motivation and Satisfaction in Internet-Supported Learning Environments: A Review.

Educational Technology & Society, 13 (2), 116–127.

Bennett, S., Harper, B., & Hedberg, J. (2002) Designing real life cases to support authentic design activities.

Australasian Journal of Educational Technology, 18(1), 1-12.

Colasante, M., Kimpton, A. & Hallam, J. (in press) A curriculum model to promote (chiropractic) critical

thinking with video-case annotation. In M. Gosper and D. Ifenthaler (eds.), Curriculum models for the 21st

century: Using Learning Technologies in Higher Education, Springer Science+Business Media : New York

Colasante, M. & Lang, J. (2012). Can a media annotation tool enhance online engagement with learning? A

multi-case work-in-progress report. In J. Cordeiro, M. Helfert and M.J. Martins (Eds.), Proceedings of the

4th International Conference of Computer Supported Education (vol.2, p.455-464) 16-18 April, 2012, Porto.

Colasante, M. (2011). ‘Using video annotation to reflect on and evaluate physical education pre-service teaching

practice’. Australasian Journal of Educational Technology, 27(1), 66-88.

Dalziel, J. (2010). Practical eteaching strategies for predict – observe – explain problem-based learning and

role plays. LAMS International, Sydney.

Davies, A., Pantzopoulos, K. & Gray, K. (2011). Emphasising assessment 'as' learning by assessing wiki writing

assignments collaboratively and publicly online. In J. Waycott & J. Sheard (Eds), Assessing students' Web

2.0 activities in higher education. Australasian Journal of Educational Technology, 27(Special issue, 5),

798-812.

Fowler, F. (2009). Survey research methods, 4th

ed. SAGE Publications Ltd.: London.

Hou, H-T. (2012). Analyzing the learning process of an online role-playing discussion activity. Educational

Technology & Society, 15(1), 211-222.

John, P. D., & Wheeler, S. (2008). The digital classroom; Harnessing technology for the future. Routlege:

Abingdon, UK.

Johnson B. & Douglas K. (2010). Asynchronous Online Role-plays Using a Blended Learning Design:

Integrating Theory and Practice’ in Martin J and Hawkins L (eds) Information Communication Technologies

for Human Services Education and Delivery: Concepts and Cases, IGE Publishing, 2010.

Kirkwood, A., 2009. E-learning: you don't always get what you hope for. Technology, Pedagogy and Education,

18(2), 107-121.

McGreal, R. & Elliot, M. (2008). Technologies of Online Learning (E-learning). In T. Anderson (Ed.), Theory

and practice of online learning (pp. 143-165). Canada: Athabasca University.

Medina, R., & Suthers, D. D. (2008). Bringing representational practice from log to light. In proceedings of the

International Conference for the Learning Sciences, Utrecht, June 24-28, 2008

Rich, P. & Hannafin, M. J. (2009). Video annotation tools: Technologies to scaffold, structure, and transform

teacher reflection. Journal of Teacher Education, 60(1), 52-67

Robinson, L. (2007). Developing reflective thought in preservice educators: Utilizing role-plays and digital

video. Journal of Special Education Technology, vol. 22, no. 2 (2007), p. 31-43

Rowntree, D 1990, Teaching through self-instruction; How to develop open learning materials, Kogan Page,

London.

Teddlie, C. & Yu, F. (2007) Mixed Methods Sampling; A Typology With Examples. Journal of Mixed Methods

Research, 1(1), 77-100.

Walter, S. & Thanasiu, P. (2011). Enhancing role-play activities with pocket camcorder technology: Strategies

for counselor-educators. Journal of Creativity in Mental Health, vol. 6, no. 4 (2011), p. 284-297

30th


Author contact details:

Meg Colasante, [email protected]

Michael Leedham, [email protected]

Please cite as: Colasante, M. and Leedham, M. (2013). Motivation and satisfaction for vocational education

students using a video annotation tool. In H. Carter, M. Gosper and J. Hedberg (Eds.), Electric Dreams. Proceedings ascilite 2013 Sydney. (pp.167-177)

Copyright © 2013 Colasante, M. and Leedham, M

The author(s) assign to ascilite and educational non-profit institutions, a non-exclusive licence to use this

document for personal use and in courses of instruction, provided that the article is used in full and this

copyright statement is reproduced. The author(s) also grant a non-exclusive licence to ascilite to publish this

document on the ascilite web site and in other formats for the Proceedings ascilite Sydney 2013. Any other use

is prohibited without the express permission of the author(s).

mailto:[email protected]

mailto:[email protected]

motivation and satisfaction for vocational education...

Documents