wiki pedagogy a tale of twowikis · system (lms) in the advanced web technology unit, but within...

Wiki Pedagogy -A Tale of Two Wikis

Matt Bower, Karen Woo, Matt Roberts, and Paul Watters, Member, IEEE

Abstract-This paper reports on a research projectinvestigating the deployment of two different wikis in twoMasters of IT subjects. As well, two different learning taskdesigns were used (weekly whole class extension question tasksversus semester long group projects) providing further basis forcomparison and contrast within the project. Data collectionmechanisms included a 22 item student survey, interviews withacademics and observations on student contributions to the wikis.Key results regarding students' patterns of behaviour, the impactof learning task design upon contributions, the effect of training,and the quality of the wiki applications themselves are discussed.Based on these results, a set of 12 principles for implementingwiki based learning tasks are recommended.

Index Terms-collaborative learning, online learning,pedagogy, wikis.

I. INTRODUCTION

IN order to ascertain the utility of deploying wikis tofacilitate collaborative groupwork in our Information

Technology courses, Postgraduate Professional DevelopmentPrograms at Macquarie University conducted a researchproject involving two different two different learning taskdesigns using two different wiki applications in two of our

Master of IT subjects. The research was performed duringsemester two of 2005.

Wikis had been identified as a particularly interesting e-

learning technology to deploy in our Masters subjects becauseof their identified potentials for developing expert thinking.Whereas synchronous technologies such as instant messagingand desktop sharing were seen as important for noviceprogrammers to instantaneously share declarative andtroubleshooting knowledge, the ability to dynamically yetasynchronously edit and restructure web content using wikiswas seen to afford Masters students the opportunity to developand share the sophisticated and highly structured knowledgeschemas more akin to expert thinking.

Before embarking upon the project the following researchquestions of interest were identified:

Manuscript received April 1, 2006. This work was supported by a

Macquarie University Strategic Curriculum Innovation Grant.M. Bower, K. Woo, M. Roberts, and P. Watters are with Postgraduate

Professional Development Programs at Macquarie University, 2109, Australia.Phone: +61-2-98509501; fax: +61-2-98509509; e-mail: {mbower, kwoo,mattr, pwatters} ics.mq.edu.au.

1-4244-0406-1/06/$20.00 ©2006 IEEE.

1. To what extent does the structure of the learning taskinfluence students' patterns of behaviour?

2. To what extent does the wiki application itself impactupon the learning experience?

3. How much and what type of training is required forstudents to use wikis effectively?

4. How do wikis compare to other e-learning tools(such as discussion boards) and face to faceapproaches for facilitating collaboration?

5. What principles can be suggested for successfulimplementation of learning activities using wikis?

The two wiki applications used to explore these researchquestions were the wiki built into the Moodle LearningManagement System and a Jotspot wiki. The first learningdesign involved weekly whole-class extension question tasksbeing posed in an Object Oriented Technology subject, andstudent responses being elicited via the Moodle wiki. Thesecond learning design involved a semester long group projectin an Advanced Web Technology subject, facilitated throughthe Jotspot wiki. This paper begins by describing how thiswork relates to other research on the use of wikis foreducational purposes, and then goes on to articulate theresearch methodology adopted in this case, the resultsobserved during the data collection process, and proposesrecommendations for deploying wikis in tertiary courses basedon the observations and analysis performed.

II. PREvIous RESEARCH ON WIKIS IN EDUCATION

When the first wiki was released by Ward Cunningham in1995, the efficacy of the tool for educational purposes wasself evident. The capacity to create, edit, and restructure webbased content could allow students to represent data in a morefreeform manner than existing asynchronous collaborativetools, such as discussion boards. Since then there have beennumerous documented cases of using wikis in educationgenerally, and computer science education specifically [1, 2].

Some of the learning designs based on wikis that have beenutilised include:

* collaborative artifact creation tasks* review activities* construction of a case library* straight information distribution* wiki Micropedias* FAQ wikis* Consensus documents* Branching story/Tree simulation tasks

Authorized licensed use limited to: MACQUARIE UNIV. Downloaded on November 18, 2008 at 23:34 from IEEE Xplore. Restrictions apply.

0

0

[1, 3,4]

problem solving wikisproject spaces

The flexibility of wikis to dynamically restructure andhyperlink information contained within a web space obviouslyaffords educational developers a myriad of learning designopportunities. However, there has been relatively littleresearch into how best to deploy wikis for educationalsuccess. The scarce work in the area is summarized below.

In his research into the use of wikis for distributedasynchronous brainstorming, Davies [5] identifiesunderstanding, trust, and value as significant factors forsuccessful deployment. On the basis of his case study andanalysis, he recommends:

* "careful seeding" with appropriate modeling but notso extensively so that students lose ownership

* training (both technical and social)* encouraging successful behaviours via reference to

other examples.Davies emphasizes the relationship between the convener andthe student and the importance of explicitly developingpositive student behaviours as crucial.

Georgia Tech has been exploring the use of wiki-like tools forseveral years, with a variety of learning designs. As a result,their CoWeb catalog instructor's guide [1] recommends:

* introductory activities to get students thinking aboutthe role of the tool and to learn about its mechanics

* some not-required-but-useful activity to convincestudents that it can be valuable to participate

* a period of open use by students, with a few activities* some closing activities.

This is an explicit approach to developing positive patterns ofstudent behaviour within the wikis.

On a different tact, other research proposes that an effectivestrategy to improve implementations of wiki-based learningtasks is to analyse the roles adopted by students. According toGuzdial et al. [6] users may be authors, purpose agents,central users, peripheral users, site designers, developers,administrators, or even support staff. By analyzing the rolesadopted in the wikis and implementing tactics to support eachof them, users are able to collaborate more effectively andthus the quality of overall learning can be improved.

The different features and functionalities of the various wikisthemselves can impact upon the effectiveness of the learningexperience. Tonkin [7] rates features such as version control,permissions, file attachment capabilities, and so on for ninepopular wiki tools, and suggests ways in which the differentfeatures of the various wikis will be more appropriate forcertain tasks but not others. The key idea that Tonkinemphasizes is that during the tool selection process thefunctionality of the specific wiki tool needs to match thecollaborative requirements of the learning task.

Despite the identified potential of wikis to offer affordancesbeyond discussion boards and email for online asynchronouscommunications (affordances such as restructuring ofinformation, version control and dynamic markup of pages),in many cases teachers have found the use of wikis ineducational contexts to be unsuccessful, or at least not meetdesign expectations [5, 8].

The study described in this paper attempts to extend upon thisprevious work in the area to ascertain important principles forsuccessfully implementing wiki-based learning activities, bothin tertiary education generally and IT education specifically.

III. METHODOLOGY

Our multiple case study design commenced by selecting twoMaster of IT subjects within which to apply wiki basededucational approaches. These subjects were ITEC802 - anObject Oriented Programming unit with 28 students enrolled,and ITEC831 - an Advanced Web Technology unit with 29students enrolled. The learning objectives of each subjectwere used to develop appropriate wiki based learning tasks.Then two different wiki tools were assigned to each subjectbased on these requirements, providing further means forcontrast and comparison within the project.

A. The two cases1) Weekly whole class extension question tasks in anObject Oriented Technology subject

"Object Oriented Programming" is a core unit for MacquarieUniversity's Master of IT program. This unit used the Moodlewiki that comes bundled with the open source MoodleLearning Management System to implement a weeklyextension question task for students.

Different styles of questions were proposed each week for the"WikiWork" activities, in order to provide further insight intohow the type of cognitive engagement required by students(not just the learning task structure) affects their patterns ofcontributions. Styles of tasks included:

o Hints, tips, and wish-list tasks - such as "describesome useful features of Java that may not be knownby your peers"

o Coding questions - requesting that students write aprogram that achieves a certain specification

o Descriptive tasks- such as "describe the meaning ofcovariance and contravariance"

o Compare and contrast tasks - such as "compare andcontrast the event handling mechanisms of differentlanguages"

o Evaluative tasks - requiring students to evaluate thevalidity of a claim or the quality of an artifact.

As well as contributing to the WikiWork tasks, students wereexpected to complete tutorial and practical exercises andattend face to face lectures.


The Moodle wiki that was used to collect the weekly studentresponses provides basic wiki functionality, including theability to:

* create new pages* link to existing pages* format text using WYSIWYG interface* ability to roll back the wiki to previous versions.

where they could choose to either complete the (non-assessable) tutorial exercises, or to meet with their group.

The Jotspot wiki is a fee for service, https secured applicationthat incorporates all the functionality of the Moodle wiki, plusthe ability to:

* Attach files to pages and email to and from them* Set a sophisticated variety of permissions to pages* Insert "applications" into wiki pages.

Figure 1 - The Moodle Wiki

An assessment weighting of 10% was allocated to theWikiWork in order to avoid lack of participation problemsobserved in other research projects [5]. Students were requiredto provide a significant contribution to at least one of thetwelve WikiWork extension activities and a minorcontribution to five others in order to score full marks. Minorcontributions could be restructuring or commenting uponother people's contributions. Students were able to contributeto each week's WikiWork task until the end of semester.

2) Semester long group project tasks in an Advanced WebTechnology subject

The Advanced Web Technology subject is an optional unit forstudents completing a Master of IT at Macquarie University,but a compulsory one for those completing a Web Technologymajor. A Jotspot wiki was used to not only facilitate groupcollaborations during the students' major project but alsofacilitate the subject's administration (such as disseminationof subject related information, distribution of tutorialexercises, and so on).

The group project itself involved teams designing and thendeveloping a multi-tiered web-based application. Students selfselected themselves into one of eight teams of between twoand five members, based on the type of application theywished to develop. Teams were then allocated a group spaceon the wiki and could choose to use that space in any waythey designated. Other than collaboration via the wiki,students had two hours of face-to-face contact every fortnight

Jotspot calls itself an "application wiki" because it allowstools such as blogs, calendars, forms, discussion boards and soon to be integrated into the wiki by embedding simple scriptswithin the wiki pages. However, in order to draw a moredirect comparison of how wikis were used within the twosubjects, these extra applications were not applied in theAdvanced Web Technology case.

An assessment weighting of 15% was attached to student'scontributions to the wiki as part of their group project mark.Once again this was deemed necessary to encourageparticipation.

Note that in the Object Oriented Programming subjectcontributions were made by the whole class to the same wikibut to different tasks each week. On the other hand, in theAdvanced Web Technology unit the same task was prescribedthroughout the semester but was being completed by differentgroups. Designing the study in this way allowed researchers toobserve variations in patterns of contributions based ondifferent learning tasks in the first case, and variationsbetween groups on the same learning task in the second case.

Also note that the wiki was used as the Learning ManagementSystem (LMS) in the Advanced Web Technology unit, butwithin the LMS in the Object Oriented Programming unit.This allowed researchers to observe two different ways inwhich wikis could be situated in relation to other e-learningtechnologies.


B. Student Training in Using the WikisA minimalist approach to training students how to use thewikis was adopted, for the following reasons:

o As Master of IT students it was assumed thatparticipants would possess a degree of ITcompetence, and thus only require minimal technicalsupport on how to use the wikis to developproficiency

o As a first case investigation into the use of wikis atMacquarie University, researchers were interested inhow far students could progress with minimaltraining - a control case of sorts.

Students completing the weekly whole class extensionquestions were given a half hour face to face introduction on

how to use the wiki. As well, part of the first week'sWikiWork task requested that students contribute hints andtips on how to use the wiki, and a document explaining thenature of the WikiWork tasks was also distributed.

Students undertaking the semester long group projects were

also provided with a half-hour introduction on how to use thewiki. However rather than requiring students to contributehints and tips to familiarize themselves with their wiki,students were referred to the range of multimedia onlinetutorials available from the Jotspot website for information on

how to use the various features and facilities integrated intothe tool.

C. Data CollectionData on how students used the wikis and their perceptions ofthem was collected via three techniques:

1. Observations of students' wiki contributions2. A 22 point student survey

3. Interviews with academics involved in the project.

Academics deliberately made no contributions to the studentcollaborative spaces. This allowed researchers to observe howstudents interacted with one another without any outsideinterference, and thus provide another baseline for futureresearch in the area.

The 22 point survey was designed half way through semester,based on observations of student contributions to that pointand also the initial research questions identified for the study.The survey included both open-ended and Likert scalequestions. The Likert scale used in each case contained 7rating points ("Strongly Disagree", "Disagree", "MildlyDisagree", "Neutral", "Mildly Agree", "Agree", "StronglyAgree"). The survey instrument itself was distributed aftercompletion of the coursework but before the final examinationfor each subject. In this way students had fully completedtheir experience with the respective wikis and this experiencewas still familiar to them. Students could choose to completethe survey anonymously so as to not bias their responses inany way.

The survey instrument is attached as an appendix to this paper(ref. Appendix A).

Statistical analysis was performed upon student responses toLikert scale questions, based on a numeric allocation (of 0through to 6) to the 7 possible response categories. While thisapproach is commonly used by researchers, academicsinvolved in this project note that performing such atransformation from descriptive to quantitative data is notwithout limitations and any results need to be considered incontext. All tests performed on Likert scale questionresponses were two-tailed student T-tests.

In addition to the observations on the wikis themselves andthe survey, academics involved in the running of each unitwere interviewed for their impressions of how the wiki wasused. This interview data was then triangulated withobservations of contributions to the wikis and surveyresponses to provide a more complete and consistentrepresentation of the events that had transpired throughout thestudy.

IV. RESULTS

Key results derived from observations of student contributionsto the wiki, the student survey, and academic observations aresummarized in turn below. These results are then drawn uponin the next section of this paper to propose causal relationsbetween tasks, training, tools, and the collaborations thatoccurred on the wikis. These relationships are then used tomake recommendations about ways in which these variablesunder the control of academics (task, training and tools) canbe leveraged to improve collaborations.

A. Observations on student contributions - Patterns ofBehaviour

The analysis of student contributions provided greatest insightinto the patterns of behaviour for both the weekly extensionquestion tasks and the semester long groupwork task.

1) Weekly extension question tasksAccording to responses to the survey, students in the ObjectOriented Programming unit spent approximately 1 hour perweek on the weekly WikiWork tasks. Different patterns ofbehaviour emerged, according to the type of task prescribed:

* Tasks requesting hints, tips and ideas resulted inpiecemeal contributions. The final page or series ofpages was not well integrated.

* Coding questions produced wikis that evolved andcontained contributions that were well integrated.Students indicated that they valued this style of more"concrete" task specification more than open endedtasks.

* Descriptive tasks were also generally well integrated.* Compare and contrast tasks could be well integrated

but could also result in piecemeal wiki work.


* Evaluative tasks resulted in piecemeal, list stylecontributions that were only loosely integrated.

In summary, tasks that require a singular solution (such as asingle document or program) resulted in a more integratedproduct. For tasks that require individual opinions (be theydeclarative or evaluative in nature) a more linear andfragmented pattern of contribution evolved.

Students generally didn't take advantage of the ability tohyperlink to new pages - assumedly they either felt that thequantity of information didn't warrant new pages or that itwas advantageous to have all information represented on asingle page.

2) Semester long group projectStudents in the Advanced Web Technology unit spentapproximately 3.5 hours per week using the wiki. The numberof pages in each group's wiki space ranged from between 2pages to up to 15 pages. There was a great variety in the waythat teams used the wiki space. Some groups simply postedtextual descriptions of their project outline and intendedoutcomes, while other groups used their wiki space tofacilitate the entire project collaboration process.

Uses of the wiki for the groupwork project included:o Brainstorming ideaso Forming a design specificationo Creating a repository of technical informationo Project Management (establishing timelines,

allocating responsibilities)o Providing a noticeboard / discussion forum facilityo Posting meeting minuteso Sharing documents and resources

No groups incorporated all of these features into their wikispace, and some groups only integrated one or two of theseuses.

There was also a noticeable difference in the degree ofintegration that occurred within each group's wiki space.Some groups had individuals creating the entire content of apage, which was then not modified by other members. Thisimplies that the final product has not benefited from thesuggestions or refinements of others. On the other hand, somegroups had all members contributing to several of the pages intheir wiki space on a regular basis.

B. Responses to the student surveyFor the student survey, 39 responses were received. For theweekly extension question tasks 17 out of 28 studentsresponded (61%), whereas 22 out of the 29 students whocompleted the group project provided responses (76%).

The student survey addressed several different dimensions ofthe experience, including student perceptions of:

1. the training provided2. the features of the wikis

4. how the wikis compared to face to face collaboration5. how the wikis compared to collaboration via

discussion boards.

As previously mentioned, a combination of Likert scalequestions and open ended questions were incorporated intothe survey instruments. Key results from the students'responses are summarized below.

1) TrainingIn response to Likert scale item 5 on the survey "I receivedadequate training to use the wiki", students completing thegroup project expressed a significant belief that they had ( x3.59, c= 1.37, p = 0.042).

5. received adequate training to use the Wiki.

I

SD D MD N MA

Agr eement

Figure 3 - Distribution of responses to survey question 5

This may have been due to the multimedia tutorials they couldaccess via the internet, but more likely their increasedfamiliarity with the wiki by the end of the semester as a resultof using it more frequently than students in the weeklyextension question group.

In an open ended question asking them to describe what extratraining they would like to have received, most studentscommented that the wiki was intuitive and didn't requiremuch training. Some students suggested that some guidanceshould be given on ways in which the wikis should be used(ie, propose patterns of contribution rather than technicaladvice).

2) Features ofthe wikisBased on the response to survey question 7 "What featuresand functionalities of the wiki enhanced your learningexperience in this unit?" Students completing the weeklyextension question tasks felt that the most valuable features ofthe Moodle wiki were the capacity to see what others hadwritten on the topic, to share knowledge, as well as editingeach other's work. In contrast, students undertaking thesemester long group project indicated the most valuablefeature of the Jotspot wiki was simply as a communicationtool, not as a learning tool:

3. the learning tasks prescribed

*ITEC802

* ITEC831

SA


"I see the wiki pages as a tool which makes it easier tocommunicate between lecturer and students and alsobetween the group members, but not as a tool thatenhances learning experience."

(Group project student)

In response to question 8 "What additional features andfunctionalities would you like to see incorporated into theWikis in order to improve your capacity to learn using them?"students completing the weekly extension question tasks hadvery few requests, indicating that the Moodle wiki wassufficient for the types of activities prescribed for them tocomplete.

On the other hand, students completing the semester longgroup project requested a gamut of additional features,including:

* notices of change* interactive whiteboard* real-time chat* voice capabilities.

This indicates that the features available in the Jotspot wikidid not adequately match the (more sophisticated)collaborative requirements of the group project task, or thatstudents were not able to use the tools to their full potential.

3) The learning tasks prescribedIn response to survey questions 9 and 10 which asked studentsto evaluate the learning tasks that had been prescribed, thosecompleting the weekly extension question tasks asked formore concrete problems requiring an objective solution (ratherthan more subjective tasks involving discussions or opinions).Also, requests were made for more lecturer feedback.

Some students completing the semester long group projectseemed to struggle with the "clean slate" that was presented tothem and would have preferred a template by which tostructure their team space:

"I would be tempted to have [the same?] 'starter'project(s) allocated to each group, and then let thegroup revise, enhance, release as they see fit."


Note that in both cases students request intervention byacademics.

* the ease with which complicated concepts could bediscussed

* the ability to draw diagrams and perform non-textualdiscourse

* the capacity to receive immediate feedback.

However, there were some remarks made in favour of wikibased collaboration on the bases that information could beprocessed and provided at a time and pace of the student'schoosing.

Students involved in Advanced Web Technology groupproject mentioned the value of meeting their group membersface to face as a means of not only interacting in ways thatwere not possible via the wiki, but also in order to betterestablish a sense of team and interpersonal connection.

12. learnt more from my peers through collaborating via thewiki than if we were completing the same learning task face-

to-face.

I

SD D MD N MA A SO

Agr eement

Figure 4 - Distribution of responses to survey question 12

Based on responses to survey item 13 "My time was usedmore efficiently in conducting collaboration via the wiki thanvia face to face", students completing the weekly extensionquestion tasks felt that collaborating via the wiki wassignificantly less time effective than collaborating face-to-face(x = 1.88, cG = 1.62, p = 0.013). Explanations that studentsprovided for this included that it took longer to type than totalk, and that when you type you need to express yourselfmore carefully. As well, for any creative tasks that wouldbenefit from immediate feedback by others, studentssuggested that wiki based

* ITEC802

* ITEC831

4) Comparison toface tofaceBased on the Likert scale survey question 12 asking whetherthey felt they learnt more by collaborating using the wiki thanif they had collaborated on the same learning task face-to-face, students completing the weekly extension question taskfelt that face-to-face would have been a significantly moreeffective mode by which to collaborate (x = 1.76, G= 1.48, p= 0.004).

In the subsequent open ended question, the following reasonswere provided for this:


13. My time was used more efficiently in conductingcollaboration via the wiki than via face to face.

10

4

2

this - they enjoyed the ability to modify and restructurecontributions but this sometimes came at the expense of amore amorphous set of information.

22. Using a wi ki a owed our group to form a more negotiated meaningthan if we were using a discussion board.

* ITEC802

* ITEC831

I * ITEC802

* ITEC831

SD D MD N MA A SO

Agr eement

Figure 5 - Distribution of responses to survey question 13interaction extracted the collaborative process over a fargreater time span than if working face to face.

A student completing the semester long group projectidentified other reasons that collaborating via the wiki couldbe more time consuming as opposed to collaborating face toface:

* "the need to find' the latest update buriedsomewhere in the wiki

* the need to log into it to check what had changed[maybe there was an email notification mechanism,but we didn't use it...?]

* the need to restructure information and the wikiorganisation as the design progressed & as the teamidentified different rolesfor individuals

* the need to use electronic drawing programs to sharepictures rather than sketches drawn on paper or awhiteboard

* the need to figure out why a change to the wiki pagehad been lost, or why a damn link was broken. "


Students also alluded to the fact that the capacity for informalinteractions (such as thanking someone or discussing non-course related matters) was another benefit of face to facecollaboration over collaboration via wikis.

However students did identify that collaboration via wikisafforded some opportunities that face to face interactioncouldn't provide. Advantages that students cited included:

o the ability to share ideas and knowledgeo providing a central space to access informationo 24/7 access to datao The ability to trace the history of the collaborationso The capacity to interact asynchronously.

5) Comparison ofwikis to discussion boardsStudents were also asked how they felt learning through thewikis compared to using a discussion board. Studentresponses to the open ended survey questions were divided on

Figure 6 Distribution of responses to survey question 22In response to question 22 "Using a wiki allowed our group toform a more negotiated meaning than ifwe were workingfaceto face", students completing the group project expressed asignificant preference for wikis over discussion boards forforming negotiated meanings ( x = 3.61, cy= 0.98, p = 0.020).

The ability for students attempting the design based (groupproject) task to have greater control over the structure of theinformation was considered more valuable in terms ofdeveloping a common vision than for students attempting theweekly extension questions.

C. Observations by AcademicsObservations by academics allowed the data collected throughobservations of student contributions to the wikis and studentresponses to the survey to be confirmed and contextualized.As well, academics were able to provide information thatanalysis of student contributions and survey responses couldnot, such as an overall technical evaluation of the two wikisbeing used.

1) General observations made by academicsAcademics involved in the project appreciated the ability tomonitor and trace student collaborations. They also noted thatthe ability to restructure information allowed a set of skills tobe assessed that were not observable using other approachessuch as discussion boards (due to the rigidity of such tools) orface to face (due to the pragmatics of observing allcollaborations).

Also, academics were more explicit in identifying the physicaltemporal flexibilities that were afforded by wikis than werethe students. To try and implement a whole class collaborativeextension question exercise face to face, or to have studentscomplete a semester long group project task without the wikiswould not have been nearly as successful, in the academics'opinions.

Finally, academics also noted that there were no cases of wikivandalism, which because of their open and editable naturehas been an overriding concern about the use of wikis for

MA A SASD D MD N

Agreement


educational and other uses [8-10]. Nor were there any cases ofedit wars, where two or more individuals repeatedly editedeach others' entries based on a dispute over contentcredibility. It appears that despite the fact that there was nolecturer presence in the wiki spaces, students still assumed anidentity and as such behaved responsibly.

2) Technical Evaluation ofthe two wikisIn the opinion of academics involved in this study, the JotSpotwiki did not perform as reliably as the Moodle wiki. Sometechnical issues were experienced with the JotSpot wiki:

o Browser crashes were also reported several times bystudents

o there were several instances where special characterswere added into the wiki as a result of a bug.

o password changes needed to be performed by thesystem administrator (the only other securityalternative was to have everyone able to changeeveryone else's password!).

As an indication of how significantly this impacted the studentexperience, the most common response by students using theJotspot wiki to the request for additional features, was"improved stability".

On the other hand no major technical problems were reportedwith the Moodle wiki, which didn't include file attachmentcapabilities or sophisticated embeddable applications, butperformed much more reliably. It is a tried and tested wikiwith which academics involved in this study experienced nosignificant difficulties.

Another advantage of the Moodle wiki is that it is opensource, and as such tools can be custom designed for use withthe wiki. For instance, the convener of the Object OrientedTechnology subject made a purpose built tool to retrieve everycontribution that each student made to the wiki throughout thesemester, and compare them to other contributions from thepage where they occurred.

In terms of ease of use, both the Moodle wiki and the Jotspotwiki performed comparably, with one tool not significantlyoutperforming the other, according to academic perceptions.

V. DISCUSSION

By triangulating students' contributions to the wikis, datacollected from interviews with academics, and responses tothe student survey, researchers were able to derive someimportant recommendations for future implementations ofwikis, which are outlined below. As well, some generalmatters of interest that arose from this study are discussed atthe end of this section.

A. Proposedprinciples for integrating wikis into courses

1) Task type effects degree of integration reflected in thewiki

Based on observations by academics on the student responsesto the weekly extension questions, tasks requiring a singleproduct (such as a description or a piece of code) produce awiki containing more integrated student responses.Researchers note that Wikipedia is exactly this type of task,where each object represented in the encyclopedia requiresdefinition, and contributions by many different authors areintegrated into the one description. On the other hand thosetasks requesting individuals contribute hints and tips orevaluations were not well integrated, because information isnot required to be related to entries of other people.

2) Wikis are suited to tasks requiring negotiated meaningStudent responses to survey question 22 and observations byacademics implied that wikis are well suited to tasks requiringnegotiated meaning. If the task requires a single product thenthe fact that students read all other contributions and placetheir ideas within that context encourages learners tochallenge the meanings proposed by their peers.

3) Tasks requiring less integrated and subjectiveinformation are better suited to discussion boards

If the final artifact produced by students doesn't need to bewell integrated, then the information contributed by studentswill benefit from chronological and linear structureautomatically provided by discussion boards. As well, if theinformation is subjective, then the identity assignmentautomatically provided within discussion boards betterfacilitates inter-student discourse. On the other hand wikis aremore suitable for more concrete tasks where the identity of thecontributor is not essential to the task.

4) Providing students with suggestions for patterns ofcontribution is desirable

In this study academics involved in leading the group projectactivity did not provide students with suggestions on how tostructure each of the wiki spaces. This lead to a great varietyin the extent and quality of the wikis, and it is proposed thatby describing ways in which wikis could be utilized moreextensive and effective collaborations would have resulted.Researchers note that some student responses to the surveyexplicitly requested ideas or a "templates" for using the wikispaces.

5) Time is required for students to become proficientusers ofthe wikis

Even though the participants in this study were Master of ITstudents, there were several indications that students didn'tknow how to use the wiki proficiently. On more than oneoccasion students completing the weekly extension taskinadvertently deleted other students' responses, or admittedthat they didn't know how to use the tool to perform theoperation they intended. On the other hand, the studentscompleting the semester long group project used the wikimore extensively, felt that they had received adequate trainingand exhibited fewer indications that they did not know how touse the wiki.


6) Technical guidance throughout the use ofthe wikis can

improve the student experienceAlthough some initial training was provided, students were

not using the full range of technical capabilities of the wiki.An example is the failure by students to include RSS feedsinto the wiki to notify them of changes to the wiki. As well, itis noted that for students who are not from an IT backgroundit may take longer and more guidance for them to becometechnically proficient with using wikis.

7) Care needs to be taken to advance match the toollsprovided to the collaborative requirements of the learningtask

Student undertaking the group project requested a host ofadditional synchronous features within the wiki in order tofacilitate their activities. They indicated that wikis were usefulfor constructing a shared space for resources and ideas butthat for tasks requiring immediacy (such as brainstorming)other e-learning tools such as instant messaging or

whiteboarding were desired. Students completing the weeklyextension questions did not request these tools, nor even thefile attachment facility provided by the Jotspot wiki, becausethe tasks they were performing didn't require suchfunctionality. To optimize the effectiveness of the learningexperience (and student satisfaction) academics should inadvance anticipate the collaborative requirements of the tasksbeing prescribed, and then make every effort to ensure the toolor range of tools provided meet those requirements.

8) Task authenticity impacts on student contributionBased on the results to the survey, students completing theweekly extension question task were more opposed to usingthe wikis than those undertaking the semester long group

project. The weekly extension questions were considered bysome to simply be an additional workload requirement thatwasn't directly related to developing the skills and conceptsrequired in the unit. On the other hand students completing thesemester long group project were completing an authenticactivity through the wiki, and provided more favourableresponses about its role in the subject. This highlights the needfor more authentic tasks, where the wiki serves a valuablefunction and / or the task produces a valuable artifact in itself.

9) Academics need to be aware of the contributionreview trade offwhen choosing wikis as opposed toface toface collaboration.

Students completing the weekly extension question activityindicated that making face to face collaborative contributionswould have been much faster than using the wiki due to thefact they wouldn't have had to type in their ideas and couldhave more easily represent diagrammatic information. Aswell, if consensus is required, students indicated that the needto wait for other student's feedback before they could postresponses extracted the collaborative process over a far greatertime span than if they had been working face to face. On theother hand, the process of reviewing contributions is muchmore efficient in wikis. When working face to face, studentsneed to listen to each of their peer's contribution one at a time,

whereas in a wiki all contributions can be reviewed at once, atthe students' own pace. The larger the group, the greater thetemporal gain in reviewing efficiency that wikis will afford.

10) Wikis are more appropriate than face to facecollaboration when you require students to make wellconsidered, asynchronous contributions that aretrackable.

In responses to the survey, students identified that they had toreflect more carefully when they post their contributions to thewiki than if they were working face to face. Also, asopportunities for face to face collaboration become lessavailable (as distance learning becomes more prevalent) theappropriateness of wikis as opposed to face to facecollaboration increases. As a matter of assessment, a lecturerwill often wish to be able to review student contributions to acollaborative process. Wikis are more suitable than face toface collaborations when a learning task has any of these threeaforementioned requirements. However, survey responsesfrom group project participants indicated that face to faceengagement was a valuable complement to wiki basedinteractions because it provided the opportunity to conductricher interpersonal interactions and to create a stronger senseof group.

11) For mission critical implementations, investigatingthe performance of the wiki tool being considered is timewell spent.

Bugs, an unintuitive interface, and poor application design allimpact on student satisfaction. Sound performance on all ofthese dimensions is a baseline for integrating a wiki into acourse. Finding and evaluating web reviews from reputable /unbiased sources before selecting a wiki is a necessaryundertaking.

12) Making wiki contributions assessable and providingsome level of ongoing academic feedback upon thosecontributions appear effective andragogic strategies forencouraging participation.

Responses to the student survey by students completing theweekly extension task indicated that they would have likedmore academic feedback about the quality of the content theywere contributing. It could be expected that content evaluationand receiving additional information from academics couldimprove the degree of learning that occurs. Some students inthis group also indicated that they viewed the WikiWork as anadditional requirement that needed to be fulfilled rather thanas an integral component of their learning. To protect againstlack of participation by students who do not value the wikitasks prescribed, attaching an assessment weighting tocontributions may be a necessary strategy.

B. Other matters of interestWikis may be an effective tool for facilitating cross abilityinteraction. The lecturer in charge of the weekly extensiontasks noted that all students can participate in contributing tothe problem regardless of their expertise in the area. Moreknowledgeable students can differentiate themselves by


making substantial and indepth contributions. Students whoare less expert in the domain can learn from these more adeptentries, but at the same time can contribute by reorganisingand making minor edits to the submissions of others.

Students may have been more favourable of using wikis intheir courses if they had enrolled for distance courses than forface to face units. The fact that these were face to face unitmeant that some students did not see the point of conductingcollaboration via the wiki. An entirely different set offeedback may be derived from applying these techniques indistance courses where students don't have the capacity tointeract face to face and expect to collaborate using onlinetools.

Students may not be aware of the extent to which the activitiesthey perform within the wikis are facilitating learning.Responses to the survey indicated that students completing thegroup project saw the wiki as a communication tool and not asa learning tool. Explicitly drawing students' attention to theways in which collaborating via wikis is developing theircapacities and understanding may improve their motivationand thus the quality and quantity of their contributions. Aswell, it should be pointed out to students that there is anintrinsic value in learning to interact effectively using wikis; itis a contemporary collaborative competency that may easilybe required in future educational and commercial contexts.

As a matter of determining the suitability of a task to wiki use,researchers wonder whether if assessment is required toensure student engagement, are the tasks being proposedadequately authentic? Also note, adding assessment to wikitasks can have issues related to other forms of onlineasynchronous group contribution tasks, such as the racebetween students to be the first to contribute.

The relationship between wikis and other technologies is aninteresting one. For the Object Oriented Programming subjectthe wiki was adopted within the Moodle LearningManagement System, whereas in the Advanced WebTechnology subject the Jotspot wiki was used as the LMS. Inthe Moodle LMS a discussion board was also used as part ofthe subject's communication offering. In the Jotspot wikidiscussion board type collaborations were occurring on somepages, and indeed Jotspot allows discussion boards to beembedded into wiki pages. As e-learning technologies becomemore sophisticated and the line between them becomes moreblurred, which tools to use and how to integrate them requiresmore consideration. In each case the main point is to identifythe technological requirements of the learning task andprovide a suite of tools that meets those requirements.

Finally, some technical and training matters that arose duringthe study are worth mentioning. First, it should be noted thatrunning several group projects from within the one wiki canlead to problems if two groups wish to create a page with thesame name (for instance, "Timeline"). If only one wiki is to beused for group projects - as opposed to separate wikiinstallations for each group - this potential naming conflict

needs to be pointed out to students. Secondly, care needs to betaken to teach students how to be proficient users ofCamelCase. CamelCase is the formatting convention used bywikis that indicates a piece of text is to link to a new page. Itis especially important to teach IT students how to escapeCamelCase so that they can post streams of computer code tothe wiki without their variable and method names referring tonew pages.

VI. CONCLUSIONAsynchronous e-learning will feature more heavily in coursesof the future because of the temporal and physical flexibilitythat they afford. The question then becomes one of how tomost effectively deploy such technologies, and the mostappropriate circumstances under which to apply the range ofavailable approaches.As wikis become more frequently used in educational settings(including secondary school) it will be less necessary toprovide explicit direction on how to best collaborate usingwiki tools. Many students will have formed the appropriateepistemology and technical skills required to interacteffectively using wikis and will be able to act as exemplars toless proficient students. Until then, however, it seemsnecessary to provide some explicit instruction on how tointeract and "think" in wiki based tasks, yes from a technicalperspective but more so from a collaborative perspective.

Several areas for further research have been raised as a resultof conducting this study:

* Can changing people's learning objective fromindividual learning to form a collaborativeunderstanding of a subject through wikis affect thequality and quantity of contributions to the wiki?

* Does creating this sort collaborative understandingdescribed in the previous point alter the quality oflearning that occurs for different levels of ability (eg,do weaker students improve more substantially butbetter students remain at the same level?)

* How do contributions to a wiki differ fromcontributions to a discussion board for the same task,and to what extent does task type alter the degree ofdifference in patterns of contributions between thetwo tools?

* How do contributions to a wiki differ from face toface contributions for the same task, and how doestask type alter the degree of difference?

* Does the degree of integration of student responsesthat occurs within wikis correlate to studentunderstanding?

* Do learning designs that foster a sense of team andtake place over an extended time frame lead to moreeffective collaboration and learning?

* To what extent do preferences for wiki tasks asopposed to face to face depend on learning stylepreferences? For instance, do oral learners preferface to face collaborations as opposed to textuallearners who prefer wikis? How do other factors such


as face to face confidence and desire for humaninteraction affect student performance?

Like much research into emerging educational technologies,this study has probably raised more questions than it hasanswered.

Researchers involved in this study have identified the capacityfor wikis to facilitate multi-user asynchronous creation,editing and restructuring of information as affordingeducational opportunities that other e-learning tools cannot.The data described herein demonstrates the large variationsthat can occur between tasks and between groups, based onthe task specification and capacities of students to collaborateusing the wikis. This variation indicates that carefulapproaches to educational design and implementation have thepotential to greatly affect the success of learning tasks. It ishoped that sharing the data, analysis, and resultingrecommendations derived from this study allows otheracademics considering deploying wikis in their courses to doso more expertly.

VII. ACKNOWLEDGEMENT

This research was made possible by a Macquarie UniversityStrategic Curriculum Development Grant.

REFERENCES

[1] Collaborative Software Lab College of Computing Georgia Tech. (2000)A Catalog of CoWeb Uses. [Online]. Available:

[2] J. Bergin, "Teaching on the Wiki Web," in, 7th Annual Conference onInnovation and Technology in Computer Science Education, Denmark,2002, pp. 195.

[3] TECFA. (2002) What is a Wiki? [Online]. Available:http://tecfa.unige.ch/guides/tie/html/wikis/wikis-2.html

[4] B. Dodge. (2005) The Incubator - EDTEC wiki. [Online]. Available:

[5] J. Davies. (2004) Wiki Brainstorming and Problems with Wiki BasedCollaboration. [Online]. Available: http://w

_Davies.html[6] M. Guzdial, J. Rick, and B. Kerimbaev, "Recognizing and Supporting

Roles in CSCW," in, ACM Conference on Computer SupportedCollaborative Work, Philadelphia, PA, 2000, pp. 261-268.

[7] E. Tonkin, "Making the Case for a Wiki," ARIADNE, 2005.[8] G. E. Gorman, "Is the wiki concept really so wonderful?," Online

Information Review, vol. 29, pp. 225-226, 2005.[9] IBM. (2003) History Flow. [Online]. Available:

[10] K. Q. Seeyle,"Snared in the Web of a Wikipedia Liar," New York Times,December 4th 2005.


VIII. APPENDIX A-STUDENT SURVEY QUESTIONS

1. What is your student number?

2. Which unit are you enrolled in this semester?(ITEC802, ITEC83 1)

3. On average, how many hours a week did you spendusing the wiki? (Drop-down menu: 1-20+ hours)

4. How much and what type of training did you receivein order to use the wiki?

5. I received adequate training to use the Wiki.(Strongly Disagree, Disagree, Mildly Disagree,Neutral, Mildly Agree, Agree, Strongly Agree)

6. What type and how much training do you think youshould have received in order to use the wiki? (If youfeel like extra training should have been provided,please describe.)

7. What features and functionalities of the wikienhanced your learning experience in this unit?How?

8. What additional features and functionalities wouldyou like to see incorporated into the Wikis in order toimprove your capacity to learn using them?

9. What aspects of the learning tasks set using the wikido you think enhanced your learning experience?(E.g. What did you like about the way this learningtask was designed and run?)

10. How could the learning tasks set in the wiki havebeen improved?

11. Besides using the wiki, in what other ways and howoften did you collaborate with your peers in this unit(e.g. group meetings, emails, formal and informaldiscussions)?

15. What are the disadvantages of using a wiki tocollaboratefor IT related tasks?

Can you see other possible applications for wikis inIT related learning tasks?

16. I was more willing to contribute to the task whenworking through the wiki than if working face-to-face. (Strongly Disagree, Disagree, Mildly Disagree,Neutral, Mildly Agree, Agree, Strongly Agree)(How come?)

17. I was willing to modify and restructure otherpeople's work in the wiki. (Strongly Disagree,Disagree, Mildly Disagree, Neutral, Mildly Agree,Agree, Strongly Agree)

18. For the assigned task, the wiki was a more effectivelearning tool than a discussion board. (StronglyDisagree, Disagree, Mildly Disagree, Neutral, MildlyAgree, Agree, Strongly Agree)

19. In general, what are the advantages of a wiki ascompared to a discussion board for collaborativelearning tasks?

20. In general, what are the disadvantages of a wiki ascompared to a discussion board for collaborativelearning tasks?

21. Using a wiki allowed our group to form a morenegotiated meaning than if we were workingface toface. (Strongly Disagree, Disagree, Mildly Disagree,Neutral, Mildly Agree, Agree, Strongly Agree)

22. Using a wiki allowed our group to form a morenegotiated meaning than if we were using adiscussion board. (Strongly Disagree, Disagree,Mildly Disagree, Neutral, Mildly Agree, Agree,Strongly Agree)

12. I learnt more from my peers through collaboratingvia the wiki than if we were completing the samelearning task face-to-face. (Strongly Disagree,Disagree, Mildly Disagree, Neutral, Mildly Agree,Agree, Strongly Agree) (Describe.)

13. My time was used more efficiently in conductingcollaboration via the wiki than via face to face.(Strongly Disagree, Disagree, Mildly Disagree,Neutral, Mildly Agree, Agree, Strongly Agree)(Describe.)

14. What are the advantages of using a wiki tocollaboratefor IT related tasks?


wiki pedagogy a tale of twowikis · system (lms) in the advanced web technology unit, but within...

Documents