structured online debate and learning

The Effects of Structured Online Debates on Learning

The Florida State University

Li Jin- Instructional Systems

Dr. Allan Jeong- Instructional Systems

Structured Online Debate

Structure online debate is one type of online discussion: One group supports and another group opposes a given position.

Types of messages students can contribute to structured online debates are limited. In our study, four types of messages are specified

based on Toulmin’s (1958) argumentation model: Argument Challenge (critique) Explanation Evidence

Structured Online Debate and Learning--Li Jin, Allan Jeong—The Florida State University

Why Structured Online Debate

Structured online debates can enhance cognitive learning by: Engaging students in specific exchanges –

eliminate off-topic discussions Triggering specified cognitive processes – these

processes otherwise would rarely occur Directing activities toward higher level cognitive

learning – analysis, synthesis, and evaluation (Bloom, 1956)


Studies about Structured Online Debate

Weingerger, et al. (2001, 2004): Greater gains in students’ knowledge of transfer Increased divergence of knowledge and viewpoint

Stegmann, et al. (2007): Increased quality of messages generated by students by

including supporting information, assessments of validity of claims

Schellens, et al. (2007): Higher level of knowledge construction Efforts of testing and constructing new structures

The conclusion is: Structured online debate can enhance learning.


Research QuestionsQ1: To what extent is each level of cognitive

learning achieved within each of the four types of postings observed in the online debates?


Argument

Challenge

Evidence

Explanation

Learning

Learning

Learning

Learning

Knowledge (?%)Comprehension

(?%)Application (?%)

Analysis (?%)Synthesis (?%)Evaluation (?%)

Research QuestionsQ2: Which types of messages are more likely to

demonstrate a particular level of cognitive learning?


Argument

Challenge

Evidence

Explanation

Learning

Learning

Learning

Learning

Knowledge (?%)Comprehension

(?%)Application (?%)

Analysis (?%)Synthesis (?%)Evaluation (?%)

Which is the greatest?

Research QuestionsQ3: What types of exchanges are most likely to

exhibit and trigger subsequent responses that demonstrate a particular level of learning?


Examples of exchanges: Argument ChallengeArgument EvidenceChallenge Explanation……

Higher Level of Learning

AnalysisSynthesis

Evaluation

For each pair on the right list, which on the left list has the strongest association with it?

Data 298 messages from 4 structured online

debates Participants are graduate students enrolled

in an online course in fall 2007.


Debate No. of Participant No. of Posting

1 15 72

2 15 78

3 15 79

4 12 69

Total No. of Posting Analyzed 298

Results Q1-argument: To what extent each level of

cognitive learning has been achieved in argument messages?


Debate

Know Comp App Ana Syn Eva

1 0 11 1 1 2 0

2 0 9 3 9 0 0

3 0 1 1 9 15 0

4 0 7 6 10 3 0

Total 0 31.8% 12.5%

33.0% 22.7% 0

Results Q1-challenge: To what extent each level of

cognitive learning has been achieved in challenge messages?


Debate


1 0 8 6 11 4 1

2 0 5 6 8 1 0

3 0 2 0 26 8 2

4 0 2 6 12 1 5

Total 0 14.9% 15.8%

50.0% 12.3% 7.0%

Results Q1-evidence: To what extent each level of

cognitive learning has been achieved in evidence messages?


Debate


1 0 8 6 3 0 0

2 0 11 9 2 0 0

3 0 2 2 2 2 0

4 0 3 3 2 0 0

Total 0 43.6% 36.4%

16.4% 3.6% 0

Results Q1-explanation: To what extent each level of

cognitive learning has been achieved in explanation messages?


Debate


1 0 4 4 1 0 1

2 0 4 4 7 0 0

3 0 2 1 3 1 0

4 0 3 0 5 0 1

Total 0 31.7% 22.0%

39.0% 2.4% 4.9%

Results Q2: What type(s) of message is more likely to

demonstrate a particular level of cognitive learning?


Restraint

Know

Comp App Ana Syn Eva

Argument

0 9.4%* 3.7% 9.7%* 6.9% 0

Challenge

0 5.7% 6.0% 19.1%*

4.7% 2.7%

Evidence 0 8.0%* 6.7% 3.0% 0.7% 0

Explanation

0 4.4% 3.0% 5.4%* 0.3% 0.7%*significant at .05, compared to 0, does not assume equal variances.

Results Q3: Which message-response pairs exhibited

higher levels of learning? (Top three exchange pairs observed for each higher levels of learning, percentage is over total number of messages of corresponding learning level)


Exchange Analysis Synthesis Evaluation

Argument-challenge 24.3% 29.7% 50%

Challenge-challenge 15.3% 5.4% 30%

Argument-argument 10.8%

Argument-explanation 11.7%

Explanation-explanation 10%

Challenge-explanation 10%

Results Q3: Which message-response pairs exhibited

higher levels of learning? (Top three exchange pairs observed for each higher levels of learning, percentage is over total number of messages of corresponding exchange pairs)


Exchange Analysis Synthesis Evaluation

Evidence-challenge 68.8%

Challenge-challenge 60.7% 10.7%

Challenge-argument 33.3%

Argument-argument 50% 28.6%

Argument-challenge 19.3% 8.8%

Explanation-explanation 100%

Conclusions


1. Lowest level of learning (knowledge) was not observed in structured online debates, which indicates that structured online debates effectively directs learners’ activities to other higher level learning activities.

2. A total of 53.2% of all the postings demonstrate higher level learning (analysis, synthesis, and evaluation). Compared to studies of regular types of online discussion, this indicates structured online debates can effectively enhance higher level learning.

Conclusions


3. Challenge messages are most likely to exhibit higher level learning, including analysis, synthesis, and evaluation, compared to other types of messages.

4. Exchanges that end with challenges are more likely to trigger higher levels of learning.

Use of structured online debate can enhance learning.Encouraging students to challenge other’s ideas or opinions can effectively trigger higher level learning.

Questions? Comments?

Presenter Contact: [email protected]

The Florida State University

Li Jin- Instructional Systems

Dr. Allan Jeong- Instructional Systems

structured online debate and learning

Education