effects of revision training on l2...
TRANSCRIPT
25
English Teaching, Vol. 65, No. 1, Spring 2010
Effects of Revision Training on L2 Writing
Yeon Hee Choi
(Ewha Womans University)∗
Choi, Yeon Hee. (2010). Effects of revision training on l2 writing. English
Teaching, 65(1), 25-56.
This study purposes to examine the effects of two types of revision training (teacher
feedback-aided and protocol-aided revision training) on L2 student writers’ revision and
written product and quality, compared with their self-revision and also to analyze L2
writers’ perceptions on revision training through a pre- and post-training questionnaire
survey. The results of the study show positive effects of both types of training; more
positive effects of teacher feedback-aided revision training on the quality and product of
writing, especially in less proficient writers; more significant effects of protocol-based
revision training on revision frequency, content changes, and success ratio of revision,
especially in less-proficient writers; and more local (content preserving) revisions
triggered by teacher feedback, especially in more proficient writers. Though teacher
feedback-aided revision was favored over protocol-aided revision after training, the
participants’ perception on the former’s positive effects and utilization rate decreased,
but their perception on the latter’s positive effects and utilization rate enhanced. The
present study suggests protocol-aided revision as an alternative revision method, and
different revision methods depending on the focus of revision and L2 writing
proficiency.
I. INTRODUCTION
With the increasing focus on writing process, revision has become a main issue in L2
writing class (Ferris, 1995, 1997; Hyland, 1998; Min, 2005, 2006; Sachs & Polio, 2007;
Zamel, 1982, 1983). It has been studied with a variety of topics including differences in L1
and L2 writing (Hall, 1987, 1990; Stevenson, Schoonen, & Glopper, 2006); the effect of
writing tasks/types (Lindgren & Sullivan, 2006b); revision mode (e.g., computer-
∗ This work was supported by the Korea Research Foundation Grant funded by the Korean
Government(KRF-2008-2366-1).
Yeon Hee Choi
26
mediated) (Phinney & Khouri, 1993; van Waes & Schellens, 2003); the effect of revision
on writing quality (Yeonhee Choi, 2006, 2007; Stevenson et al., 2006); and revision
strategy instruction (Sengupta, 2000); and the benefits of peer review (Lundstrom & Baker,
2009). These studies of L2 writing on revision illustrate that generally in L2 writing,
especially in demanding writing tasks, the frequency of revision is higher with a focus on
linguistic forms, and this is salient among learners with low writing proficiency.
Reader awareness is one of the important factors influencing revision. Previous studies on
revision have shown that writers need to be good readers in order to revise their writing from
the reader’s viewpoint (Hayes, 1996). Thus, ‘reading as the reader’ has been considered
important in revision. Research on reader awareness illustrates that proficient writers
examine if their readers can understand their intentions correctly (Hayes, Flower, Schriver,
Stratman, & Carey, 1987) and that skilled writers are more aware of potential readers in
their writing (Hayes et al., 1987; Sommers, 1980). However, whether reader awareness
influences the quality of writing and revision is still inconclusive in L2 writing (Yeonhee
Choi, 2006; Jourderais, 2001; Holliway & McCutchen, 2004; Sachs & Polio, 2007).
Furthermore, how to teach L2 writers to revise effectively has not been explored much,
except in a few studies on revision strategy instruction (Sengupta, 2000) and peer feedback
training (Berg, 1999; Cho, 2005; Liou & Peng, 2009; Min, 2005, 2006; Stanley, 1992).
Sengupta’s (2000) investigation of revision strategy instruction illustrated its positive
effects on L2 secondary learners’ writing performance. Other studies such as Stanley
(1992), Berg (1999), Yunkyoung Cho (2005), Min (2005, 2006), and Liou and Peng (2009)
focused on peer review training and demonstrated that training for peer review was helpful
for revision. However, training for other revision methods such as teacher feedback-aided
or protocol-aided revision has not been explored yet.1
Therefore, the present study purposes to devise a revision training model including
teacher feedback-aided and protocol-aided revision by raising reader awareness. The effect
of two revision training methods (teacher feedback-aided and protocol-aided revision
training) will be compared in terms of quality of writing, written product, revision frequency
and type, and L2 writers’ perceptions on revision training through a pre- and post-training
questionnaire. Effects of revision training will also be compared with self-revision. In
addition, the effects will be examined across L2 writing proficiency levels since it has been
noted that revision methods differ by L2 writing proficiency levels (Yeonhee Choi, 2007;
Zamel, 1983).
The research questions for the study are as follows.
1. What are the effects of teacher feedback-aided and protocol-aided revision training
1 Protocol-aided revision is recommended in L1 writing as a way of increasing reader awareness
(Hayes, 2000). However, it has not been used much to revise writing in L2 as well as in L1.
Effects of Revision Training on L2 Writing
27
on the quality and product of L2 student writing, compared with self-revision? Does
the influence of revision training vary with L2 writing proficiency?
2. What are the effects of teacher feedback-aided and protocol-aided revision training
on the frequency and types of revision, compared with self-revision? Does the
influence of revision training vary with L2 writing proficiency?
3. What are the effects of teacher feedback-aided and protocol-aided revision training
on the success and utilization rate of revision, compared with self-revision? Does the
influence of revision training vary with L2 writing proficiency?
4. What are the effects of teacher feedback-aided and protocol-aided revision training
on L2 student writers’ perception on revision methods?
II. PREVIOUS STUDIES ON REVISION TRAINING
Revision has drawn a great deal of attention in L1 and L2 writing research; however,
there is not much research on revision training, except for studies on peer feedback
training (e.g., Stanley, 1992; Berg, 1999; Min, 2005, 2006). Since a good writing in
English is defined as a text clear to the reader, reader-oriented writing is essential (Hayes,
1996), especially in revision, as Jones and Myhill (2007) suggest that the process of
revision itself requires more understanding of the reader. Writers with higher writing
proficiency appear to revise more often and have the ability to consider potential readers of
their writing (Hayes et al., 1987; Sommers, 1980). Utilizing feedback provided by the
teacher or the peer is a way to revise writing with a consideration on the reader since they
play the role of the reader when responding to the writing. Although some studies have
concluded that teacher feedback is not always useful (Truscott, 1996), in general, teacher
feedback is recognized most effective when it is provided during the writing process
(Ferris, 1995; Leki, 1990; Zamel, 1985). L2 learners prefer teacher feedback (Huyyun
Choi, 2002; Hedgcock & Lefkowitz, 1994; Lee, 2008; Nelson & Carson, 1998; Yang,
Badger, & Yu, 2006), that is, they appear to prefer specific teacher feedback that includes a
concrete suggestion (Ferris, 1995, 1997, 2003). Writing teachers’ corrective feedback has
gained more interest in recent research (Bitchener, 2008; Bitchener & Knoch, 2009; Ellis,
Sheen, Murakami, & Takashima, 2008). But, they lack clear evidence whether L2 writers
successfully incorporate teacher feedback, that is, whether feedback results in a higher
quality of the revised writing, and ultimately whether it improves students’ writing
proficiency (Conrad & Goldstein, 1999; Ferris, 1997, 2003; Goldstein & Conrad, 1990;
Hyland & Hyland, 2001). Furthermore, training on how to incorporate teacher feedback
effectively has not been specifically explored yet.
Yeon Hee Choi
28
Another way to enhance writers’ awareness of potential readers and help writers become
strategic readers is protocol-aided revision, in which the writer verbalizes whatever comes
into his/her mind while revising. Its positive effects have been revealed in Jourderais
(2001), and Stratman and Hamp-Lyons (1994), such as increased concentration and
learning opportunities, and more accurate revision; however, Sachs and Polio’s (2007)
study in L2 writing illustrates a negative effect on the accuracy of revision. Thus, it is still
uncertain on how think-aloud, that is, protocol-aided revision, can assist the L2 revision
process. In addition to the inconclusive results of the effects of protocol-aided revision,
there is not much research on how to teach L2 writers to become strategic readers in the
writing process.
Therefore, the present study aims at investigating L2 writers’ revision process when they
consider their potential readers, and how revision or quality of the writing changes in
accordance with the writing process. Specifically, the study purposes to devise a revision
training model using teacher feedback-aided and protocol-aided revision by letting the
students become their own readers. The effect of these revision training methods will be
compared with that of self-revision. Ultimately, the study aims at finding an effective
revision method for L2 writers by examining how the two types of revision training affect
the quality of the writing in English after revision training, and how L2 writers evaluate
their revision training.
III. RESEARCH METHOD
1. Participants
Eighteen Korean university students voluntarily participated in the study, including
eight seniors, three juniors, and seven freshmen.2 They were all female, ranging in the age
of 20 to 25. All of the participants were majoring or double-majoring in English Education
(Teaching English as a Foreign Language). Most of them had some experience of learning
composition in English or Korean. All of them stated in the pre-questionnaire that they had
some experience of revising their own writing, but only about a half of them received
2 The study had planned to have two independent groups to compare the impact of revision training
using two revision methods. But, it failed to find EFL writing teachers who were willing to take part in the research. Thus, L2 student writers were recruited. The number of the voluntary participants was not large enough to have two independent groups; thus, the training was administered to one group. The participants might have had more positive attitudes toward English writing or revision, compared to students of a regular English writing course.
Effects of Revision Training on L2 Writing
29
formal instruction on revision. Based on the pretest score (Writing 1),3 which was scored
by the Test of Written Examination (TWE) scoring criteria (six scales), the participants
were classified into three groups: high-level proficiency (HP) group (five students) (M =
4.08), intermediate-level proficiency (IP) group (nine students) (M = 3.16), and low-level
proficiency (LP) group (three students) (M = 2.17).4 In the analysis of the data, one of the
seniors’ data was excluded because she didn’t complete all the requirements.
2. Materials
The research materials included four writing tasks for the pretest and training, four
sample writings for training, a pre-questionnaire for the participants’ background
information, and two post-questionnaires.
2.1. Writing Tasks
Four argumentative writing tasks were chosen from the prompts of the TWE developed
by ETS, as shown below.5
Writing 1: preference of class type
Writing 2: the issue of athletes’ and entertainers’ high salaries
Writing 3: preference of study type
Writing 4: the most important characteristic in successful life
The TWE prompts were selected since they could be relevant and familiar to the participants.
The first draft of Writing 1 was used as a pretest to divide participants by writing proficiency.
Writing 1 and Writing 3 were the main writing tasks and Writing 2 and Writing 4 were tasks
for in-class revision training sessions. In order to compare revisions in the main tasks, the
prompts of the main writing tasks were similar to each other. The prompts of tasks for
training were also similar to each other. The participants were required to complete each task
within 30 minutes, which is the time limit used in the TWE. However, no time limit was set
for revision. In addition, no word or paragraph limits were set for the writing tasks.
3 One of the participants wrote the essay completely off the given prompt of Writing 1, which was
intended to be used both as a pretest and as one of the main writing tasks. Thus, there were some problems to classify the participants based on the scores of Writing 1 only. As a result, the participants were classified by the combined score of Writing 1 and Writing 2, which was planned to be used for teacher feedback-aided revision training.
4 The number of the low-level proficiency group was very small, which is a main limitation of the research.
5 The topic difficulty was not measured.
Yeon Hee Choi
30
2.2. Sample Writings for In-class Revision Training
For the in-class revision training, four writing prompts were also selected from the TWE,
as shown below.
Sample Writing 1: preference of morning or night person
Sample Writing 2: preference of communication type
Sample Writing 3: the issue of mass media’s attention to public figures and celebrities
Sample Writing 4: the issue of smoking in public places
Sample writings were needed to be used for in-class training, that is, for teacher
demonstrations and group revision work; thus, Korean speakers were recruited to write
sample writings. Their TOEIC scores ranged from 800 to 850. The average length of sample
writings was 234 words, which was shorter than the average length of the pretest of the
participants (311 words). Sample Writing 1 and Sample Writing 2 were used for teacher
feedback-aided training, and thus during the training session teacher feedbacks were
provided in terms of grammar, expression, content, paragraph structure, and overall
organization.6
2.3. Questionnaires
A pre-questionnaire and two post questionnaires were constructed for the study. The pre-
questionnaire was consisted of two sessions. The first part was constructed to obtain
information on the participants’ experience of learning English writing and revision, English
proficiency scores including the TWE score, and the length of residence in English-speaking
countries (five yes-no questions, two choice questions, and 13 open-ended questions). The
second part included questions on their perceptions of teacher feedback-aided and protocol-
aided revision. It contained six 6-point Likert-scale items, two choice questions, and eight open-
ended questions on the participants’ attitudes towards the two types of revision including
their usefulness, foci, difficulties and problems, exploitability, and preference for long-term
effects, which were compared with those after the two training sessions. Two post-questionnaires
were also constructed to examine the participants’ perception of the two revision training
methods. While the questions were identical to those in the second part of the pre-training
questionnaire, they included open-ended questions on suggestions for revision training methods.
6 The teacher feedback in the main writing task and sample writings consists of three parts:
underlining to indicate ungrammatical or awkward expressions, marginal notes stating frequently made errors among underlined errors, and endnotes expressing the general comments on the essay including the comments on the paragraph structure, organization, and the content.
Effects of Revision Training on L2 Writing
31
3. Data Collection Procedures Including Revision Training
The revision training was conducted by a native writing teacher with a doctoral degree
in TESOL for five weeks, between March 9 and April 11, 2009. The teacher had taught
university-level English composition for five years. All the participants were trained
through two revision training types. Two sessions of revision training were scheduled for
each type of revision training on the second and fourth week of the revision training
experiment (see Figure 1).
FIGURE 1 Revision Training Procedure
Week 1 Pre-questionnaire for background information and participants’ experience and
perception on teacher feedback-aided and protocol-aided revision
↓
Writing 1 (1st draft)
↓
Writing 1 self-revision (W1-SR)
↓
Writing 2 (1st draft)
↓
Week 2 In-class revision training 1 for utilizing teacher feedback
- Class 1: teacher’s revision demonstration of Sample Writing 1 and individual
and group revision of Sample Writing 2
- Class 2: individual revision of Writing 2 based on teacher feedback
↓
Writing 3 (1st draft)
↓
Writing 3 self-revision (W3-SR) ↓
Week 3 Writing 1 revision utilizing teacher feedback (W1-TF)
Post-questionnaire about teacher feedback-aided revision training ↓
Writing 4 (1st draft) ↓
Writing 4 self-revision (W4-SR) ↓
Week 4 In-class revision training 2 for protocol-aided revision
- Class 3: teacher’s revision demonstration of Sample Writing 3 and group
revision of Sample Writing 4
- Class 4: individual think-aloud revision of Writing 4 ↓
Week 5 Writing 3 revision aided by think-aloud protocol (W3-TA)
Post-questionnaire about protocol-aided revision training
The first training session for teacher feedback-aided revision was held in the second
week and it consisted of two two-hour training classes. The second training session for
protocol-aided revision was held on the fourth week and the number of classes and the
Yeon Hee Choi
32
length of the class were the same as the first training session. In other words, each student
received about 8 hours of in-class revision training. Teacher feedback-aided revision
training preceded protocol-aided revision training since the majority of the participants had
no experience of protocol-aided revision in English writing and it was assumed that
protocol-aided revision might be more demanding.
Before the training, a specific guideline describing the training schedule and the
requirements were distributed to the participants via e-mail. The participants were asked to
submit a pre-questionnaire, the first drafts of Writing 1 and Writing 2, and the self-revised
draft of Writing 1 (W1-SR) by the end of the first week. In the second week the students
had two classes of in-class revision training using teacher feedback.
The first in-class revision training was designed to model how to revise writing based on
teacher feedback. The teacher demonstrated how to revise writing utilizing various types
of teacher feedback such as grammar, vocabulary, expression, content, and organization, with
Sample Writing 1. (See the teacher feedbacks provided with Sample Writing 1 in Figure 2.
They include specific marginal comments on language problems, end notes on content,
organization, and overall problems. All problematic words or phrases are underlined.)
Then the students were asked to revise Sample Writing 2 based on teacher feedback, first
individually and then in a group. After revising Sample Writing 2, the students made
presentations on how they had revised it and submitted their revised drafts. The second in-
class training aimed to give further teacher feedback-aided revision training to the
participants. The students revised their own second writings (Writing 2) in class, which
were marked with teacher feedback. The teacher and a research assistant moved around to
answer questions about how to utilize teacher feedback in their revision.
After the two classes of training, the participants submitted the first and self-revised
draft of Writing 3. In the third week, they submitted the teacher feedback-aided revision of
Writing 1 (W1-TF) with the first post-questionnaire about teacher feedback-aided revision
training, and also the first and self-revised draft of Writing 4.
The second in-class revision training was designed to model how to revise writing doing
think-aloud. The basic structure of the second in-class revision training session (the third
and fourth class in Figure 1) was similar to the first one. During the third class, the teacher
first introduced the concept of think aloud and pointed out the importance of reader
awareness in revision process and then she demonstrated how to revise Sample Writing 3
while she thought aloud, as shown in Figure 3.
After the teacher’s demonstration, the students were asked to revise Sample Writing 4
by taking turns in a group while they thought aloud. Revising while thinking aloud was not
conducted individually due to the fact that most of the computers available were too slow
to be used effectively. In a group, the members thought aloud in turn while revising. Their
think-aloud was all recorded using MP3 players. While the participants were engaged in
Effects of Revision Training on L2 Writing
33
FIGURE 2 Teacher Feedback on Sample Writing 1 and Teacher’s Revision
Original Draft Teacher Feedback
Teacher Feedback-Aided Revised Draft
I prefer to wake up early in the morning and I prefer to start the day’s work.
Because I believe getting up early in the morning is a more suitable schedule for Human’s life cycle, and it’s also better for our health. However there is a gap between preference and my real life. Logically, I prefer to get up early in the morning and start the day’s work as I mentioned above, but, practically I usually stay my bed just before I can go to work on time. And usually I get up later in the day and stay my bed all day long on weekend. I am doing nothing. I prefer to stay at home and watch movies instead of do work. Therefore there are a lot of efforts to support my preference. For example I usually go to bed before 10p.m to get up early in the morning tomorrow and take some vitamin supplements for increase physical strength. …… So I prefer to wake up early in the morning even if sometimes it is difficult to get up early in the morning and start the day’s work.
need comma after transition missing preposition
verb form word order
I prefer to wake up early in the morning and start the day’s work. I believe getting up early in the morning is a more suitable schedule for human’s life cycle, and it’s also better for our health. However, there is a gap between preference and real life. Logically, I prefer to get up early in the morning and start the day’s work, but actually I usually stay in my bed just before I go to work on time. Nevertheless, I still believe it is important to wake up early in the morning. Waking up early helps me feel
refreshed in the morning. When I get up early in the morning, I can spare time for reading a newspaper and thinking about the day’s work. So in order to get up early in the morning, I try to do my best. For example, I usually go to bed before 10 p.m. to get up early in the morning and take some vitamin supplements for increasing physical strength. And I do not meet my friends during weekdays. If I meet my friends, I try to get home by 9:30. … … work. So I prefer to wake up early in the
morning because it helps me feel refreshed in the morning and prevents me from being lazy on weekends. Sometimes it takes a lot of efforts for me to wake up early, but I think it is worthwhile making efforts to do so.
Comments 1) no clear paragraphing/no indentation
2) no clear structure/organization: introduction (x), body (x), conclusion (x)
3) no clear thesis statement
4) no clear topic sentences/supporting sentences
5) sentences contradict each other
6) sentences do not provide examples to support writer’s choice; rather, they give a description of a
daily routine
the think aloud process, the teacher and a research assistant moved around to remind the
participants to do think aloud when they were silent for more than 15 seconds. However,
they tried not to interfere with the participants’ thinking process. After completing the
revision, the participants listened to their own think-aloud protocols recorded in a group
and discussed to what extent they were aware of readers and to what extent they thought
Yeon Hee Choi
34
aloud about their revision process. In the fourth class the students had opportunities to
individually revise their own writing (Writing 4) while thinking aloud. After the
protocol-aided revision training, the participants submitted their revised draft of Writing
3 (W3-TA) with their think aloud audio files and the second post-questionnaire about
protocol-aided revision training (in the fifth week).
All writing tasks and questionnaires were submitted via e-mail. When the participants
revised Writing 1 and Writing 3, they were asked to revise their first draft without referring
to their self-revision. There was a 2-week interval between self-revision and post-training
revision in order to minimize the influence of self-revision on post-training revision.
FIGURE 3 Teacher’s Demonstration of Protocol-aided Revision
Original draft Think-Aloud Protocols Protocol-aided
Revised draft
Okay, uhm I’m going to demonstrate think aloud for all of
you ……, the purpose of doing this think aloud is to um,
develop a, an awareness of, you know, the fact that
whenever you write anything at all, particularly an
argumentative essay, you have to understand that you have
a reader. Okay, you have a reader. …... So……so you are
going to have a uh, imaginary conversation with your
reader. ……, so you have to imagine that there is
somebody sitting in front of you and that person is…your
reader, and that person is constantly throwing questions
about the essay. What’s wrong with this sentence, how
should I fix it, why did this person write this way. So try to
ask as many questions as possible and you’re going to try
to answer it, and as you have this imaginary conversation
going on, you’re going to revise the essay
simul…simultaneously, okay? ……
I agree that nowadays
television,
newspapers,
magazines, and other
media pay too much
attention to the
personal lives of
public figures and
celebrities. …… in
front of others.
“I agree that nowadays television, newspapers, magazines,
and other media pay too much attention to the personal
…… that they stand naked in front of others.” I think…I
think this introduction for one thing um<1>starts with the
sentence. The first sentence, it has a lot of problems
because the…prompt is …… So I’m going to…take
this…sentence here, which actually belongs at the end of
the paragraph, and it should…be placed at the end.
So…that…is going to function as my thesis statement.
So<2>now I have to delete this sentence…and…the
beginning of the paragraph is now a little bit confusing
…… But as a leading statement it sounds very confusing
so I’m going…I think I need to add something. I need to
add some kind of statement to give a hint what this whole
essay is going to be about.
Effects of Revision Training on L2 Writing
35
I agree that nowadays
television,
newspapers,
magazines, …… and
celebrities.
{hear typing sound} Do celebrities and public
figures…have a right to maintain their private lives or
should…they…reveal their personal lives to the general
public? So I took the thesis statement I think…and then put
it at the end of paragraph so that it shows the
reader…uh…shows the reader what my position is which
means, which my position I’m trying to argue, that the
media pays too much attention to the personal lives.
But…at the end of intro…at the beginning of the
introduction I start by saying or asking a question uh…to
let the reader know what the essay is going to be about in
general.
Do celebrities and
public figures have a
right to maintain their
private lives, or
should they reveal
their personal lives to
the general public?
I think that is
certainly an act of
invading other’s
privacy.
Um… “I think that is certainly an act of invading other’s
privacy.” This sentence doesn’t seem to connect with the
leading sentence for some reason. So I have to make it a
little bit clearer to connect with the first sentence. The first
thing I’m going to do is change…the statement because I
don’t like the fact that it is in the first person pronoun which
doesn’t sound very academic and professional. So I’m
going to try to connect it to the sentence before. That comes
before.
I think that is
certainly an act of
invading other’s
privacy.
{starts typing} Taking too much interest in the lives of
famous people is certainly an act of invading
other’s…privacy. So I didn’t change the sentence that much
but I think I got rid of…using too much first person
pronouns because it doesn’t sound very professional. I
changed the sentence so that I can avoid using ‘I’ in the
sentence. Uh…and then I just tried to keep the rest of the
sentence the content of the rest of the sentence here.
Taking too much
interest in the lives of
famous people is
certainly an act of
invading other’s
privacy.
Note: think-aloud protocols are italicized, except for the original sentences read and the sentences
typed while verbalizing; six dots indicate parts not specified in the figure.
However, some of the participants did not follow the instruction and they referred to their
self-revision when they did post-training revision. In those cases, the changes already
made in self-revision were not counted as post-training revisions.
4. Data Analysis
The participants’ writing was holistically scored using the TWE CBT/PBT scoring
criteria (6 scales) by the researcher and a native composition teacher who conducted the
in-class revision training. The participants were classified into three L2 writing proficiency
levels based on their pretest scores, as mentioned before.
Before revision coding, the participants’ think-aloud files were transcribed into think-
Yeon Hee Choi
36
loud protocols.7 The transcription was based on the way the participants spoke. This
means that spellings were coded in the way they were articulated.
After counting all the revisions before and after training, revisions were also coded in
terms of their purpose, which was based on Faigley and Witte (1981), and Lindgren and
Sullivan (2006a). First, a distinction was made between revisions that changed the
meaning of the text (content change: CC) and those that did not (content preserving: CP).
The revisions that did not change the meaning of the text were then classified into five
categories: paraphrase, vocabulary, grammar, mechanics (punctuation, capitalization,
paragraph indentation, and spelling correction), and style/tone changes. The category of
style/tone illustrated the revision where the writer tries to make his/her style or tone
neutral.8 The content changes were coded at two levels: 1) micro-level changes (MI)
included addition of new content, deletion of existing content, or substitution of the
sentences; and 2) macro-level changes (MA) were major changes which influenced the
summary of a text.
In addition to coding in terms of revision frequency and types, the degree of success and
the utilization of teacher feedback and think aloud were also coded (see Appendix 1). In
case of teacher feedback, the coding scheme was divided to four categories: ‘utilization
(U),’ ‘no utilization (NU),’ ‘not known (NK),’ and ‘self-revision.’ The term ‘utilization’
refers to a revision triggered by teacher feedback, regardless of the degree of success. The
term ‘no utilization’ refers to a case that the writer does not refer to teacher feedback at all
when revising. The term ‘not known’ refers to a case that the writer already corrected
errors in self-revision prior to teacher feedback. The term ‘self-revision’ is a case that the
writer makes a self-revision though there is no teacher feedback on it. The ratio of
utilization of teacher feedback was calculated by the total number of utilization divided by
the total number of revision.
The utilization of think aloud was coded in terms of three categories: ‘revision while
think aloud (TA-R),’ ‘revision after a pause longer than one second (P-R),’ and ‘revision
7 When the think-aloud files were transcribed, a set of symbols was used: a question mark for a
rising intonation at the end of a phrase; a period for a closing of an idea unit, or an utterance; three dots for a pause shorter than 1 second; the brackets < > for a pause longer than 1 second without any verbalization; the number within < > for the pause length in second; the brackets { } for observation; underlining for the verbalization made while the writer is writing the text; and quotation marks for what the writer was reading aloud, the writing prompt or previously written production.
8 A senior student in the HP group (H4-1) changed the sentence of “If I study in group, I have a chance to contact with other people’s thoughts” to the sentence “It gives more chance to contact with other people’s thoughts.” She deleted the word ‘I’ and rewrote the sentence from the third- person point of view; thus, it sounded more neutral. This kind of revision was coded as a style/tone change. (In the present study, when a participant is specified, the number after the group initial (L, I, and H) refers to her university year (e.g., 4 for senior); and the number after the hyphen is the number of the participant within the group.
Effects of Revision Training on L2 Writing
37
without any noticeable action (N-R).’ The term ‘revision while think aloud’ refers to a
revision triggered by think aloud, that is, a case that the writer revises his/her writing while
doing think aloud. The term ‘revision after pause longer than one second’ is a case that the
writer makes a revision after a pause longer than one second without any verbalization.
The term ‘revision without any noticeable action’ is a case when the writer revises, that
is, he/ she makes a revision while reading the first draft itself without any verbalization on
his/her thinking process. The ratio of utilization of think-aloud was calculated by the total
number of revision while think aloud divided by the total number of revision.
The degree of success was divided into three categories: ‘successful (S),’ ‘unsuccessful
(US),’ and ‘no difference/improvement (ND/I).’ The term ‘successful’ indicates that
revision enhances the quality of writing in terms of revision purpose (e.g., grammar,
vocabulary, and content) (see Appendix 1). The term ‘unsuccessful’ indicates that revision
fails to improve the writing in terms of its purpose. The term ‘no difference/improvement’
indicates that there is no significant difference or improvement after revision.
The success ratio of revision was measured by the total number of successful revision
divided by the total number of revision, whereas the failure ratio was measured by the total
number of failed revision divided by the total number of revision. The success ratio of
teacher feedback-triggered (U) revision was calculated by the total number of successful
teacher feedback-triggered revision divided by the total number of teacher feedback-
triggered revision. That of revision triggered by think-aloud was measured by the total
number of successful revision while think aloud divided by the total number of revision
while think aloud.
After two research assistants coded revision in a student’s self-revised and post-training
revised drafts with the researcher, one of the research assistants coded the self-revision and
teacher feedback-aided revision of Writing 1, while the other assistant coded the self-
revision and protocol-based revision of Writing 3. After completing the coding, they cross-
examined their coding. When there were any discrepancies about the coding, they
discussed the matters, sometimes with the researcher, and resolved the discrepancies. After
revisions were coded and cross-examined, their frequencies were counted according to
revision types. The frequency of each revision category per 100 words of the original text
besides ratio of each revision category in each revision type was counted to compare pre-
and post-training revisions and also revisions across three L2 writing proficiency groups.9
This method was used because those who write longer would revise more.
9 When revision frequency was counted per 100 words, Hall (1990), New (1999), Stevenson et al.
(2006), and Yeonhee Choi (2007) used the total number of the revised draft. In the present study, however, revision frequency was divided by the total number of the original draft to make a comparison between self-revision and post-training revision.
Yeon Hee Choi
38
Variations across proficiency levels were analyzed in writing quality, written product,
revision frequency and ratio, using one-way ANOVA.10 The ratio of content preserving
(CP) and content change (CC) revision was compared in each revised draft by Wilcoxon
test because the sample size was small and equal variance was not assumed.11 The
participants’ perception on revision training before and after training was also analyzed by
Wilcoxon test.
IV. RESULTS AND DISCUSSION
1. Quality and Product of Pre- and Post-Training Writing
The writing of each participant for the two main writing tasks including their self-
revised (pre-training) drafts (W1-SR and W3-SR) and their post-training revised drafts
(W1-TF and W3-TA) (total three writings per training) was scored holistically using six
scales to see the impact of revision training on the overall quality of writing. As shown in
Figure 4 and Table 1, the scores gradually increased from the first draft to the post-training
revised draft, especially after teacher feedback-aided revision training. Such a positive
effect was also noted in the studies on teacher feedback (Ferris, 1997; Hyland, 1998; Jihye
Lee, 2003; Jeong-Won Lee & Young-Joo Hong, 2001) and peer feedback training (Berg,
1999; Min, 2006; Sengupta, 2000). The score difference was much more noticeable
among W1, W1-SR, and W1-TF (in the teacher feedback-aided revision training) than
among W3, W3-SR, and W3-TA (in the protocol-aided revision training). The score
difference between W1 and W1-TF was larger from that between W3 and W3-TA. These
results suggest a more positive effect of teacher feedback-aided revision training. However,
a caution is required in such interpretation since the effect of the revision training order
(teacher feedback-aided revision training first and then protocol-aided revision training)
may not be excluded.
Moreover, the overall writing quality was compared before and after revision training
across the three L2 writing proficiency groups, as shown in Table 1, to see whether the
impact of revision training varies with L2 writing proficiency. The ANOVA results
show a significant difference in each of the first, self-revised, and post-training drafts in
10 The sample size was too small to run a statistical analysis, which is one main limitation of the
study. The lower-level students were only three students, as mentioned before. Nevertheless, proficiency group variations were analyzed by one-way ANOVA to see the impact of two different types of revision training across L2 writing proficiency groups.
11 No statistical analysis was conducted on writing quality, written product, and revision frequency and ratio between post-training teacher feedback-aided and protocol-aided revision nor between these revisions and self-revision because the repeated treatment effects could not be excluded.
Effects of Revision Training on L2 Writing
39
FIGURE 4 Pre- and Post-Training Writing Scores
TABLE 1 ANOVA Results of Writing Scores by L2 Writing Proficiency and Post-hoc Analysis
Proficiency Level
Writing Score LP IP HP F
M SD M SD M SD
W1 score 2.17 .76 3.16 .35 4.08 .24 20.925***
A B C
W1-SR score 2.50 .50 3.28 .50 4.20 .27 14.326***
A B C
W1-TF score 2.90 .85 3.49 .43 4.20 .27
7.329**
A B
W3 score 2.83 .76 3.74 .53 4.04 .15 5.603*
A B
W3-SR score 2.90 .85 3.83 .49 4.04 .15 5.269*
A B
W3-TA score 2.90 .85 3.89 .54 4.04 .17 4.953*
A B
Note: A, B, and C indicate grouping in the post-hoc analysis in all the tables henceforth.
* p < .05, ** p < .01, *** p < .001
both training methods. The post-hoc analysis shows that the HP group outperformed the
IP and LP groups in the first draft and self-revised draft before the teacher feedback-
aided revision training, but no significant difference was noted between the HP and IP
groups in all the three drafts before and after protocol-aided revision training as well as
3.2
3.4
3.6
3.8
(writing)
(score)
Original
draft
Self-revised draft
(SR)
Post-training revised
draft (TF/TA)
3.25
3.41
3.59
3.67
3.733.76
(W3)
(W1)
3.2
3.4
3.6
3.8
(writing)
(score)
Original
draft
Self-revised draft
(SR)
Post-training revised
draft (TF/TA)
3.25
3.41
3.59
3.67
3.733.76
(W3)
(W1)
Yeon Hee Choi
40
the post-training draft of teacher feedback-revision training. The LP and IP group’s score
of W3 (the original draft before protocol-aided revision training) was much higher than
that of W1 (the original draft before teacher feedback-aided revision training). The score
difference between the first draft (W1) and the post-training draft (W1-TF) was much
larger in the LP and IP group than the HP group (LP, M = .73; IP, M =.33; and HP, M = .12).
Compared to the LP and IP group, the HP group’s scores for all the six drafts did not
vary much. These results seem to illustrate a more positive effect of teacher feedback-
aided revision training on the LP and IP group, the impact of revision training sequence
(writing and revision practice) and the ceiling effects.
The length of student written products was measured in terms of the total number of
words and sentences, as shown in Table 2. The number of words and sentences tended to
increase from the first draft of W1 to the draft of W3-TA (total number of words from
269.53 to 334.00; that of sentences from 15.00 to 20.18). These results imply that the
participants wrote longer with more sentences over the training period. The revised
drafts (W1-TF, 304.88 words and 16.88 sentences; W3-TA, 334 words and 20.12
sentences) were much longer than the first drafts (W1, 269.53 words and 15 sentences;
W3, 307.12 words and 18.65 sentences), but not than the self-revised drafts (W1-SR,
305.35 words and 16.82 sentences; W3-SR, 327.29 words and 19.59 sentences). Such
text length difference was more noticeable in teacher feedback-aided revision training
than in protocol-aided revision training. Interestingly, the first draft before protocol-
aided revision training (the second training) was slightly longer than post-training
teacher feedback-aided revision (W1-TF, 304.88 words; W3, 307.12 words). This also
suggests the effect of writing and revision practices throughout the two sessions of
revision training.
A significant variation in text length (total number of words and sentences) was also
noted in the results of one way ANOVA among the three proficiency groups in the drafts of
W1, W1-SR, and W1-TF (all of them were significant below the .01 level), because of a
significant difference between the LP and HP groups (see Table 2). Such a difference
was not found in those of W3, W3-SR, and W3-TA. This suggests that the three groups’
text length became similar after the first revision training. The LP group’s post-training
draft was much longer in teacher feedback-aided revision training, compared with the
other groups and also their post-training protocol-aided revision. Interestingly, the IP group
wrote longer than the HP group in W3, W3-SR and W3-TF. These findings imply a
significant benefit of the first revision training (teacher feedback-aided revision training)
for the LP and IP group. Moreover, they suggest writing and revision practices, as noted in
the writing scores. Before the participants submitted W3, they wrote the original, self-
revised, and post-training revised draft of W1, the revised draft of Sample Writing 2 during
the onsite training, and the original and revised draft of W2, a total of six pieces of writing.
Effects of Revision Training on L2 Writing
41
TABLE 2 ANOVA Results of Writing Products by L2 Writing Proficiency and Post-hoc Analysis
Proficiency Level Length of Written Product LP IP HP F
M SD M SD M SD
Total No. of words in W1 139.00 68.55 272.11 56.91 343.20 78.56 9.137**
A B
Total No. of words in W1-SR 175.33 26.01 305.67 60.49 382.80 64.71 11.926**
A B
Total No. of words in W1-TF 217.33 70.81 298.56 42.38 368.80 61.36 7.768**
A A B
Total No. of words in W3 241.67 112.08 328.11 95.07 308.60 32.49 1.159
Total No. of words in W3-SR 256.00 102.56 358.22 99.87 314.40 23.52 1.677
Total No. of words in W3-TA 247.67 99.03 362.89 99.35 333.80 34.05 2.026
Total No. of sentences in W1 9.33 3.06 14.56 2.19 19.20 4.97 8.377**
A B
Total No. of sentences in W1-SR 11.33 1.15 15.67 3.35 22.20 4.09 10.843**
A B
Total No. of sentences in W1-TF 15.00 3.00 14.78 2.82 21.80 3.96 8.312**
A B
Total No. of sentences in W3 18.00 8.00 19.11 4.68 18.20 2.17 .091
Total No. of sentences in W3-SR 18.67 8.62 20.44 4.16 18.60 2.30 .319
Total No. of sentences in W3-TA 18.33 8.50 20.67 4.36 20.20 3.77 .244
** p < .01
Such writing and revision practices seemed to help the IP group write better before they
started the second revision training.
2. Frequency and Ratio of Pre- and Post-Training Revision
The total number of pre-training (self-revision) and post-training revision (R) in both
teacher feedback- and protocol-aided revision training (see Figure 5) illustrates that all the
participants made more revision after the training regardless of the revision methods,
except for the IP group after teacher feedback-aided training (see Table 3). Revision
frequency per 100 words before and after training displays a similar pattern noted in the
total frequency: a larger number of revision per 100 words after both teacher feedback-
and protocol-aided revision training (see Figure 6). The total number of revision after
protocol-aided revision training was larger than that after teacher feedback-aided revision,
and its difference from that of self-revision was also larger than the difference of revision
Yeon Hee Choi
42
frequency between self-revision and post-training teacher feedback-aided revision.
However, such differences were not noted in revision per 100 words.
FIGURE 5
Pre-and Post-Training Frequency of Total Revisions and Revision per 100 Words
FIGURE 6 Post-Training Revision Frequency per 100 Words by L2 Proficiency
(writing)W1-SR W1-TF W3-SR W3-TA
12.47
16.94
15.12
7.73
4.87
22.29
4.95
7.27
Total R
R per 100
words
(writing)W1-SR W1-TF W3-SR W3-TA
12.47
16.94
15.12
7.73
4.87
22.29
4.95
7.27
Total R
R per 100
words
(writing)W1-SR W1-TF W3-SR W3-TA
12.47
16.94
15.12
7.73
4.87
22.29
4.95
7.27
Total R
R per 100
words
(R frequency per
100 words)
(writing)W1-SR W1-TF W3-SR W3-TA
4.30 4.33
7.48
6.064.80
3.73
16.56
5.22
11.44
4.79
7.00
8.51
LP
HP
IP
(R frequency per
100 words)
(writing)W1-SR W1-TF W3-SR W3-TA
4.30 4.33
7.48
6.064.80
3.73
16.56
5.22
11.44
4.79
7.00
8.51
LP
HP
IP
Effects of Revision Training on L2 Writing
43
TABLE 3 ANOVA Results of Pre-and Post-Training Frequency of Total Revisions and Revision per 100
Words by L2 Writing Proficiency
Proficiency Level Revision Frequency LP IP HP F
M SD M SD M SD
Total No. of R in W1-SR 5.33 3.06 13.89 7.04 14.20 7.79 1.976
Total No. of R in W1-TF 21.67 7.77 11.33 6.69 24.20 13.03 3.710
Total No. of R in W3-SR 18.67 11.59 16.11 11.45 11.20 5.63 .598
Total No. of R in W3-TA 25.67 9.02 19.11 6.49 26.00 9.70 1.551
R per 100 words in W1-SR 4.80 3.94 5.22 2.81 4.30 2.61 .155
R per 100 words in W1-TF 16.56 2.94 4.33 2.81 7.00 3.32 18.941***
A B
R per 100 words in W3-SR 7.48 1.77 4.79 2.35 3.73 2.05 2.805
R per 100 words in W3-TA 11.44 3.03 6.06 2.23 8.51 3.22 4.881*
A B A
* p < .05, *** p < .001
Proficiency level-sensitive variations were noted across the three proficiency levels in
the frequency difference between the total number of self-revision and post-training
revision (see Table 3). The LP group made much more post-training teacher feedback-
aided revision, compared with their self-revision (revision frequency difference of W1-SR
and W1-TF, M = 16.33, SD = 8.14). They also made a great deal of self revision in the
third writing (before protocol-aided revision training). These results illustrate that teacher
feedback-aided revision training led the LP group to make more revision. On the other
hand, a positive effect of protocol-aided revision training was noticeable in the HP group
(revision frequency difference of W3-SR and W3-TA, M = 14.80, SD = 6.97). The IP
group revised less after teacher feedback-aided revision training, though their writing score
of the post-training revised draft (W1-TF) was higher than that of the self-revised draft
(W1-SR).
Revision frequency per 100 words before and after training also displays proficiency-
sensitive variations (see Table 3). Statistical analyses of ANOVA reveal a significant
impact of teacher feedback-aided revision training on the LP group (see Figure 6): their
revision per 100 words was significantly larger than that of the IP and HP group after the
training. Their protocol-aided post-training revision frequency was also distinctively
higher than the IP group (see the results of the post-hoc analysis in Table 3).
Revision was also analyzed in terms of its purpose. It was classified into seven
categories (e.g., paraphrase, grammar, style, and major content change), as explained in
III.4, which was reclassified as content preserving (CP) changes (e.g., grammar and
mechanics) and content changes (CC) (macro- and micro-structure changes). As shown
Yeon Hee Choi
44
in Figure 7 and Table 4, all the participants tended to make CP revision more than CC
revision regardless of revision and training types, except for post-training protocol-aided
revision (W3-TA) due to the IP group’s CC ratio. A high frequency of local revision was
also noted in L2 writers’ online revision in New (1999), Stevenson et al. (2006), and
Yeonhee Choi (2007). CC ratio gradually arose and its average was the highest in
post-training protocol-aided revision, except for the HP group; a noticeable increase
was found in the LP and IP group. A significant difference between the ratio of CP and
CC was found by repeated measures ANOVA because of the significantly higher CP
revision ratio in both W1 self-revision (CP, M = .63; CC, M = .37; F = 7.106, p = .018)
and post-training teacher feedback-aided revision (CP, M = .61; CC, M = .39; F =
11.663, p = .004), but not in W3 self-revision (CP, M = .58; CC, M = .42) and post-
training protocol-aided revision (CP, M = .49; CC, M = .51). A much higher CC ratio of
post-training protocol-aided revision was noted than that of post-training teacher
feedback-aided revision. These results imply that the more L2 writers are trained for
revision, the more they make content changes, that is, the quality change in their
revision, which has been noted in Berg’s (1999) and Min’s (2006) peer review training.
The ratio of grammar changes was much higher in post-training teacher feedback-aided
revision (M = .38) than in post-training protocol-aided revision (M = .17), while that of
micro-level content changes was noticeably higher in post-training protocol-aided
revision (M = .46) than post-training teacher feedback-aided revision (M = .25). The
high ratio of grammar changes might have been led by the nature of teacher feedback.
That is, among the seven categories of teacher feedback given to the participants,
grammar feedback was the most frequent (e.g., grammar, M = 12.65; word choice, M =
4.88; content, M = .82; organization, M = 2.12). This might provide an account for the
fact that in post-training teacher feedback-aided revision, the HP group’s CP revision ratio
FIGURE 7 Pre- and Post-Training CP and CC Revision Ratio
(ratio)
(writing)
0.630.61
0.58
0.49
0.37
0.51
0.42
0.33
CC
CP
(ratio)
(writing)
0.630.61
0.58
0.49
0.37
0.51
0.42
0.33
CC
CP
Effects of Revision Training on L2 Writing
45
TABLE 4 ANOVA Results of Pre-and Post-Training Revision Ratio and Frequency per 100 Words across
Revision Purposes by L2 Writing Proficiency
Revision Ratio or Revision
per 100 Words
by Purpose Types
Proficiency Level
Total LP IP HP F
M SD M SD M SD M SD
CP ratio in W1-SR .63 .19 .73 .29 .66 .17 .51 .15 1.481
CP ratio in W1-TF .61 .25 .63 .35 .59 .23 .79 .06 2.958
CP ratio in W3-SR .58 .21 .65 .14 .60 .19 .50 .27 .504
CP ratio in W3-TA .49 .15 .54 .06 .44 .18 .53 .13 .814
CC ratio in W1-SR .37 .19 .27 .29 .34 .17 .49 .15 1.481
CC ratio in W1-TF .33 .21 .37 .35 .40 .21 .21 .06 1.403
CC ratio in W3-SR .42 .21 .36 .14 .40 .19 .50 .27 .504
CC ratio in W3-TA .51 .154 .46 .06 .56 .18 .47 .13 .814
CP per 100 words in W1-SR 3.05 1.66 2.95 1.60 3.39 1.70 2.49 1.82 .452
CP per 100 words in W1-TF 4.35 4.23 9.80 7.25 2.26 1.59 4.83 2.83 5.752*
2.96 A B A
CP per 100 words in W3-SR 2.96 1.94 5.10 2.46 2.66 1.47 2.20 1.81 2.869
CP per 100 words in W3-TA 4.00 2.22 6.16 1.80 2.81 1.56 4.84 2.38 4.337*
2.05 A B A
CC per 100 words in W1-SR 1.99 1.66 2.35 3.25 1.92 1.52 1.91 .97 .074
CC per 100 words in W1-TF 2.04 2.04 4.63 4.00 1.61 .91 1.24 .82 4.229*
A B
CC per 100 words in W3-SR 2.05 1.07 2.58 .78 2.13 1.27 1.59 .77 .835
CC per 100 words in W3-TA 3.73 1.34 5.28 1.46 3.24 1.12 3.67 1.14 3.348
A B A
* p < .05
outnumbered their CC revision ratio and also the LP and HP group’s CP revision ratio. On
the other hand, the high ratio of micro-level content changes might be an impact of
repeated revision training or that of protocol-based revision training.
Both CP and CC revision frequency per 100 words was higher after each training than
in self-revision (see Table 4). Compared with CP revision per 100 words in self-revision, a
larger number of CP revisions in post-training revision was noted. CC revision frequency
per 100 words in post-training protocol-aided revision outnumbered that in W3
self-revision and also that in post-training teacher feedback-aided revision.
A comparison of the three proficiency groups illustrates the LP and IP group’s higher
ratio of CP revision over their ratio of CC revision, except for the IP group’s post-training
protocol-aided revision; and also their higher ratio of CP revision in self-revision. A
decline was noted in their ratio of CP revision after each revision training, while an
Yeon Hee Choi
46
increase was found in their ratio of CC revision. This suggests an influence of revision
training on the LP and IP group’s revision purpose, that is, they made more attempts to
make global revisions. On the contrary, the HP group’s ratio of CP revision increased after
each revision training, especially in their post-training teacher feedback-aided revision,
while their ratio of CC revision dropped from .49 to .21 in post-training teacher
feedback-aided revision and slightly increased from .50 to .53 in post-training protocol-
aided revision. These findings are contradictory to those from Zamel’s (1983) pen-and-
paper ESL writing, and van Waes and Schellens’ (2003) and Yeonhee Choi’s (2007)
online FL writing (more proficient L2 writers make more higher-level revisions). This can
be explained by the nature of the teacher feedback they received. As mentioned above,
most of the teacher feedback focused on grammar and word choice including awkward
expressions, which might have triggered CP revisions. Within the three proficiency groups,
the HP group (feedback on grammar, M = 17.80; word choice, M = 8.20) received these
types of teacher feedback more frequently than the other two groups (IP, feedback on
grammar, M = 11.78; IP, word choice, M = 5.00; LP, feedback on grammar, M = 6.67; LP,
word choice, M = 6.00) because their writing was much longer. This might have
influenced their teacher feedback-aided revision.
The ANOVA results of comparing the three groups in the revision frequency per 100
words illustrate group variations, as shown in Table 4. A much larger number of CP
revision was noted in the LP group’s post-training teacher feedback-aided revision,
which might be accounted for by their high frequency of grammar revision (M = 6.04).
After revision training, moreover, the HP and LP group’s revision patterns became more
similar, as noted by the post-hoc analysis. Interestingly, that of the IP group appears
distinctive from the other groups: their CP and CC revision frequency declined after
teacher feedback-aided revision.
3. Success Ratio of Pre- and Post-Training Revision and Utilization Ratio
of Teacher Feedback and Protocol
To see the effect of revision training, the extent to which the participants successfully
revised their first draft after revision training was measured. Compared with the self-
revision, the success ratio of revision after teacher feedback-aided revision training
declined, while the failure ratio increased (see Table 5). It was more noticeable in more
proficient L2 writers (the IP and HP group). On the contrary, it was the opposite after
protocol-aided revision. These results imply a more positive effect of protocol-aided
revision training. But, a caution is required since it is plausible that the declined success
ratio and the increased failure ratio in teacher feedback-aided revision might have resulted
from the fact that a larger number of revisions were triggered by teacher feedback, but the
Effects of Revision Training on L2 Writing
47
participants did not have enough writing ability to revise successfully.
TABLE 5 ANOVA Results of Success and Failure Ratio of Pre- and Post-Training Revision and
Utilization Ratio of Teacher Feedback and Protocol by L2 Writing Proficiency
Success, Failure, and
Utilization Ratio
Proficiency Level
Total LP IP HP F
M SD M SD M SD M SD
Success ratio in W1-SR .76 .16 .71 .26 .77 .17 .79 .10 .235
Success ratio in W1-TF .71 .28 .81 .17 .66 .38 .73 .06 .299
Success ratio in W3-SR .76 .17 .54 .29 .77 .11 .87 .09 4.966*
A B
Success ratio in W3-TA .81 .12 .73 .13 .82 .13 .83 .10 .840
Failure ratio in W1-SR .20 .17 .29 .26 .19 .17 .17 .13 .499
Failure ratio in W1-TF .23 .23 .19 .17 .23 .30 .24 .09 .036
Failure ratio in W3-SR .11 .09 .17 .08 .12 .08 .05 .07 2.644
Failure ratio in W3-TA .10 .08 .18 .10 .09 .09 .09 .05 1.561
Utilization ratio of teacher
feedback in W1-TF
.51 .21 .54 .29 .44 .21 .60 .17 .935
Successful utilization ratio of
teacher feedback in W1-TF
.65 .33 .84 .16 .63 .38 .58 .33 .556
Utilization ratio of protocol in
W3-TA
.59 .20 .67 .12 .60 .23 .52 .21 .537
Successful utilization ratio of
protocol in W3-TA
.78 .19 .70 .18 .81 .17 .77 .25 .400
* p < .05
In teacher feedback-aided revision, interestingly, the LP revised more successfully than
the IP and HP group, which was different from the questionnaire survey on correct
incorporation of teacher feedback in Lee (2008) (a more positive response by higher-level
Hong Kong secondary school students). They recognized their problems more since they
were pointed out by the teacher; thus, they made an attempt to revise them. These attempts
were relatively successful, as shown by their high successful utilization ratio of teacher
feedback in W1-TF (see Table 5). On the other hand, the higher level students were
able to recognize their problems by themselves and revise them in their self-revision. The
problems which were not self-recognized might have been beyond their ability to revise
successfully. Their successful utilization ratio of teacher feedback was much lower than
the LP group’s. A significant difference in the success ratio of revision in the self-revision
of W3 (W3-SR) was noted between the LP group and the IP and HP group, but not in the
post-training protocol-aided revision of W3 (W3-TA). These results also suggest that the
LP group can take more advantage of revision training than the other two groups. However,
Yeon Hee Choi
48
the IP and HP group’s higher successful utilization ratio of protocol imply that protocol-
aided revision training may be more helpful for more proficient L2 writers.
4. L2 Writer Perception on Revision Training
A pre-questionnaire showed that all the participants had some experience of revising
their own writing, but only nine out of seventeen participants received formal instruction
on revision. Teacher and peer feedback-aided revision were the most frequently
employed revision methods (21 responses out of 34 responses) and they were followed
by protocol-aided revision (eight responses out of 34 responses). The participants
indicated that they would revise their writing mostly in the areas such as organization
and structure. In the responses to the post-questionnaires, the difficulties they pointed
out in teacher feedback-aided revision were that teacher feedbacks were incomprehensible
(11 responses) and unspecific (seven responses), as the illegibility problem of teacher
feedback was noted in Sommers (1982) and Lee (2008). The difficulties they pointed out
in protocol-aided revision were that the participants did not know how to revise (six
responses out of 17 participants) and they did not know appropriate expressions due to
their lack of writing and grammatical competence (10 responses out of 17 participants).
Their lack of writing and grammatical competence was indicated as the main cause of
their failure to revise successfully in both teacher feedback-aided and protocol-aided
revision.
As to the question of effectiveness of each revision method, 13 participants pointed
out that teacher feedback was helpful because they were corrective and they were given
by a teacher, i.e., a writing specialist. Four participants mentioned that protocol-aided
revision was more effective than teacher feedback-aided revision and one of them
pointed out reader awareness as a helpful factor to revise successfully. With regard to
long-term effects on revision, the majority preferred teacher feedback-aided revision (13
participants). Nevertheless, nine participants also preferred protocol-aided revision
because revision could be done at any time by the writers (one participant) and is an
individual work (four participants).
The responses of participants to a pre-questionnaire and two post-questionnaires were
compared to see the participants’ perceptions on revision training and their relationships
with revision and writing quality. The results of Wilcoxon test reveal that after revision
training the participants responded that they revised in their teacher feedback-aided
revision significantly more than in their protocol-aided revision at the .018 level (see Table
6 and Figure 8). After revision training, however, their perception on the positive effect of
teacher feedback-aided revision significantly dropped at the .005 level and their perception
on its utilization in revision also downgraded, whereas their perception on the positive
Effects of Revision Training on L2 Writing
49
effect of protocol-aided revision significantly enhanced at the .024 level and their perception
on its utilization in revision also significantly increased at the .046 level. The participants’
reader awareness enhanced after revision training, but no significant change was noted.
FIGURE 8 Pre- and Post-training Questionnaire Responses on the Effects and Utilization of Teacher
Feedback- and Protocol-aided Revision and Reader Awareness
TABLE 6 Pre- and Post-training Questionnaire Responses by L2 Writing Proficiency
Questions Pre-/Post-
training
Total
Proficiency Level
LP IP HP
M SD M SD M SD M SD
Positive effects of
teacher feedback
Pre-training 5.35 .60 5.33 .58 5.44 .53 5.20 .83
Post-training 4.00 1.27 4.00 2.00 3.78 1.30 4.40 .89
Positive effects of
think-aloud
Pre-training 3.47 .80 2.67 .58 3.67 .71 3.60 .89
Post-training 3.88 1.32 4.67 .57 3.33 1.32 4.40 1.34
Use of teacher feedback Pre-training 4.94 .56 4.67 .58 5.11 .33 4.80 .84
Post-training 4.47 1.01 3.33 2.08 4.56 .53 5.00 .00
Use of think-aloud Pre-training 3.29 .77 2.67 .58 3.33 .71 3.60 .89
Post-training 3.76 .75 3.33 1.52 3.78 .67 4.00 .00
Reader awareness Pre-training 3.47 .80 3.00 1.00 4.00 .87 3.60 .89
Post-training 3.76 1.20 3.33 1.15 3.78 1.39 4.00 1.00
(question)Positive
effects of
teacher
feedback
5.35
3.29
3.76
3.47
4.474.00 3.88
4.94
(responses)
Positive
effects of
think-aloud
Use of
teacher
feedback
Use of
think-aloud
Reader
awareness
3.71
3.76
(question)Positive
effects of
teacher
feedback
5.35
3.29
3.76
3.47
4.474.00 3.88
4.94
(responses)
Positive
effects of
think-aloud
Use of
teacher
feedback
Use of
think-aloud
Reader
awareness
3.71
3.76
Yeon Hee Choi
50
The comparison of the three proficiency groups’ responses reveals that compared with
those of HP and LP groups, the IP group showed a drastic change in the positive effects
of teacher feedback-aided revision, which dropped from 5.44 to 3.78. On the other hand,
all the responses to the categories of protocol-aided revision became more positive after
training regardless of the groups (except for the IP group’s responses on its
positive effects). A large change was noted in the perceptions of LP group on
protocol-aided revision. Their perception of positive effects of think-aloud on revision
increased from 2.67 to 4.67. A major group variation was noted between the IP group’s
responses and those of the HP and LP group. Except for one category, the use of
think-aloud in revision, the IP group showed negative responses to all the other
categories such as the effects of teacher feedback and that of think-aloud, the use of
teacher feedback, and reader awareness.
V. CONCLUSION
The findings from the study suggest the positive effects of revision training on writing
quality. Teacher feedback-aided revision training displayed more positive effects on the
quality and product of writing than protocol-based revision training. As mentioned above,
this might be accounted for by the sequence of training, since teacher feedback-aided
revision training preceded protocol-aided revision training. Thus, more research is needed
if the former influences revision more effectively than the latter when it is investigated
with two independent groups. As for the scores of the drafts, moreover, the HP group’s
score remained relatively steady, whereas that of the LP and IP group continuously
increased. Thus, the latter groups seem to benefit more from revision training.
In addition, text length differences were more noticeable in teacher feedback-aided
revision training than in protocol-aided revision training. The three groups’ text length
became similar after the first revision training. The LP group’s increase in text length
outnumbered the other groups’ in teacher feedback-aided revision training. These
findings imply a significant benefit of teacher feedback-aided revision training for
lower-level L2 writers.
The comparison of the total number of pre- and post-training revision and revision per
100 words also illustrates the positive effects of revision training regardless of the
revision training types. The total number of revision after protocol-aided revision
training outnumbered that after teacher feedback-aided revision. Compared with the HP
and IP group, the LP group’s noticeable frequency difference between pre- and post-
training revision and the large number of their self revision in the third writing suggest
that teacher feedback-aided revision training led the LP group to make more revision.
Effects of Revision Training on L2 Writing
51
On the other hand, a positive effect of protocol-aided revision training was noticeable in
the HP group. This implies that effects of different types of revision training vary with
L2 proficiency groups.
A positive effect of protocol-aided revision was noted in the analysis of revision by its
purpose, especially for the lower-level participants. In general, content preserving
revisions outnumbered content changes; however, the ratio of content changes gradually
enhanced and it outnumbered that of content preserving revisions in post-training
protocol-aided revision. The increase was more noticeable in the LP and IP group than
in the HP group. These results imply that the more L2 writers are trained for revision,
the more global revisions they make. Moreover, the HP group’s higher ratio of content
preserving revision in post-training teacher feedback-aided revision than in their self-
revision suggests the impact of the nature of teacher feedback: the high frequency of
feedback on grammar might have triggered local revisions because their writing was
much longer.
The comparison of the success ratio of pre- and post-training revision illustrates a
declined success ratio of post-training teacher feedback-aided revision in the more
proficient L2 writers and an increased success ratio of post-training protocol-aided
revision in the less proficient L2 writers. These results might imply a negative effect of
teacher feedback-aided revision training for higher-level L2 writers but a positive effect
of protocol-aided revision training for lower-level L2 writers. However, a caution is
required in their interpretation because a possibility cannot be excluded that lower-level
writers revise their writing problems explicitly indicated by their teacher which they are
not able to notice by themselves, whereas higher-level writers’ problems pointed out by
the teacher are beyond their competence since they can revise self-recognized problems.
This account seems to be supported by the fact that the higher-level participants’
successful utilization ratio of teacher feedback was much lower than the lower-level
ones.
Although more participants still preferred teacher feedback-aided revision over
protocol-aided revision after training and they utilized teacher feedback more, the results
of the questionnaire survey suggest a more positive effect of protocol-aided revision
training than teacher feedback-aided revision training. In general, the incorporation rate
of teacher feedback after training was lower than that of peer feedback after training in
Yunkyoung Cho (2005) and Min (2006). One of the main reasons seems to be the
incomprehensibility and unspecificity of teacher feedback, as pointed out by the
participants. This implies the importance of the quality or nature of teacher feedback in
teacher feedback-aided revision; thus, the future research on teacher feedback-aided
revision training should take a consideration on what kind of teacher feedback to use
Yeon Hee Choi
52
and how to solve the problems raised by its incomprehensibility or miscommunication
between the teacher and the L2 writer.
The variation in reader awareness before and after revision training was not much
noticeable in the questionnaire survey. The previous L2 writing studies on reader
awareness such as Raimes (1985) and Yeonhee Choi (2006), which provided a prompt
with specified audience, also noted that in L2 writing raising reader awareness was not a
simple process. More studies are thus needed to see whether revision training including
teacher feedback- or protocol-aided revision can enhance reader awareness, especially
for more proficient L2 writers, as noted in Hayes et al. (1987), and suggested in Jones
and Myhill (2007).
Think-aloud, concurrent verbal protocols, is not commonly used to help L2 writers
revise their writing. The findings from the present study suggest that protocol-based
revision can be an alternative to teacher or peer feedback-aided revision. To explore its
effects further, a comparison among different revision types is needed with an
independent larger-sample group for each type. Such a comparison would shed light on
their effects on writing quality and revision and make a more conclusive suggestion on
effective types of revision for different L2 writing proficiency groups.
References
Berg, E. C. (1999). The effects of trained peer response ESL students’ revision types and
writing quality. Journal of Second Language Writing, 8, 215-241.
Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second
Language Writing, 17, 102-118.
Bitchener, J., & Knoch, U. (2009). The relative effectiveness of different types of direct
written corrective feedback. System, 37, 322-329.
Cho, Yunkyoung. (2005). The effectiveness of peer feedback in Korean EFL writing
classrooms. English Language Teaching, 17, 33-59.
Choi, Huyyun. (2002). A study on Korean students’ perceptions on teachers’ feedback in
English writing classes. Unpublished master’s thesis. Sookmyung Women’s
University, Graduate School of TESOL, Seoul.
Choi, Yeonhee. (2006). Effects of L2 writing proficiency and reader awareness on EFL
students’ writing product and process on-line: Computer-based observation. The
Journal of Asia TEFL, 3(2), 37-78.
Choi, Yeonhee. (2007). On-line revision behaviors in EFL writing process. English
Teaching, 62(4), 69-93.
Effects of Revision Training on L2 Writing
53
Conrad, S. M., & Goldstein, L. M. (1999). ESL student revision after teacher-written
comments: Text, contexts, and individuals. Journal of Second Language Writing, 8,
147-180.
Ellis, R., Sheen, Y., Murakami, M., & Takashima, H. (2008). The effects of focused and
unfocused written corrective feedback in an English as a foreign language context.
System, 36, 353-371.
Faigley, L., & Witte, S. (1981). Analyzing revision. College Composition and
Communication, 32, 400-414.
Ferris, D. R. (1995). Student reactions to teacher response in multiple-draft composition
classrooms. TESOL Quarterly, 29, 33-53.
Ferris, D. R. (1997). The influence of teacher commentary on student revision. TESOL
Quarterly, 31, 315-339.
Ferris, D. R. (2003). Responding to writing. In B. Kroll (Ed.), Exploring second language
writing (pp. 119-140). Cambridge: Cambridge University Press.
Goldstein, L. M., & Conrad, S. M. (1990). Student input and negotiation of meaning in
ESL writing conferences. TESOL Quarterly, 24, 443-460.
Hall, C. (1987). Revision strategies in L1 and L2 writing tasks: A case study. Unpublished
doctoral dissertation. University of New Mexico, Albuquerque, NM.
Hall, C. (1990). Managing the complexity of revising across languages. TESOL Quarterly,
24, 43-60.
Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing.
In C. M. Levy & S. Ransdell (Eds.), The science of writing: Theories, methods,
individual differences and applications (pp. 1-27). Mahwah, NJ: Lawrence
Erbaum.
Hayes, J. R. (2000). A new framework for understanding cognition and affect in writing.
In R. Indrisano & J. R. Squire (Eds.), Perspective on writing: Research, theory, and
practice (pp. 6-44). Newark, DE: International Reading Association.
Hayes, J. R., Flower, L, Schriver, K., Stratman, J., & Carey, L. (1987). Cognitive
processes in revision. In S. Rosenberg (Ed.), Advances in applied psycholinguistics,
Vol 2: Reading, writing and language learning (pp. 176-240). Cambridge, MA:
Cambridge University Press.
Hedgcock, J., & Lefkowitz, N. (1994). Feedback on feedback: Assessing learners'
receptivity to teacher response in L2 composing. Journal of Second Language
Writing, 3, 141-163.
Holliway, D. R., & McCutchen, D. (2004). Audience perspective in young writers'
composition and revising. In L. Allal, L. Chanquoy, & P. Largy (Eds.), Revision:
Cognitive and instructional process (pp. 87-101). Boston, MA: Kluwer Academic
Publishers.
Yeon Hee Choi
54
Hyland, F. (1998). The impact of teacher-written feedback on individual writers. Journal
of Second Language Writing, 7, 255-186.
Hyland, F., & Hyland, K. (2001). Sugaring the pill: Praise and criticism in written
feedback. Journal of Second Language Writing, 10, 185-212.
Jones, S., & Myhill, D. A. (2007). Discourses of difference? Questioning gender
differences in linguistic characteristics of writing. Canadian Journal of Education
30(2), 456-482.
Jourderais, R. (2001). Protocol analysis and SLA. In P. Robinson (Ed.), Cognition and
second language acquisition (pp. 354-375). New York: Cambridge University
Press.
Lee, I. (2008). Student reactions to teacher feedback in two Hong Kong secondary
classrooms. Journal of Second Language Writing, 17, 144-164.
Lee, Jihye. (2003). Hanguk daehaksaeng-euy yeongeo gulssugi sujeong jeonryak
sayong-ey gwanhan saryey yeongu (A case study on the revision strategies of
Korean university students’ English composition). Unpulished master’s thesis.
Ewha Womans University, Seoul.
Lee, Jeong-Won, & Hong, Young-Joo. (2001). Gwajeongjooshim feedback-i yeongeo
ssuginungryeok hyangsang-ey michinun yeonghyang (The effects of process-
based feedback on the development of English writing proficiency). English
Teaching, 56(2), 265-285.
Leki, I. (1990). Coaching from the margins: Issues in written response. In B. Kroll (Ed.),
Second language writing: Research insights for the classroom (pp. 57-68).
Cambridge: Cambridge University Press.
Lindgren, E., & Sullivan, K. P. H. (2006). Analysing online revision. In K. P. H. Sullivan
& E. Lindgren (Eds.), Studies in writing, Vol. 18: Computer keystroke logging and
writing (pp. 157-188). Oxford: Elsevier.
Liou, H.-C., & Peng, Z.-Y. (2009). Training effects on computer-mediated peer review.
System, 37, 514-525.
Lundstrom, K., & Baker, W. (2009). To give is better than to receive: The benefits of peer
review to the reviewer’s own writing. Journal of Second Language Writing, 18,
30-43.
Min, H.-T. (2005). Training students to become successful peer reviewers. System, 33,
293-308.
Min, H.-T. (2006). The effects of trained peer review on EFL students’ revision types and
writing quality. Journal of Second Language Writing, 15, 118-141.
Nelson, G., & Carson, J. (1998). ESL students' perceptions of effectiveness in peer
response groups. Journal of Second Language Writing, 7, 113-131.
Effects of Revision Training on L2 Writing
55
New, E. (1999). Computer-aided writing in French as a foreign language: A qualitative
and quantitative look at the process of revision. The Modern Language Journal, 83,
80-97.
Phinney, M., & Khouri, S. (1993). Computers, revision, and ESL writers: The role of
experience. Journal of Second Language Writing, 2, 257-277.
Raimes, A. (1985). What unskilled ESL students do as they write: A classroom study of
composing. TESOL Quarterly, 19, 229-258.
Sachs, R., & Polio, C. (2007). Learners’ uses of two types of written feedback on L2
writing revision tasks. Studies in Second Language Acquisition, 29, 67-100.
Sengupta, S. (2000). An investigation into the effects of revision strategy instruction on L2
secondary school learners. System, 28, 97-113.
Sommers, N. (1980). Revision strategies of student writers and experienced adult writers.
College Composition and Communication, 31, 378-388.
Sommers, N. (1982). Responding to student writing. College Composition and
Communication, 33, 148-156.
Stanley, J. (1992). Coaching student writers to be effective peer evaluators. Journal of
Second Language Writing, 1, 217-233.
Stevenson, M., Schoonen, R., & de Glopper, K. (2006). Revising in two languages: A
multi-dimensional comparison of online writing revisions in L1 and FL. Journal of
Second Language Writing, 15, 201-233.
Stratman, J. F., & Hamp-Lyons, L. (1994). Reactivity in concurrent think-aloud protocols:
Issues for research. In P. Smagorinsky (Ed.), Speaking about writing: Reflections
on research methodology (pp. 89-112). Thousand Oaks, CA: Sage.
Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language
Learning, 46, 327-369.
Van Waes, L., & Schellens, P. J. (2003). Writing profiles: The effect of the writing mode
on pausing and revision patterns of experienced writers. Journal of Pragmatics, 33,
829-853.
Yang, M., Badger, R., & Yu, Z. (2006). A comparative study of peer and teacher feedback
in a Chinese EFL writing class. Journal of Second Language Writing, 15, 179-200.
Zamel, V. (1982). Writing: The process of discovering meaning. TESOL Quarterly, 16,
195-209.
Zamel, V. (1983). The composing process of advanced ESL students: Six case studies.
TESOL Quarterly, 17, 165-187.
Zamel, V. (1985). Responding to student writing. TESOL Quarterly, 19, 79-102.
Yeon Hee Choi
56
Appendix 1 Coding Scheme Used to Classify the Degree of Success in Revision
Categories Examples
Successful
(S)
(I9-1) ‘When I study in a group to get bad results,’ to ‘When I got bad results in
group study,’
Unsuccessful
(US)
(I3-4) ‘Thinking in various ways enables creative thinking’ to ‘Thinking in
various ways possible creative thinking’
No Difference/
Improvement
(L2-1) ‘To solve problems is easy in a study-group’ to ‘Solving problems is easy in
a study-group’
(ND/I) (I6-3) ‘…it needs much time to cooperate with others rather than do it alone’ to
‘…it needs less time to deal with problem than with a group’ {Here ‘it’
refers to ‘studying alone.’}
Applicable levels: tertiary
Key words: revision training, teacher feedback, think-aloud, protocol, self-revision, L2 writing
Yeon Hee Choi
Dept. of English Education
Ewha Womans University
11-1 Daehyun-dong, Seodaemun-ku
Seoul, 120-750, Korea
Tel: (02) 3277-2655
Email: [email protected]
Received in December, 2009
Reviewed in January, 2010
Revised version received in February, 2010