prospectus draft 2.6
TRANSCRIPT
-
8/8/2019 Prospectus Draft 2.6
1/12
Amanda Wilson
October 24, 2010
EdS Thesis Prospectus
How Students Perceive Their Learning
An exploration of student reflections on assessment
Statement of the Problem
There has been a shift in higher education from teaching objectives to student learning
outcomes. Much of this drive has precipitated from a shift of focus from regional accreditors of
schools. The focus on student learning outcomes facilitates a culture of continual improvement
of the education institution as a whole that strives to engage the entire academic community.
Authentic assessment measures are becoming more common as supplements to more traditional
assessment measures to provide valuable data needed to inform this culture of change and
improvement. The outcomes assessment movementin foreign language classes has embraced
these authentic assessment measures, mostly in the form of portfolio-style projects for students.
Unfortunately, there is less research on how students are receiving these various forms of
assessment and what they perceive as the benefits or drawbacks of each. The purpose of this
study is to explore the reflections of students in a beginning level Spanish class towards various
forms of traditional and authentic assessment tools.
Significance of the Problem
References to Cross & Hendricks and Background/Interest
Literature Review
A Focus Shift: Student Learning Outcomes
-
8/8/2019 Prospectus Draft 2.6
2/12
A culture change is occurring in higher education that shifts the primary focus of
educators from teaching the subject matter of specific disciplines to a perspective of student
learning (Allen 2004, p. 1). As departmental, organizational, and institutional cultures undergo
change, and as the focus of that change is less on teaching and more on learning, a commitment
to sustainable outcomes assessment becomes essential (Hernon, et al. 2006, p. 1). According to
Allen, this type of assessment occurs when, empirical data on student learning is used to refine
programs and improve student learning (2004, p. 2).
At the classroom-level, Palmer encourages viewing student assessment as a strategic
tool for enhancing teaching and learning (2004, p. 194). He goes on, a few pages later to add
that, Continuous assessment starting early in the semester has the benefit of quickly identifying
those students falling behind and perhaps at risk of dropping out, so remedial action can be
taken (p. 198).
Allen comments that, while classroom assessment examines learning in the day-to-day
classroom, program assessment systematically examines student attainment in the entire
curriculum (2004, p. 1). In their 2001 work, Ratcliff, et al., point out that the continuous
improvement cycle should begin with clear departmental goals that identify what a student can
expect to gain as a result of studying a particular field or discipline (p. 25). Hernon, et al. add
that, programs and institutions need to develop a strong and sustainable commitment to
assessment as a process and as a means to improve learning based on explicit student learning
outcomes (2006, p. 11). Ratcliff, et al. further point out that, While a colleges or universitys
general goals for student achievement can be measured at the university level, the accreditation
self-study must address student academic achievement in the discipline. Therefore, departments
-
8/8/2019 Prospectus Draft 2.6
3/12
and programs must contribute to the accreditation self-study by assessing their students learning
at the department level (2001, p. 32).
Classroom, program (or department), and institutional assessment support a foundation for
accreditation. As this culture shift continues to push the focus towards student learning,
accreditation standards continue to link student outcomes assessment to continued accreditations
of programs and schools (Ratcliff, et al. 2001, p. 13).
Regional Accreditors on Assessment
Allen tells us on page 18 of her 2004 text that, accrediting organizationsgenerally
focus on two major issues: capacity and effectiveness. She goes on to explain that capacity is
the bean counting of the process. It is when tallies are taken of the resources any institution has
to support its student such as libraries, technology, physical space, and student support services.
The focus, however, has really turned more towards a long-term commitment to improving
student learning (Hernon, et al. 2006, p. 1). Allen continues on to say that accrediting
organizations, expect campuses to document their impact on student learning (2004, p. 18) and
that, when accrediting bodies require assessment, campuses pay attention (2004, p. 2). She
cautions, however, that, assessmentshould be implemented because it promotes student
learning, not because an external agency requires it (Allen 2004, p. 2).
Closing the Loop
On pages 163 and 164, Allen imparts some friendly suggestions, one of which is to
close the loop, stating that good assessment has impact (2004). The foundation of all this
assessment is that it will drive change towards the continual improvement of the quality of the
educational system. As Ratcliff, et al. put it, Assessment and accreditation are both premised on
the importance of quality assurance (2001, p. 17).
-
8/8/2019 Prospectus Draft 2.6
4/12
A Community of Assessors: Collaboration is Key
To move toward these lofty goals, faculty, institutional research offices, and everyone in
the educational enterprise, has [the] responsibility for maintaining and improving the quality of
services and programs (Ratcliff, et al. 2001, p. 17). Assessment of student learning outcomes,
includes all members of the [educational] community as they strive to contribute to and enhance
the educational enterprise, as a whole (Ratcliff, et al. 2001, p. 17).
Authentic Assessment as Means to Focus on Student Learning Outcomes
Brown, et al. expound the functions of assessment on page 47 of their 1999 work as six
points:
1. Capturing student time and attention.2. Generating appropriate student learning activity.3. Providing timely feedback which students pay attention to.4. Helping students to internalize the disciplines standards and notions of quality.5. Marking: generating marks or grades which distinguish between students or which
enable pass/fail decisions to be made.
6. Quality assurance: providing evidence for others outside the course (such as externalexaminers) to enable them to judge the appropriateness of standards on the course.
With these purposes in mind, assessment methods can be examined to determine their validity.
On pages 62 and 63, the authors discuss traditional unseen written exams and how they function
as assessments: In particular, this assessment format seems to be at odds with the most
important factors underpinning successful learningthere is cause for concern that traditional
unseen written exams do not really measure the learning outcomes which are the intended
purposes of higher education (Brown, et al. 1999). Palmer echoes these concerns on page 194 of
-
8/8/2019 Prospectus Draft 2.6
5/12
his 2004 paper on authenticity in assessment, stating that, traditional forms of assessment can
encourage surface learning rather than deep learning. Banta goes a bit further, with her colorful
simile to discourage purchasing more traditional assessment measures for the purposes of
improving student learning: Just as weighing a pig will not make it fatter, spending millions to
test college students is not likely to help them learn more (2007, p. 2).
Watson points out, a need for more authentic, learner-friendly methods to encourage
[student] engagement (2008, p. 1) which seems to align with Brown, et al.s first and second
functions of assessment listed above (1999, p. 47). Watson goes on to point out a bit further on
that, the assessment of authentic performancehas the potential to address a number of
contemporary criticisms of assessment, (2008, p. 1) such as those quoted above of Banta,
Palmer, and Brown, et al. Banta also notes that, authentic and valid assessment approaches must
be developed and promoted as viable alternatives to scores on single-sitting, snapshot measures
of learning that do not capture the difficult and demanding intellectual skills that are the true aim
of a college education (2009, p. 3). She continues in the same paper on page 4 that the point, is
knowledge creation, not knowledge reproduction (Banta 2009, p. 4).
Brown, et al., bring it together nicely when they state that, ultimately, assessment should be
for students[as] a formative part of their learning experience and that students who develop
their test-taking skills, the best tend to succeed in assessment whether or not they are the most
qualified in their field (1999, p. 58). In other words, their first two functions of assessment,
capturing student time and attention and generating appropriate student learning activity,
along with the fourth and sixth, helping students to internalize the disciplines standards and
notions of quality and providing for quality assurance, are just as important as the fifth,
marking, which tends to get all the attention but most often seems more prone to be partially
-
8/8/2019 Prospectus Draft 2.6
6/12
invalid in the case of many traditional assessment measures (Brown, et al. 1999, p. 47). Whereas
authentic assessment brings the focus of assessment to student learning outcomes because, the
onus is on lecturers to be able to demonstrate that assessment is measuring well what it is
intended to measure, thereby forcing an increase in the validity of the assessments (Brown, et
al. 1999, p. 59).
It is important to note, that the point is not to throw away traditional assessment measures.
They can still serve some of the functions of assessment well. As Ratcliff, et al. state on page 28
of their 2001 text, formative and summative assessment methodologies provide the department
or program with evidence of their students learning. While traditional summative assessments
can, and should, support the functions of assessment processes, their results cannot stand alone to
inform the process of continual improvement of learning (Ratcliff, et al. 2001, p. 28). To really
get at that sixth purpose of quality assurance, a balance is needed (Brown, et al. 1999, p. 47).
Outcomes Assessment in Foreign Languages
Trends over the last couple of decades in foreign language methodologies have espoused
communicative goals of instructionyet, examinations in foreign language courses typically
are pen and paper exercises that single out discrete points of grammar or vocabulary (Higgs
1987, p. 1). Higgs raised the warning almost twenty-five years ago that if foreign language
educators really want to set communication as a goal for their students, then, assessment
procedures must test for communicative function (1987, p. 1). While these pen and paper exams
are still common place, language classes have seen an influx of authentic assessment measures
(Sullivan 2006, p. 590).
Most of these assessments have come in the form of portfolio-style projects. Banta
commented in her 2007 article on assessment that portfolio assessments would be the most
-
8/8/2019 Prospectus Draft 2.6
7/12
authentic because students develop the content themselves (p. 4). Studies of English as a Foreign
Language (EFL) learners have found that portfolio-style assessments have contributed to student
learning, especially when combined with other assessment measures, and that portfolio
assessments help students take ownership of their learning (Barootchi, et al. 2002 & Caner
2010). Additionally, one study found that some EFL students in writing courses preferred the
portfolio assessments over more traditional assessments (Caner 2010, p. 1).
There are many styles of portfolio assessments depending upon the specific needs of the
assessment, but the seemingly most popular version in language learning is the self-assessment
portfolio. The European Language Portfolio (ELP) was the model for the American adaptations:
LinguaFolio and the Global Language Portfolio (Cummings, et al. 2009, p. 1). These portfolios
present a learner-empowering alternative to traditional assessment (Cummings, et al. 2009, p.
1) Moeller points toward these self-assessment models as more valid forms of assessment than
more traditional assessment models (Moeller2010 Self-assessment in the foreign language
classroom). She describes LinguaFolio, in particular, as, a portfolio that focuses on student self-
assessment, goal setting and collection of evidence of language achievement (Moeller2010
LinguaFolio). The students set their language goals and, based on the evidences of their own
work that they collect, determine when their goals are met (Fasciano 2010, slide 9). Moeller
points out that if language educators are using LinguaFolio effectively, it will necessitate moving
away from teacher-centered methodologies and toward learning-centered outcomes because it is
by its very definition a learner-centered self-assessment tool that facilitates the processes of goal
setting and self-reflection and establishes intrinsic motivation in students (Moeller2010
LinguaFolio). Brown, et al. also mention that a major advantage of these types of assessments is
-
8/8/2019 Prospectus Draft 2.6
8/12
that they promote intrinsic motivation through personal involvement because the student is
taking charge of their learning (1999, p. 75).
Student Perception on Assessments
At present, students often feel that they are excluded from the assessment culture, and
that they have to use trial and error to make successive approximations towards the performances
that are being sought in assessed work (Brown, et. al. 1999 p. 58). Because of the reflections
gathered from their students, Brown, et al. encourage innovation in assessment (1999, p. 81).
They also stated that, to some students, conventional forms of assessment appear to have no
relevance to anything outside the university and are all about judging them, sometimes on a
somewhat arbitrary basis, rather than involving them in genuine learning (Brown, et al. 1999, p.
81). Instead they found that, students appreciate assessment tasks which help them to develop
knowledge, skills and abilities which they can take with them and use in other contexts such as in
their subsequent careers and they encourage, assessment which incorporates elements of
choice, because, it can give students a greater sense of ownership and personal involvement in
the work and avoid the demotivating perception that they are simply going through routine tasks
(Brown, et al. 1999, p. 81).
Research Questions
y What are the perceptions of undergraduate students related to traditional and authenticassessments used in an introductory Spanish course?
1. What do students think are the benefits or limitations of each type of assessment ontheir learning? Do students think these assessments reflect their learning?
2. How do students feel that these assessments can enhance or detract from theirlearning experience?
-
8/8/2019 Prospectus Draft 2.6
9/12
3. What factors do students feel affect the impact of each type of assessment on theirlearning?
4. What preferences do students express toward each type of assessment? What aretheir reasonings for these preferences?
5. What recommendations do students have for enhancing their perceivedeffectiveness of each type of assessment?
Methodology
Context/Setting
My research will be conducted by gathering data from former students of my fall 2010
beginning Spanish college courses. I teach at a public state funded institution and although we
are moving towards being a more research focused institution, the current focus is more aligned
with teaching. I am fortunate to have a great deal of freedom and control in my classroom. While
it is true that my general curriculum and my textbook are mandated by the tenured faculty of my
department, I am free to choose whatever path I believe will best help me achieve the course
goals. That is to say that while I am not free to choose what I teach, I am free to determine how
to teach. While I also receive feedback from peers once a year, I feel no other demands from any
supervisors on my teaching methods. This allows me to constantly experiment with ways to
improve how I teach my classes. I am currently in my fifth semester teaching these courses and I
can say with certainty that no two semesters have held very much in common outside of my
general teaching philosophy. I am constantly trying to improve my methods based on what I
have perceived as being effective. This hands-off situation created by the administration allows
me to be fluid in my methods. While the freedom to teach the way I feel is best has many
-
8/8/2019 Prospectus Draft 2.6
10/12
advantages, it also carries heavy responsibility; I have to rely on my perceptions of my students'
learning with little feedback from anyone else.
The classes I teach are capped at 28 students. This is a moderate number of students for a
beginning foreign language class. While it would be ideal to have a smaller number because it
would allow for more individualized attention, there are advantages to this class size as well.
With this many students it is easier to employ group learning strategies, allowing students to
facilitate their own and each others' learning processes. These large classes make it easy to use
traditional assessment measures because they are easy to administer and assess, even in so large
a group. Authentic assessments, like portfolio-style assessments, are more challenging to
administer and evaluate because they take longer to facilitate, collect, and assess but can provide
richer, more detailed feedback to students.
Participants
For this project, there will be two primary participant groups. First, I will invite the 79
students taking one of the three sections of beginning Spanish 1 in fall 2010 to participate in a
broad attitude survey (see Appendix B). These students run the gambit in class rank, from
freshmen to seniors, and in age, the youngest being 17 and the older ones being over thirty, as
well as in educational experiences and majors. I hope to see at least 40% of these students
respond to the survey to ensure a valid sample.
The second group of students I plan to solicit for this study will ideally be a group of
nine. I would like to ask three students from each of the three sections to participate in individual
interviews. I will choose which students to ask to participate based on their performance levels in
class. Ideally, I will find one high-, one mid-, and one low-performing student to ensure a
-
8/8/2019 Prospectus Draft 2.6
11/12
broader range of perspectives on the assessment measures. I will use a semi-structured interview
guide (see Appendix A) and record the interviews digitally.
Research Plan
I will complete my research by two methods: individual interview and a broad attitude
survey. First, I will employ a semi-structured interview method to ask the nine students,
described above, to explore their reflections on various forms of traditional and authentic
assessment tools used their previous semester of beginning level Spanish.I will use the semi-
structured interview guide (see Appendix A) to solicit their opinions on these assessments. I will
record the interviews digitally, after having each student sign an informed consent form (see
Appendix C).
Additionally, I will email an invitation to all 79 students taking one of the three sections
of beginning Spanish 1 in fall 2010 to participate in the broad attitude survey (see Appendix B)
which will be housed online. This survey will ask these former students to comment on their
engagement and motivation levels and their perceived effectiveness on their learning experiences
of the various assessment tools used throughout the course. I hope to have at least 32 of those
students surveyed to respond, which would be approximately 40% and enough for a valid
sample. I plan to send out an invitation to these students in early January, after grades have
posted for the semester, asking them to complete the survey by February 1, 2011. On January 31,
2011, I plan to send another email reminder asking students to complete the survey. If my results
are still under 50% participation, I will send a final email request during the second week of
February 2011.
Plan for Evaluation of Data
-
8/8/2019 Prospectus Draft 2.6
12/12
Time Line
Validity
look at Commitment AR study and Hendricks
Ethics and Subjectivity
look at Commitment AR study and Hendricks