students track their own learning with sas® data...
TRANSCRIPT
Students Track Their Own Learning with SAS® Data Notebook
Lucy Kosturko SAS Institute, Inc.
Jennifer Sabourin SAS Institute, Inc.
Scott McQuiggan SAS Institute, Inc.
Keywords: Self-regulated learning, Mobile learning, Digital notebooks, Metacognition,
Motivation, Learning strategies, Communication
Abstract
The adoption of the Common Core State Standards for many schools implies greater academic rigor on
behalf of the students and an expectation of modernized, thought-provoking instructional methods
from teachers. Ultimately, students are expected to not only master academic content, but also
demonstrate proficiency for skills associated with academic independence and self-regulated learning.
Extensive research has shown that these behaviors are associated with greater academic motivation and
success (Zimmerman, 1990). One successful method for developing self-regulatory behaviors is
encouraging students to set explicit, measureable goals, use quantitative and qualitative means to track
progress, and utilize data-supported evidence to reflect on academic achievements and refine previous
goals (Zimmerman, 2008). Therefore, the purpose of this research is to investigate the utility of a Digital
Data Notebook—an aggregated digital space for setting, tracking, and reflecting on academic goals—for
improving student’s self-regulatory skills. Similar non-digital tools have been successfully implemented
in the classroom (Bryne & Schaefer, 2006) and research-based evidence supports the use of resources
targeting specific, individual facets of self-regulated learning (Zimmerman, 2008). However, to date little
empirical work has been devoted to analyzing the efficacy of digital tools for the regulation of learning
goals. Consequently, this work will explore the use of a Digital Data Notebook for fostering student self-
regulatory behaviors through direct modelling and capitalizing on the affordances of technology for data
tracking and visualization.
Theoretical Framework
The ultimate mission of the Common Core State Standards Initiative is to ensure all students leave high
school with the necessary skillset to thrive in both career and college settings (Common Core State
Standards, 2010). Therefore, the Standards challenge students to not only build and master academic
content but also become independent, high-level thinkers. Students should leave high school equipped
with the cognitive and metacognitive abilities required of the lifelong learner. In short, today’s students
should embody the characteristics of a self-regulated learner.
Self-regulated learning is a term used to describe the behaviors of students who actively control their
own learning by “[sustaining] cognitions, behaviors, and affects that are systematically directed toward
the attainment of goals” (Schunk, 2008, p. 245; Schunk & Zimmerman, 2003; Winne & Hadwin, 1998;
Pintrich, 2000). Self-regulated learners are not only equipped with a sufficient set of learning strategies,
but also have the motivational control to put forth the necessary effort to engage in these cognitive
processes (Pintrich 2000). Therefore, self-regulation is often represented by three components:
metacognitive awareness, strategy use, and motivation (Zimmerman, 2000). While students often show
great variance in their self-regulatory behaviors (Ellis & Zimmerman, 2001), proficiency of such skills has
shown to be influenced and improved through direct instruction and modelling (Zimmerman, 2002).
Furthermore, there is evidence that students who are better able to regulate their learning in an
intentional and reflective way often demonstrate greater academic motivation and achievement
(Zimmerman, 1990).
Although encouraging self-regulation is critical for the Common Core era student, the new curriculum is
intentionally ambiguous in techniques for increasing these behaviors. According to official
documentation, “The Standards do not mandate…the full range of metacognitive strategies that
students may need to monitor and direct their thinking and learning. Teachers are thus free to provide
students with whatever tools and knowledge their professional judgment and experience identify as
most helpful for meeting the goals set out in the Standards” (Common Core State Standards, 2010).
With respect to developing self-regulatory skills, research-based best-practices suggest teachers should
turn to contextualized methods that require students to explicitly engage in behaviors such as setting
specific, measureable goals, diligently monitoring progress toward such goals, engaging in strategies
that encourage metacognitive awareness, and reflecting on previous actions (Zimmerman, 1990; Schunk
& Swartz, 1993; Schunk & Zimmerman, 2003; Schunk, 1990).
One method for modelling and developing students’ self-regulatory behaviors proposed by the Baldrige
Education Criteria for Performance Excellence is through the use of a student data tracking system or
data notebook (Byrne & Schaefer, 2006; Baldrige, 2013). Data notebooks provide a single aggregated
space where students can set their own measureable goals, evaluate and reflect on their progress, and
refine their goals accordingly. Empirical studies have shown significant differences in academic
motivation between students who set and track their own goals and students who are assigned goals
(Zimmerman, 2008). Furthermore, as reported by Byrne and Schaefer (2006) several qualitative reports
such as “it is very empowering for students, giving them ownership of their education” support the use
of data notebooks in the classroom.
Beyond self-regulatory behaviors, the Common Core State Standards emphasize the use of data and
evidence as a necessary component for effective communication at all grade levels. Specifically, anchor
standard CCSS.ELA-Literacy.CCRA.SL.4 and anchor standard CCSS.ELA-Literacy.CCRA.Sl.5 respectively
state that students should “present information, findings, and supporting evidence such that listeners
can follow the line of reasoning and the organization, development, and style are appropriate to task,
purpose, and audience” and “make strategic use of digital media and visual displays of data to express
information and enhance understanding of presentations” (Common Core State Standards, 2010). With
respect to reflecting upon and detailing one’s academic progress, explicit goal setting and monitoring
provides an excellent platform for utilizing data and evidence for communication.
Therefore, the present research sought to provide preliminary findings for the following research
questions: 1) How do teachers perceive Digital Data Notebooks affect students’ self-regulatory behaviors
and academic achievement?, and 2) How are students using Digital Data Notebooks as a data point
when communicating educational progress?
Research Method
Participants
The SAS Data Notebook app was developed in partnership with teachers from a large socioeconomically
diverse elementary school in the Midwest (644 students, 61% free/reduced-price lunch); therefore,
participants for this study were recruited from the partner elementary school, as well as a blanket call
for participation extended to other teachers using SAS Data Notebook in their classroom via social
media outlets.
A total of eight participants were included in the study ranging in teaching experience, educational
technology experience, and level of comfort with technology in the classroom (see Figure 1).
Participants also represent a range of grade levels from kindergarten to 8th grade. All participants had
experience using a non-digital form for data notebooks in their classroom prior to implementing SAS
Data Notebook.
Figure 1. Participant teaching and technology experience and level of comfort with using technology in the
classroom.
Materials
Teacher Demographic Inventory is a 7-item survey to gather information about participants’ teaching
experience, technology use, and familiarity with traditional data notebooks (Appendix A).
SAS Data Notebook (Figure 2) is a free tool that brings data notebooks from the traditional 3-ring binder
to the iPad. Data Notebook provides tools for students to set personal goals, monitor their own learning,
reflect on previous work, and communicate their progress to teachers and parents using real data.
Within the notebook, students can use the built-in templates to create mission statements, set goals,
generate checklists, reflect using plus/deltas, create and practice spelling lists, and plot histograms. Data
Notebook also includes a blank page and scratch paper template that enables students to load pictures,
drawings, and more into their notebook. Students can also add sections for additional organization.
Thus, students’ goals, daily progress, and achievements are aggregated in one place for easy access
allowing for reflection and communication with parents and teachers.
0
1
2
3
4
5
P1 (10+) P2 (6-10) P3 (10+) P4 (10+) P5 (6-10) P6 (10+) P7 (6-10) P8 (3-5)
Rat
ing
Participant (Teaching experience in years)
Technology Experience Technology Comfort
Figure 2. SAS Data Notebook.
Teacher Reflection Survey is a 15-item, researcher-constructed metric designed to collect use,
satisfaction, efficacy, communication and observational data regarding the use of SAS Data Notebooks
over the course of the 2013-2014 school year. Questions were answered using a 5-point Likert-scale,
and open-ended reflections in order to extend and understand the effect of using the Data Notebook
app had on students’ self-regulatory behaviors (Appendix A).
Procedure
Over the course of the 2013-2014 school year, teachers and students utilized the SAS Data Notebook
tool as part of their regular coursework. Depending on how the teacher decided to implement Data
Notebook, students updated their notebooks by collecting and archiving data, revising goals, and
reflecting on performance on a daily or weekly basis. Notebooks were also reportedly used during
parent-teacher conferences as a data point for student reviews. Although all teachers at the target
school were required by administration to implement Data Notebook in their classroom, only those who
choose to participate in the research study were asked to complete the Teacher Demographic Inventory
and Teacher Reflection Survey. Participants were asked to complete the surveys in the spring of 2014.
Results
Using a mixed-methods exploratory design, quantitative ratings of overall satisfaction, student self-
regulatory behaviors, and communication with parents were triangulated with qualitative reports to
further validate the data.
How do teachers perceive Digital Data Notebooks affect students’ self-regulatory behaviors and
academic achievement?
To understand teacher’s perceptions of self-regulatory behaviors, participants were asked to specifically
reflect on the components of self-regulated learning: metacognition, strategy use, and motivation
(Zimmerman, 2000). As reflected in Figure 3, on average, participants felt SAS Data Notebook did have a
positive effect on their students’ metacognition (M = 3, SD = 1.18), strategy use (M = 3.63, SD = 1.44),
and motivation (M = 3.63, SD = 1.40). In line with these findings, participants commented that their
students “are eager to enter their data and see their scores on the graphs” and “like to track their data
and see progress.” With regard to metacognition, one teacher reported that having “successful data to
support it” was useful when “students are trying to meet their weekly and daily goals.” Finally, teachers
felt that using Data Notebooks was beneficial to overall academic performance (M = 3.75, SD = 1.38).
Figure 3. Participants’ perceptions of SAS Data Notebook’s effect on students’ self-regulatory behaviors and
academic performance.
How are students using Digital Data Notebooks as a data point when communicating educational
progress?
Similar results were found for using SAS Data Notebook as a means to communicate educational
progress (Figure 4). On average, participants felt the app was beneficial for aiding their communication
with parents and students (M = 3.38, SD = 1.68). One noted that “Keeping track of how each child does
is helpful when I meet with each student. During our conference time, we use their data from SAS to
discuss weekly and daily goals.” Another stated, “During conferences, I shared the students’ SAS work
with their parents. Also, I use their data to share with parents during the school year.” Furthermore,
participants felt the app helped students communicate with their parents about their academic
performance (M = 3.89, SD = 1.33). One teacher pointed to the portability of the iPad as particularly
useful for communication stating that “It's nice to have the data for each student right there on their
iPad. It can also be shared with parents daily when the child takes his/her iPad home each night.”
0
1
2
3
4
5
P1 P2 P3 P4 P5 P6 P7 P8
Rat
ing
Participant
Motivation Strategy Use Metacognition Academic Performance
Figure 4. Participants’ perceptions of SAS Data Notebook’s effect on teacher-student/parent
communication and student-parent communication.
Implications
The educational rigor mandated by the Common Core State Standards challenges students to go beyond
what has been traditionally required by state curricula. Furthermore, as students approach college and
career readiness, they are expected to demonstrate academic independence through both cognitive and
metacognitive actions and embody the characteristics of a self-regulated learner. Such qualities (e.g.,
motivational control, metacognitive awareness, and effective learning strategy implementation) are
beneficial for thriving in the Common Core era.
While self-regulatory behaviors are believed to be acquired over time through guidance and practice,
research suggests students, even those of the same age, demonstrate a wide range of proficiency (Ellis
& Zimmerman, 2003). Such findings motivate the need for more explicit, contextualized self-regulated
learning instruction. Flexible tools, such as SAS Data Notebook, provide a great resources for
encouraging students to take control of their own learning across the content areas as reflecting in the
findings from this study. Overall, integrating the SAS Data Notebook app was perceived to have a
positive effect on students’ self-regulatory behaviors and overall academic achievement. Furthermore,
students’ Data Notebooks were posited as a useful tool for communicating educational progress. In fact,
only one of the participants indicated she does not plan to integrate SAS Data Notebook in her
classroom instruction next year.
However, it should be noted, implementing programs like SAS Data Notebook must be done with care.
Students’ self-regulatory skills are developed overtime and proficiency often requires direct instruction
and modelling (Zimmerman, 2002). Therefore, instruction around the use of these tools and the
motivation for setting goals, monitoring progress, and reflecting on results is critical. In fact, two of the
eight participants in this study felt SAS Data Notebook did not improve their students’ self-regulatory
skills; however, these teachers pointed to conceptual misunderstanding as a potential problem. One
noted that she, “will spend more time in August emphasizing how to stay organized and updated. This
0
1
2
3
4
5
P1 P2 P3 P4 P5 P6 P7 P8
Rat
ing
Participant
Teacher-Student/Parent Student-Parent
will alleviate issues that I have encountered this year.” The other said he will, “have more upfront
training for students to be more independent with their Data Notebook.”
Finally, the results should be generalized with care as there are several, notable limitations. First, the
sample used for this investigation was very small and does not necessarily represent the larger
population of classrooms utilizing the SAS Data Notebook app. Furthermore, the degree to which Data
Notebook was integrated over the course of the 2013-2014 school year was not controlled nor was the
manner in which students used the app; therefore, direct comparisons between the participants should
be made with caution. Moreover, it is important to consider students’ self-regulatory skills prior to using
the Data Notebook app in order to better understand the direct effect of using the tool throughout the
school year. Lastly, the measures used for this study were researcher-constructed and self-report in
nature leaving room for measurement error. More rigorous investigations with more vigilant
experimental control, sufficient power for more sophisticated statistical analyses, and validated
measures for each of the study’s variables are necessary to ensure the validity of these preliminary
findings. Additionally, as self-regulated learning behaviors develop overtime, it would be interesting to
understand the differences in effectiveness of using Data Notebook between grade levels. Nonetheless,
design considerations for SAS Data Notebook are grounded in research-proven principles and the
findings from this exploratory study show promise for supporting the development of students’ self-
regulatory skills.
References
Byrne, S. & Schaefer, C. (2006). The Baldrige Program: Self-Assessment for Continuous Improvement.
Principal, 24-27.
Ellis, D. & Zimmerman, B. J. (2001). In H. J. Hartman (Ed.), Enhancing self-monitoring during self-
regulated learning of speech (pp. 205-228). Dordrecht: Kluwer.
National Institute of Standards and Technology (2013).
Baldrige Criteria for Performance Excellence 2013-2014. Washington, DC. National Governors
Association Center for Best Practices & Council of Chief State School Officers. (2010).
Common Core State Standards for English language arts and literacy in history/social studies, science,
and technical subjects. Washington, DC: Authors.
Pintrich, P. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. Pintrich, &
M. Zeidner (Eds.), Handbook of self-regulation. San Diego, CA: Academic Press.
Schunk, D. H. (2008). Attributions as motivators of self-regulated learning. In D. H. Schunk & B. J.
Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 267-
286). New York: Routledge.
Schunk, D. H. & Schwartz, C. W. (1993). Goals and progress feedback: effects on self-efficacy and writing
achievement. Contemporary Educational Psychology, 18(3), 337-354.
Schunk, D. H. & Zimmerman, B. J. (2003). In W. M. Reynolds & G. E. Miller (Eds.) Self-regulation and
learning (Vol. 7, pp. 59-78). New York: Wiley & Sons.
Winne, P. H. & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A.
C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 279-306). Hillsdale, NJ:
Erlbaum.
Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: an overview. Educational
Psychologist, 25, 3-17.
Zimmerman, B. J. (2002). Becoming a self-regulated learner: an overview. Theory into Practice, 4(2), 64-
70.
Zimmerman, B. J. (2008). Goal setting: A key proactive source of academic self-regulation. In D. H.
Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and
applications (pp. 267-286). New York: Routledge.
Appendix A
Teacher Demographic Inventory
1. What grade(s) do you teach? (Pick all that apply)
Kindergarten
1st grade
2nd grade
3rd grade
4th grade
5th grade
6th grade
7th grade
8th grade
Specialist
Administrator
Tech Coordinator
2. How long have you been teaching?
1-2 years
3-5 years
6-10 years
10+ years
3. Rate your level of experience with using technology in the classroom.
1 - No Experience
2 - Novice
3 - Moderate Experience
4 - A lot of Experience
5 - Expert
4. Rate your comfort level with using technology in the classroom.
Very Uncomfortable
Uncomfortable
Uncomfortable at times
Comfortable
Very Comfortable
5. What technology(ies) do you use in the your classroom? (Pick all that apply)
iPad/iPod/iOS devices
SmartBoard
Andriod mobile devices
Chromebooks
Desktop computers
Laptop Computers
No technology/I am not a classroom teacher
6. On average, I use SAS Data Notebook in my classroom:
Daily
2-3 times per week
Once a week
Never
7. Before using SAS Data Notebook, I used: (Pick all that apply) (Required.)
3-ring binders
other
I did not use data notebooks in my classroom
Teacher Reflection Survey
1. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' motivation.
Disagree
Slightly disagree
Neither agree nor disagree
Slightly agree
Agree
2. Explain.
3. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' study habits/strategies.
Disagree
Slightly disagree
Neither agree not disagree
Slightly agree
Agree
4. Explain.
5. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' metacognition.
Disagree
Slightly disagree
neither agree nor disagree
Slightly agree
Agree
6. Explain.
7. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' academic performance.
Disagree
Slightly disagree
Neither agree nor disagree
Slightly agree
Agree
8. Explain.
9. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my communication with parents about students' academic performance.
Disagree
Slightly disagree
Neither agree not disagree
Slightly agree
Agree
10. Explain.
11. To what extent to you agree with the following statement? I believe SAS Data Notebook improves students' communication with their parents about their academic performance
Disagree
Slightly disagree
Neither agree nor disagree
Slightly agree
Agree
12. Explain.
13. I plan to integrate SAS Data Notebook into my classroom again next school year.
No
Yes
Undecided
14. If yes, what will you change for next year? What will you keep the same?
15. Do you have any best practices, lessons learned, or advice for integrating SAS Data Notebook into the classroom?
Submit Your Answ ers