A Digital Reflection Tool to Support Transformative Reflection for Primary School Teachers Bernice d’Anjou

Industrial Design Eindhoven, the Netherlands

b.e.m.d.anjou@student.tue.nl

ABSTRACT Primary education in the Netherlands is struggling with a number of challenges [6, 17], most importantly that they are dealing with a feeling of powerlessness in their daily working life. There is an opportunity to give teachers the feeling that they have more influence and control in their work environment. Research shows that when practicing the act of reflection, there is no question that when teachers are able to combine this with experience, they develop themselves professionally [7, 36, 11].

This paper investigates what the requirements of a digital reflection tool would be to support primary school teachers in learning to reflect transformatively [14] on their professional development. This design research study investigates the abovementioned through a design probe, a digital reflection tool, that is based on certain design assumptions, as a starting point for discussion. This tool was tested qualitatively with participants from a primary school in the Netherlands. The results of this study describe design guidelines for tools to support reflective practice of primary school teachers.

Author Keywords Transformative reflection; digital reflection tool; professional development; design research.

ACM Classification Keywords Human-Computer Interaction (HCI)

INTRODUCTION Primary education in the Netherlands is struggling with a number of challenges, mostly because of the situation teachers find themselves in. The teachers have low wages (especially in comparison with the substantially higher wages for teachers in secondary education with the same degree), high work pressure and there is a shortage of teachers [6, 17]. Next to this, the societal status of the profession of primary school teacher has been decreasing for a long period of time. These issues led to two strikes in 2017, which was quite exceptional [6]. These strikes imply that the teachers are in disagreement with governmental regulations (influencing salary inequality, work pressure and other additional regulations), but most importantly that they are dealing with a feeling of powerlessness in their daily working life.

There is an opportunity to give teachers the feeling that they have more influence and control in their work environment.

As “transformative intellectuals” [29], who attempt to reflect on and influence their work, teachers can proactively start to take control of their working lives. Through reflection they can become more empowered decision makers on their teaching practice [12]. Research shows that when practicing the act of reflection, there is no question that when teachers are able to combine this with experience, they develop themselves professionally [7, 36, 11], and will result in more effective teaching. As reflective learning is “a cognitive process based on intrinsic and extrinsic motivation of an individual” [13, p.2], this process can’t be enforced (e.g. enforced by the school management). This does imply however that other factors (such as work pressure, salary deprivation and external regulations) should not block or frustrate professional development [20].

The continuous professional development of teachers is a widely discussed issue in the field of educational design. “To cope with ongoing changes like the increasing diversity in the classrooms, new learning psychological insights and technological developments, teachers need to keep learning during their career” [34, p. 1]. This growing complexity of the practice of teaching forces teachers to continuously work at their professional development [10]. Professional development is a necessity to learn about new techniques and methods as they may have been developed elsewhere. Just relying on earlier education, training and experience does not suffice for a professional career. The importance of continuous professional development is evident, but the implications on how to achieve a continuous process differ. In this paper, the combination of reflective practice and interaction design is explored.

Reflective practice by teachers is mostly seen as a personal responsibility and is not always properly stimulated within schools. Teachers are often focused on finding quick solutions to problems of practical issues, without reflecting on the underlying problems [22, 25, 33]. This approach might be effective to keep up with hectic daily working life, but runs the risk of stagnating the professional development [25]. To prevent these unsystematic reflections, reflection models are designed for teachers, by various educational experts [23, 24, 26]. These models provide the teachers with a helping hand, but are still not focused on the effective implementation of reflective practice in their daily working lives. Farrell [12] suggests the following activities as opportunities for reflection: journal writing, group

discussions, classroom observations and teaching portfolio’s. As most professional development courses focus on the latter three activities, this paper will focus on the activity of journal writing and how that can be supported by HCI.

This paper investigates what the requirements of a digital reflection tool would be to support primary school teachers in learning to reflect on their professional development. Different levels of reflection need to be considered when supporting the learning process of reflecting, and in this study the level of description, reflective description, dialogic reflection up until transformative reflection are supported [14]. Transformative reflection [14] is a way of critically assessing your personal assumption that influence your actions and look at changing them. Thus, the research question that this paper will discuss is “In what way does a digital reflection tool for individual reflection support primary school teachers in learning to reflect transformatively [14] on their professional development?” A design probe used to investigate the aforementioned research question is based on certain design assumptions, based on both literature and a pilot research, that function as a starting point for discussion.

Implications for design for reflection and reflective practice have been of interest to the community of Human-Computer Interaction and other related communities for a while now. Previous research has been done on technology to support learning [31] and reflection in learning [5]. Despite this increasing interest, designing for reflection is still in its infancy [14, 35]. The insights from this paper therefore are interesting for HCI designers and educational designers in terms of product development or further research.

This paper describes a design research study. Firstly, related work and background research is described, followed by the method and implemented design in this study. Thereafter the results are described and design guidelines for future work are given. In this way, this paper translates the abstract findings into concrete guidelines that could be a starting point for design for HCI or educational designers as mentioned above, who focus on the reflective practice of primary school teachers. This is followed by the discussion of the study and the conclusion that summarizes the value and insights of this paper.

RELATED WORK Even though reflective journaling by both teachers and teacher students has been studied widely, research on the integration of this reflective practice into HCI is still scarce. Previous research has been done on technology to support learning, by creating a framework for Computer-Supported Collaborative Learning [31] and reflection in learning [5].Other studies have focused on the recording capacities of technology to support reflection, by means of an integrated camera in clothing [15], life-logging, which focuses on

recording every day conversations [1], or the use of online discussion forms to promote reflection for students [37].

However, the implementation of HCI into the daily working life of teachers to enable reflection on professional development through the use of reflective journaling has not been researched before, thus creating a gap for the research done in this paper.

BACKGROUND RESEARCH

Professional Development Nowadays, the professional development of primary school teachers is divided into two categories of learning: formal learning and informal learning. Formal learning comprehends organized professionalization activities (both inside and outside the work environment) such as taking part in courses or workshops by external providers [21]. Most of these courses have a traditional form: often not situated in the work environment and not directly focused on specific issues from the teaching practice [4, 39]. This category of learning has been researched more closely in the last few years, however, and certain guidelines for formal learning activities have been developed [39].

Informal learning is often unorganized and takes place within the work environment, frequently in interaction with co-workers [21]. This mostly comprehends reflection on one’s own practice, feedback sessions with co-workers, observations etc. Often in teacher education literature [40], it is stressed that reflective practice takes the form of critical reflection on one’s own actions. Although there can be no mistake that this individual reflection is essential for the learning process, little research has been done on how to stimulate and integrate these activities in the daily life of teachers [27, 34]. As this category of learning with its’ practical implications of professional development is least explored by researchers, it will be the main focus of this paper.

Reflective practice Reflective practice can be seen as the “conscious re-evaluation of past experiences with the goal to learn from them in order to guide future behavior” [13, p. 3]. This is in line with the definition given by Dewey [11], who defines reflection as an “active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusions to which it tends”. Others, who built on Dewey’s ideas, such as Argyris and Schön [2], divide reflection into two categories: single loop learning and double loop learning. Single loop learning is defined as the evaluation of your actions on a tacit level. Double loop learning comprises that the levels of planning, action and evaluation are made explicit, which can be done through e.g. individual reflective practice [12]. Schön also defined various forms of reflection, relevant for different moments of time during a learning process, namely ‘reflection-in-action’ and ‘reflection-on-action’. As this study focusses on

the evaluation of one’s teaching after the class, the specification of this type of reflection would be ‘reflection-on-action’ on the teachers’ professional development. However, if we look at reflection as the center of the continuous professional development process, it can also be seen as ‘reflection-in-action’, in a broad sense.

Fleck and Fitzpatrick [14] collected several aspects from literature, that need to be defined in order to be able to design for reflection, such as the purpose of reflection, conditions for reflections and the levels of reflection. The purpose for reflection can include learning for further reflection or self-development [28]. Also, the conditions for reflection are the time created for reflection [2], that reflection needs to be developed with support [2, 18], and that there needs to be encouragement to do so [2, 18]. Furthermore, Fleck and Fitzpatrick describe five different levels of reflection from various literature sources [14], namely R0 ‘Description’, R1 ‘Reflective Description’, R2 ‘Dialogic Reflection’, R3 ‘Transformative reflection’ and R4 ‘Critical Reflection’. In this study, these levels and their descriptions were combined with additional definitions from Prilla et al. [32] in a matrix to classify the levels of reflection. A preview of this matrix is shown in Fig. 1, the full matrix can be found in appendix 2. As the highest level of reflection, R4 ‘Critical Reflection’, is defined as “taking a broader historical, socio-political and moral context of schooling” [19] into account while reflecting, which is reported to occur very rarely, this paper focuses on supporting reflective practice up to the level R3 ‘Transformative Reflection’

Figure 1. Part of the levels of reflection matrix.

The support of reflection Farrell, as an educational academic with the focus on reflective practice, [12] developed an overall framework for reflective practice for teachers. Herein he describes several activities for teachers to engage in reflective practice: group discussions, classroom observations, teaching portfolio’s and journal writing. Others [16, 8] also describe the benefits of reflective journals to promote reflective thinking. Smyth [38] argues that asking a series of questions that would be answered in written journals could develop and promote reflective thinking. One example of this is a study by Clarke [9], in which Australian teacher students are engaging in reflective journals in order to understand the reflection process and its influence on the professional learning development of the teacher students.

METHOD

Study Approach This study was executed in collaboration with a primary school “De Vuurvogel” in Helmond, the Netherlands. The participants mentioned herein are currently teaching at that school at different class levels and working together towards the growth of their professional development.

Pilot Research. As this research focuses on the requirements of a digital reflection tool that would support primary school teachers in learning to reflect on their professional development, the first things that needed to be uncovered were the practical opportunities for reflection in the daily working life of those teachers. Through an interview with two participating teachers and classroom observations, the context and practicalities of the work environment of primary school teachers was mapped. This was followed by a one-week pilot research, in which 5 participants were asked to answer reflective questions each day through journaling in a physical booklet. The goal of this pilot research was to gain inspiration on the implementation of a reflection tool and to uncover more practical preferences of the teachers. The physical booklet included reflective questions of different levels of reflection, which the teachers had to fill in each day, on different moments of the day. This lead to insights on the preferred timing of reflection, duration of reflection and medium through which reflection could be promoted. These insights and their implementation in a design probe used in the broader research is elaborated on in the section ‘Design’.

Main research. The aim of this research was to investigate “In what way does a digital reflection tool for individual reflection support primary school teachers in learning to reflect transformatively [14] on their professional development?”. The digital reflection tool was developed after insights from the pilot research combined with suggestions from literature and was used as a design probe for this design research. Through mixed methods, the research question was investigated, with a focus on qualitative methods

complemented with a quantitative questionnaire on perceived self-efficacy [3].

Before the start of the test period, there was a meeting between the participants and the researcher, to introduce the aim of the study and explain how the digital reflection tool worked. Also, a questionnaire focused on the perceived self-efficacy of the reflective skills of the participants was conducted and a consent form was signed. The test-period had a duration of four weeks, in which the 5 participants were asked to complete reflective questions twice a week through the use of the digital reflection tool. These questions, of a duration of 15 minutes, were planned on Wednesday and Friday afternoon, but free to be rescheduled as the participants saw fit. After the first week of completing the reflective questions, each of the participants was interviewed for 15 minutes on the usage and effect of the digital reflection tool on their professional development. The weekly questions were intermediately evaluated after they were completed by the researcher. The reflective questions of the following week were adjusted to the insights of the intermediate evaluation by the researcher. In this way, the questions were personalized to every participant and were conform with the design assumptions of the digital reflection tool, which are elaborated on in the section ‘Design’. After the test-period of four weeks, each participant was interviewed again, for 30 minutes, on the usage and effects of the digital reflection tool on their professional development. Also, the same questionnaire on perceived self-efficacy on their reflective skills and their confidence in the digital reflection tool was conducted.

Data collection and analysis The data sources from this research consisted of qualitative, structured interviews and the results of a self-efficacy questionnaire. The data from the interviews was collected through audio recordings, that were transcribed afterwards. The self-efficacy questionnaire consisted of 21 statements, 16 on the perceived self-efficacy of the reflective skills of the participants and 5 on the confidence the participants had in the support of the digital reflection tool. These statements were given any amount of points on the scale from 0 to 100, with 0 representing ‘no confidence’ and 100 representing ‘absolute confidence’. The basis of the perceived self-efficacy questionnaire was derived from the validated research on perceived self-efficacy questionnaires [3] and the statements of the questionnaire were adjusted to the context of this research, namely reflective practice. An overview of the data collection and their timing in the test-period, can be seen in Table 1.

The collected qualitative data was divided into different topics, that formed the basis for an inductive affinity diagram. These topics and their corresponding clusters of data were divided into sub clusters, which were helpful in giving more insight into the data.

Week of the test period

Data collected QUAN/QUAL

pre Self-efficacy questionnaire QUAN

1 - -

2 Individual interviews

(15 min) QUAL

3 - -

4 - -

after Individual interviews

(30 min) + self-efficacy questionnaire

QUAL +

QUAN

Table 1. Overview of the data collection of this study.

The quantitative results from the perceived self-efficacy questionnaire were analyzed through the use of descriptive statistics, as the number of participants was too low for a statistically sound confidence interval. The averages of the scores on the 16 statements on perceived self-efficacy on reflective skills of all participants were visualized in a column graph (see Fig. 6) and the averages of the scores on the 5 statements on confidence of the participants in the digital reflection tool of all participants were similarly visualized in a column graph (see Fig. 7). Also, the standard deviation is visualized in these graphs, to show the validity of the comparison of data of the self-efficacy questionnaire scores collected before and after the test period. These qualitatively analyzed topics and descriptive statistical analysis will be elaborated on in the section ‘Results’.

Participants In collaboration with teacher-researcher D. Mertens, the primary school “De Vuurvogel” in Helmond was used as the context of this research. This collaboration entailed the intertwining of the research of D. Mertens, on effective professional development courses, and this research, on reflective practice on professional development. Thus, the participating teachers were simultaneously following the professional development course, given by D. Mertens.

Q# Gender Part-time/Full-time

1 Female Full-time

2 Female Full-time

3 Female Part-time 4 Female Full-time

5 Female Part-time

6 Female Part-time Table 2. Participants.

In total N=6 participants (see Table 2) participated in this research, all being female teachers of different class levels. The strengths of this research lie in the in-depth qualitative

research methods, thus the number of participants involved in this research. Participants 1 to 5 were involved in the pilot research and participants 2 to 6 were involved in the main research. When referring to a specific participant, the participant will be indicated as “P#”. For example, participant 1 will be abbreviated to P1.

DESIGN

Design assumptions To investigate in what way a digital reflection tool can best support primary school teachers in learning to reflect transformatively [14] on their professional development, a digital reflection tool was designed according to certain design assumptions. These design assumptions were based on the insights of the pilot research combined with suggestions from literature and were the starting point in the discussion on what requirements such a tool should have. The design assumptions of the digital reflection tool are:

i. The tool is able to adjust the level of depth of the reflection questions to the abilities of the participants and gradually built-up in more depth each week, which will have a positive effect on their level of depth in reflections. The levels of depth and content of questions were based on a matrix, derived from [14, 32].

ii. The reflection questions adapt to the content of the reflections of the week before, which will have a positive effect on the users’ experience.

iii. The medium and timing of the tool are integrated well into the daily work life of the user [Preliminary results of the pilot study].

Insights from Pilot research The practical requirements for a reflection tool were discovered after the pilot research, through the use of a reflection booklet (see Fig. 2.) by the participants for one week. The preferred medium for such a tool was a digital form, as that “allowed you to express yourself more fully and could be easily connected with the other work you have to do on your laptop” [P2, P3]. Also, the duration for a reflective exercise, when completed daily, was five to ten minutes [P1, P2, P3, P4, P6]. However, when the exercises occurred fewer times in the week, the duration could be extended to 15 minutes [P1, P3, P6]. Furthermore, the moment in the day that was most preferred to do some reflective exercises, were on the Wednesday or Friday afternoon, as those afternoons were used for lesson preparations. “The reflective exercises are very convenient when done during the preparation of next lessons, as you can incorporate the results in the next lesson plan” [P3]. Next to these practical requirements, I discovered, as I reviewed the content of the reflections, that the teachers were reflecting very descriptively and were not going into

depth. This provided me with the opportunity to incorporate the “learning to reflect transformatively” aspect into the research question, as in-depth reflections provide the reflective practitioner with the most insights on their behaviour and possible improvements [12].

Fig. 2. The questions within the reflection booklets of the pilot

research (in Dutch)

Research Prototype The digital reflection tool consists of a personalized website for every participant, that contains an introduction, an exercise schedule, and links to the exercises. The exercises are made in Google Docs, that enables usage by both the participant and the researcher (see Fig. 5.).

The personalization of the website is characterized by a unique lay-out, but the content of each website is the same (see Fig. 3). The website starts with introducing the goal of this research and gives a short description of the test-period. This is followed by a schedule with the preferred dates by which the participants need to complete the exercises and the dates of the interviews. The schedule is followed by a preview of each exercise, that has the function of a link to the exercise within Google Docs (see Fig. 4.).

Fig. 3. Screenshot of the first page of the personalized

reflection tool (in Dutch).

Fig. 4. Screenshot of the links to the reflection question of the

reflection tool (in Dutch).

Fig. 5. A picture of the reflection exercises within Google Docs

in use (in Dutch).

Content of reflection exercises The content of the specifically tailored reflection questions in the digital reflection tool was adjusted by hand by the researcher, however the conceptualization of the tool is that through key words, categorized in an algorithm, the tool can be made into a fully automatic system. This tool is a manpowered version to test design assumptions, which when results have been yielded can form the basis for such an algorithmic system.

The reflection questions were divided into two parts, based on the two subjects within the professional development course: feedback moments and the interpersonal connection with pupils. In this first exercise the questions were based on descriptions/revisiting [14, 32] of relevant moments focused on feedback or interpersonal connections that occurred that week (see Figure 1, for a description of the different levels of reflection)

In the second week, the reflection questions were focused on both reflective descriptions and dialogic reflections [14, 32] in which the users were exploring relationships between experiences/theory and looking for different perspectives on one situation. The descriptions of the users of moments from week 1 were used as an introduction to the reflection questions of week 2. In this way, a connection is made

between these exercises and the situations can be analyzed in more depth by the users. Also, a short introduction on the concepts of ‘different perspectives’ was given before the questions, to make sure there was no communication gap between the researcher’s and users’ interpretation of concepts. Also, the level of depth was adjusted to the answers of the participants in the week before. E.g. if a participant went more in-depth with the answers than proposed by the questions, the questions of the following week were made even more in-depth and vice versa.

In the third week, the reflection questions were focused on dialogic reflection and transformative reflection [14, 32], in which the users had to look for patterns in the situations and uncover personal assumptions that influence their thinking and actions.

In the last week of using the tool by the users, the reflection questions were focused on transformative reflection [14, 32], in which the users had to describe any changes done or planned. This week’s exercises were also meant as a closing and summarizing exercise.

RESULTS

Content of reflective exercises The level of depth of the reflection questions was highly valued by all the users, as it created in their eyes “awareness of your own actions and professional growth” [P2, P3, P4, P5, P6]. P3: “Within education, you develop a certain way of doing things and after some time you forget why you do the things that way. With these questions, you create that awareness again on your actions, if they have the desired effect and if they should change or not.” The adjustment of the level of depth of the questions to the level of the teachers also created a positive experience as “it felt as if the questions were adjusted to my level, which felt personal and pleasant” [P3]. The gradual built-up in depth of the reflective questions guided the learning process of the teacher to reflect more critically. P3: “Because of the step-by-step built up of deepening questions, I became aware of what layers of my actions consist of. My learning process of thinking this critically was guided by this built-up.”

The personalized connections between the weekly exercises created the feeling of being able to go more in-depth with certain situations. P2: “Because of the connections between the exercises it felt like relations were easy to discover and made it easier to go more in depth of certain situations. This was very pleasant for me.” The part-time teachers [P3, P5, P6] experienced these connections as very pleasant, while the full-time teachers rather wanted to use new situations that happened during the week, as for them the interaction with the children is very intensive every day and the memories of one specific situation faded more quickly [P2, P4]. P5: “The questions became more focused after the first week, which enabled me to reflect more focused.”

Furthermore, the personalization of the questions had a positive effect on the non-repetitive perception of the teachers. Without the personalization, which was the case for the questions in week 1 and week 4, the questions were perceived as repetitive (even when they were not a repetition), which created annoyance of the teachers [P2, P3, P4, P5]. With the personalization, which was the case for the questions in week 2 en week 3, the questions were perceived as non-repetitive (even when they were), which created a pleasant experience for the teachers. P6: “When there was a piece of text from my previous reflection, the questions felt less repetitive and that gave the feeling of analyzing the situation deeper.”

The introductory text to explain the meaning of the questions was partly successful in bridging the gap between the mental framework of the teachers and the researchers. The questions with the introduction were perceived as easier to understand than the questions without the introduction, even though they were focused on a deeper level of reflection. P4: “I thought the questions were more clearly explained as if you casually explained in what direction we needed to think with these questions. Because of this, I was quicker in answering the questions, which was positive.” However, bridging the gap was not successful in all cases, as some questions were too difficult for some teachers: “Some questions were very complicated, even when I read it ten times I still didn’t understand. This gave a very incompetent feeling, which wasn’t nice.” [P2].

Contribution to professional development The digital reflection tool created a more deliberate focus on the professional development for the teachers [P2, P3, P4, P5, P6]. Due to the link of the tool to the professional development course that all teachers were participating in, the focus on their professional development was evident. P6: “The tool made the learning process [of the new material from the course] more continuous and it helped me to integrate it [the new material] in my lessons more deliberately.” The tool could be seen as “a continuous extension of the professional development course” [P4]. The tool, however, is only relevant to use when connected to such a course or in a difficult situation, as “it is too much work to use in your daily working life” [P6].

These reflective exercises contributed to their critical attitude towards their own way of working as a teacher, by creating awareness of their professional identity and their actions as a teacher [P2, P3, P5, P6]. P2: “Because of these exercises I started asking the children more often how they interpreted things, to see if my personal assumptions were correct. Before, I always thought that those were logical, and now I have an attitude that questions those assumptions “is that really true?”” The teachers felt that the tool provided guidance in developing that critical attitude, and that that attitude might fade when they stopped using the tool [P2, P3, P5, P6].

Practical requirements of the digital tool The digital tool was perceived as pleasant to work with, easy to use and professional looking [P2, P3, P4, P5, P6]. P2: “I was not acquainted with the google environment, but it was easy to use and it looked very professional.” However, the activity of writing essay questions within an external digital environment was an obstacle for most of the teachers [P2, P3, P5]. P5: “I forgot to make the exercises because it was a separate environment I had to look for it myself as it wasn’t in my physical sight.” This created a (sometimes too big) threshold to do the exercises, which could be reduced by receiving reminders on either your phone or email of the participants [P2, P3, P5].

Also, the typing out of the answers to the questions did not seem necessary for most of the teachers [P2, P3, P4, P5, P6], the contemplation of their actions would be enough. P5: “I would like the tool to send me one or two questions a day that I can think about on a time that was convenient for me.” Nonetheless, the journaling aspect of the reflection was still valuable for some of the teachers, as “I find the deliberate writing down of your reflection valuable on one side, because it forces me to be more concrete in my answers. Also, it is helpful as a preparation for e.g. certain conversations with the children.” [P4]

Time-investment The biggest paradox that emerged from this research was that the appreciation and the way the teachers value the reflective exercises were in contrast with the lack of willingness to invest time in these reflective practices [P2, P4, P5, P6]. As teachers have a high work pressure, the reflective practices are the smallest priority compared to the lesson preparations etc. that they need to finish for the next day. P5: “Every time when I see that the exercises need to be done, I think of all the work I still need to do for the children. Then I drop the reflective exercises, the other things simply have more priority.” It appears that the affective state of the teachers due to time-pressure and a busy daily life [P2, P4, P5] overrules the rational values of the benefits of reflection for their teaching practice. P5: “When I think about all the work I still have to do while I’m doing the exercises I feel very stressed and frustrated.” Less time-intensive reflective practice is preferred, e.g. consisting of reflective triggers for contemplation instead of journaling [P2, P3, P4, P5, P6]. P4: “I would prefer a tool that gives reflective questions that you can think about, but that does not require writing. That would also be more time-efficient.”

Self-efficacy on reflective skills The teachers’ perceived self-efficacy on their abilities in self-reflection increased after participating in this study [P2, P3, P4, P5, P6]. In the interviews, the teachers said that “the tool gave me guidance in learning to think more critically about my own way of working and reflecting” [P3] and that “I felt I was able to reflect more critically, because of the practice of answering more critical questions each week” [P2, P5].

Fig. 6. Graph of self-efficacy on the reflective abilities of the

participants.

When we look at the visualization on the scores of the participants of the perceived self-efficacy (see Fig. 6), we can see that the perceived self-efficacy increased for all participants. The columns represent the averages of the scores on the 16 statements on perceived self-efficacy on reflective skills of all participants and the averages of the scores on the 5 statements on confidence of the participants in the digital reflection tool. Comparing the standard deviations (see Fig. 6) of the individual participants, we can see that there is one participant [P3] that is quite extreme in the differences of self-efficacy before and after the test-period. The other participants [P2, P4, P5, P6] are more stable in their answers; and an increase can be seen for all participants. This means that the perceived confidence the participants have in their own reflective abilities increased after the test-period. This complements the qualitative data, in which most of the participants [P2, P3, P5, P6] state that they feel that through the reflective questions they made during the test-period, they started thinking more critically and reflect to a deeper level.

Fig. 7. Graph of self-efficacy averages on the effective support of the digital reflection tool.

When we look at the visualization of the scores of the participants before and after on the perceived self-efficacy, we can see that for four of the five participants [P2, P3, P5, P6] there is an increase in confidence in the support of the digital reflection tool. For one of the participants [P4] there is a decrease in the confidence of the support of the digital reflection tool. This can be explained by the qualitative data

of this participants, namely that P4 stated that “I don’t feel that because of the tool I can reflect more in-depth, I could already reflect in depth before this test-period.” For the other participants, there is a growth in confidence of the support of the digital reflection tool, which is complementary to the qualitative data. The participants [P2, P3, P5, P6] all state that they feel that the tool has helped them in reflecting in more depth.

When we look at the visualization of the scores of the participants on their self-efficacy on their abilities of the different levels of reflection (see Fig. 8), we can see that for most of the reflection levels an increase occurs after the test-period. It is also interesting to see that for some participants, e.g. P4 and P5, the perceived self-efficacy on their abilities in descriptive writing is decreased. Another unexpected observation is that all participants show a fairly large increase in perceived self-efficacy on their abilities to reflect critically. As critical reflection did not appear in the digital reflection tool and it is reported to happen rarely [14, 19], it is not likely that their actual abilities in reflecting abilities increased. This means that their perceived self-efficacy on this level of reflection might be incorrect.

Fig. 8. The scores of perceived self-efficacy of the participants,

before and after the test-period, divided into the different levels of reflection. (R0=descriptive writing, R1=reflective

descriptive writing, R2=dialogic reflection, R3= transformative reflection, R4=critical reflection)

DESIGN GUIDELINES FOR FUTURE WORK These aforementioned results enable reflection on the predefined design assumptions and give the opportunity to define certain design guidelines for future (HCI or educational) designers or researchers in this field. Design assumption (i) that focuses on the effect of in-depth questions on the level of depth of the reflections. The questions did indeed have a positive effect on both the level of depth of the reflections of the teachers and on the experience of the participants. This seems to indicate that the experience of learning to reflect more in-depth creates a highly positive experience of “raised awareness” [P2, P3, P4, P5, P6], that the teachers find valuable for their practice. This more critical attitude towards their own practice was highly appreciated, even though it needed to be externally supported in order to occur. These insights can be abstracted to the following design guideline:

a. Create a tool that is both supportive of what the teachers are already comfortable with doing in terms of reflective practice and that stimulates them to take their knowledge/attitude a step further. This creates a positive experience that stimulates learning and professional growth. The matrix on levels of reflection can be highly beneficial when creating exercises to stimulate deeper forms of reflection.

Design assumption (ii) focuses on the incorporation of the content of written reflections of the teachers into the following exercises, to create a connection between the reflections. This personalization of questions had several positive effects on the perception of the questions and the exercise as a whole, according to the experiences of the teachers [P2, P3, P4, P5, P6]. The only concern is that enough freedom should be given in the context of the questions to support full-time teachers in wanting to analyze other events than the suggested ones. These insights lead to the following design guideline:

b. Personalization of the content of reflective exercises has a lot of positive benefits for the perception on the reflective exercises. When creating a positive experience for teachers, the integration of these personalizations will be a valuable contribution.

The matrix on growing levels of reflection (see appendix 2) is based on key words within each level of depth. When a system is designed for this context, an algorithm can be made that focuses on these key words within the written reflections by the teachers, and adjust the following exercises accordingly. These insights can thus form the basis of an algorithmic system.

Design assumption (iii) was focused on the practical requirements of the tool, such as medium and timing. The medium being digital fitted the teachers well, but the medium of an external system such as a website and Google Docs were not well integrated into the daily practice of the teachers. This leads to the opportunity of creating these exercises or solely reflection questions into systems/products they do use in their daily practice, e.g. email, smartboard or even their smartphone. Also, the timing wasn’t well integrated into their daily practice as the work pressure was higher than the willingness to invest this time continuously in reflective practice. Either doing research into more time-effective exercises or a lower work pressure would be beneficial to the reflective practice of teachers. This led to the following insight:

c. The integration into systems/products that are already used in the daily practice of teachers is important, the addition of new external systems is not preferred by teachers. Also, further research needs to be done in more time-effective reflective

exercises or the work pressure needs to be lowered (a change in governmental regulations is required).

DISCUSSION

Self-efficacy questionnaire One of the points of discussion is the scores of the self-efficacy questionnaire by the participants. Some of the scores were a cause for doubt if the participants understood the question. In some cases, the low levels of reflection were given a relatively low score and the highest level of reflection, namely critical reflection, were given an extremely high score (see Fig. 8.). This seems to indicate that the phrasing of the levels of reflection were hard to understand for the participants. However, even if these scores wrongly influenced the averages, these influences are very limited due to the large number of scores that all contribute to the average score. The difference between self-efficacy before and after the study are still clearly visible and a valuable insight to this study.

Time-investment paradox A large factor that influenced the interaction with the tool by the participants was the willingness to invest time in reflective practice twice a week. This willingness was heavily influence by the increasing work pressure that the teachers are feeling in their daily working life, which is an external factor that can’t be changed by the field of professional development or reflective practice. Even with all the benefits of reflective practice, as stated by all of the participants, the external work pressure must be lowered in order to give the teachers the ability to invest time in reflective practice.

Also, one aspect that has a large influence on the willingness of teachers to invest time in a product or course that contributes to their professional development is that they are doing it collectively, with colleagues. That the participants were all following the course on professional development simultaneously with using the digital reflection tool with their colleagues, may have had a certain influence on their willingness to interact with the digital reflection tool the way they did in this research. If this is the case, the connection between interacting with a digital reflection tool individually and collectively working on their professional development needs to be investigated further in future research.

Limitations The participants of the study were all female and worked in the same work environment. This sample of participants limits the generality of this study towards male teachers or teachers who work in a completely different work environment, where motivation and stimulation of professional development might be different. Also, the relatively small group of participants can be a cause for the limitations of generality. However, as the strength of this study lies in the qualitative research methods, a small sample of participants was preferred to a substantially

larger number of participants and yielded valuable in-depth results.

Some of the participants mentioned that they were aware that they were writing reflections that would be read by the researcher. E.g. P2 said that “it took more time to write it down clearly, so you would understand it.” This might have influenced the results of the study, as they were adjusting their behaviour to make it more understandable for the researcher.

CONCLUSION This study gave insight into the implementation of reflective practice on the professional development of primary school teacher through a digital reflection tool. Through qualitative research methods several design assumptions were tested, which after analysis led to certain design guidelines. These guidelines can be used as a basis for further research in this field or as a start for creating new reflection tools for primary school teachers. These guidelines include: the importance of both supporting and pushing teachers to learn how to reflect more in-depth; the benefits of personalization within the content of reflective exercises and the importance of integration of reflective exercises within systems and products that are already used by teachers in their daily practice.

ACKNOWLEDGMENTS Firstly, I would like to thank: M. M. Bekker, for her continuous coaching and expertise during this study; D. Mertens for giving me the opportunity of collaboration and helping me with finding participants; the participants of this study for participating; experts from the Eindhoven School of Education, for their openness to help me with their knowledge and giving inspiration for the educational field; W. Foppen and B. d’Anjou, for proof reading my paper and for always giving me confidence.

REFERENCES 1. Anita L. Allen. 2008. Dredging up the past:

Lifelogging, memory, and surveillance. The University of Chicago Law Review 75, no. 1 (2008), 47-74.

2. Chris Argyris and Donald A. Schon. 1974. Theory in practice: Increasing professional effectiveness. Jossey-Bass.

3. Albert Bandura. 2006. Guide for constructing self-efficacy scales. Self-efficacy beliefs of adolescents 5, no. 307-337 (2006).

4. Theo Bergen. 2005. Tien jaar Nijmeegs onderzoek naar het leren van leraren op de werkplek. VELON-Tijdschrift voor Lerarenopleiders 26.2 (2005), 4-12.

5. David Boud, Rosemary Keogh, and David Walker. 2013. eds. Reflection: Turning experience into learning. Routledge, 2013.

6. Kaya Bouma and Rik Kuiper. 2017. Leraren basisonderwijs denken aan wilde vervolgstaking in november: scholen minimaal twee dagen dicht. (October 2017). Retrieved January 4, 2018 from https://www.volkskrant.nl/binnenland/leraren-basisonderwijs-denken-aan-wilde-vervolgstaking-in-november-scholen-minimaal-twee-dagen-dicht~a4519543/

7. James Calderhead. 1989. Reflective teaching and teacher education. Teaching and Teacher Education, 5 (1989), 43-51.

8. M. Carter and R. Francis. 2000. Mentoring and beginning teachers’ workplace learning. Paper presented at the AARE Conference, Sydney, Australia.

9. Maggie Clarke. 2004. Reflection: Journals and reflective questions: A strategy for professional learning. Australian Journal of Teacher Education 29, no. 2 (2004), 2.

10. Linda Darling-Hammond. 2006. Constructing 21st-century teacher education. Journal of teacher education 57, no. 3 (2006), 300-314.

11. John Dewey. 1997. How we think. Courier Corporation.

12. Thomas Farrell. 2003. Reflective practice in action: 80 reflection breaks for busy teachers. Corwin Press.

13. Angela Fessl, Oliver Blunk, Michael Prilla, and Viktoria Pammer. 2016. The known universe of reflection guidance: a literature review. International Journal of Technology Enhanced Learning 9, no. 2-3 (2017), 103-125.

14. Rowanne Fleck and Geraldine Fitzpatrick. 2010. Reflecting on reflection: framing a design landscape. In Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction, (2010), pp. 216-223. ACM.

15. Rowanne Fleck and Geraldine Fitzpatrick. 2009. Teachers’ and tutors’ social reflection around SenseCam images. International Journal of Human-Computer Studies 67, no. 12 (2009), 1024-1036.

16. Helen Freidus. 1998. Narrative Practices: Portfolios in Teacher Education. (1998).

17. FVOV. 2017. CODE ROOD IN HET PRIMAIR ONDERWIJS: Staking donderdag 5 oktober! (September 2017). Retrieved January 4, 2018 from https://www.fvov.nl/?p=3310

18. Kent L. Gustafson and Winston Bennett Jr. 2002. Promoting learner reflection: Issues and difficulties emerging from a three-year study. Georgia university Athens dept of instructional technology, (2002).

19. Joelle K. Jay and Kerri L. Johnson. 2002. Capturing complexity: A typology of reflective practice for teacher education. Teaching and teacher education 18, no. 1 (2002), 73-85.

20. Geert Kelchtermans. 2005. Teachers’ emotions in educational reforms: Self-understanding, vulnerable commitment and micropolitical literacy. Teaching and teacher education 21.8 (2005), 995-1006.

21. KNOW. 2013. Professionele ontwikkeling van de leraar. (November 2013). Retrieved January 4, 2018 from https://www.leraar24.nl/professionele-ontwikkeling-van-de-leraar/

22. KNOW. 2013. Reflectie door de leraar. (December 2013). Retrieved January 4, 2018 from https://www.leraar24.nl/reflectie-door-de-leraar/

23. Fred Korthagen. 1982. Leren reflecteren als basis voor de lerarenopleiding, een model voor de opleiding van leraren, in het bijzonder wiskundeleraren. 's-Gravenhage, Stichting voor Onderzoek van het Onderwijs.

24. Fred Korthagen. 1998. Leren reflecteren: naar systematiek in het leren van je werk als docent. Begeleiden van docenten, reflectie als basis voor de professionele ontwikkeling in het onderwijs (1998), 43-56. Baarn: Nelissen.

25. Fred Korthagen and Angelo Vasalis. 2002. Niveaus in reflectie: naar maatwerk in begeleiding. VELON Tijdschrift voor lerarenopleiders 23.1 (2002), 29-38.

26. Johan Luttenberg. 2000. Wat te doen? Een onderzoek naar de normatieve professionaliteit van docenten. Ph.D. Dissertation. KU Nijmegen.

27. Jacobiene A. Meirink, Paulien C. Meijer, Nico Verloop, and Theo CM Bergen. 2009. Understanding teacher learning in secondary education: The relations of teacher activities to changed beliefs about teaching and learning. Teaching and teacher education 25, no. 1 (2009), 89-100.

28. Jennifer A. Moon. 2013. Reflection in learning and professional development: Theory and practice. Routledge.

29. Sonia Nieto, Stephen Gordon, and Junia Yearwood. 2002. Teachers' experiences in a critical inquiry group: A conversation in three voices. Teaching Education 13.3 (2002), 341-355.

30. Onderwijsraad. Professionele schoolcultuur. Retrieved from https://www.onderwijsraad.nl/dossiers/primair-onderwijs/item125

31. Sara Price, Yvonne Rogers, Danae Stanton, and Hilary Smith. 2003. A new conceptual framework for CSCL. In Designing for change in networked learning environments, pp. 513-522. Springer Netherlands.

32. Michael Prilla, Alexander Nolte, Oliver Blunk, Dennis Liedtke, and Bettina Renner. 2015. Analyzing collaborative reflection support: a content analysis approach. In ECSCW 2015: Proceedings of the 14th European Conference on Computer Supported Cooperative Work, 19-23 (September 2015), Oslo, Norway, pp. 123-142. Springer, Cham.

33. Dick Ratering and J. M. Q. Hafkamp. 2000. Zelfsturend leren: begeleiden van ervaringsgericht leren in organisaties. Academic Service, economie en bedrijfskunde.

34. Piety Runhaar, Karin Sanders and Huadong Yang. 2010. Stimulating teachers' reflection and feedback asking: An interplay of self-efficacy, learning goal orientation, and transformational leadership. Teaching and teacher education 26.5 (2010), 1154-1161.

35. Corina Sas and Alan Dix. 2009. Designing for reflection on experience. In CHI'09 Extended Abstracts on Human Factors in Computing Systems, pp. 4741-4744. ACM, 2009.

36. Donald A. Schön. 1983. The reflective practitioner. New York, Basic Books.

37. Jane K. Seale and Alan J. Cann. 2000. Reflection on-line or off-line: The role of learning technologies in encouraging students to reflect. Computers & Education 34, no. 3 (2000), 309-320.

38. John Smyth. 1992. Teachers’ work and the politics of reflection. American educational research journal 29, no. 2 (1992), 267-300.

39. Klaas van Veen, Rosanne Zwart, Jacobiene Meirink and Nico Verloop. 2010. Professionele ontwikkeling van leraren. Een reviewstudie naar effectieve kenmerken van

professionaliseringsinterventies van leraren (2010).

40. M. Watt and M. Lawson. 2009. Using a meta-analysis activity to make critical reflection explicit in teacher education. Teaching and Teacher Education 25 (2009), 609e616.