first pilot - makeitrealmakeitreal.info/.../intevaluationreport-2016-1-pl01... · q4.5 you are...

MAKEITREAL Project Erasmus+ KA2 2016-1-PL01-KA201-026492

1

MAKEITREAL PROJECT

Addressing underachievement in STEAM education through real product design and making practices

Erasmus+ KA2 2016-1-PL01-KA201-026492

First Pilot

MakeITReal learning intervention in ODTUGVO High School


2

Table of Contents

Abstract…………………………………………………………………………………………….3

Chapter 1: Introduction…………………………………………………………………………….4

Chapter 2: Pre and Post Test ………………………………………………………………………5

2.1 General Interest and STEM Interest…………………………………………………..5

2.2 Relevance and School Experience…………………………………………………….7

2.3 STEM Profiles………………………………………………………………………...7

2.3.1 Math…………………………………………………………………………………7

2.3.2 Science………………………………………………………………………………7

2.3.3 Technology and Engineering………………………………………………………..8

2.3.4 21st Century Skills…………………………………………………………………...8

2.3.5 Arts…………………………………………………………………………………..8

2.4 Perception and Future………………………………………………………………….9

2.5 Conclusionary Remarks………………………………………………………………..9

Chapter 3: Feedback generated by the students……………………………………………………10

Chapter 4: Feedback generated by the participant teachers………………………………………..10

Chapter 5: Conclusion……………………………………………………………………………...10


3

Abstract

This document provides detailed information on MakeITReal learning intervention carried out at

ODTUGVO High School. It provides information about the selection of the students, the

composition of the teams and activities that are implemented in the class. Analysis of data that

were collected from the tools used to monitor the learning activity is also included in this

document.


4

Chapter 1: Introduction

In order to engage high school students who demonstrate low performance in STEM education, the

teachers involved in MakeITReal intervention aimed to provide meaningful learning environments

to make learners monitor and evaluate their own thinking and learning process.

The participant teachers in the first pilot decided to select the students, considering facts about

Turkish education system. High schools in Turkey starts at 9th grade like most of the European

countries and our school starts offering advanced science and math lessons starting from the 10th

grade. That means students should choose one of four tracks (Turkish language–

mathematics, science, social sciences, and foreign languages) at the beginning of 10th grade. Our

students mostly tend to choose science track as STEM disciplines that are promising high pay

professions in the future are still the most demanding ones. The passing mark in high schools is 50

over 100 points scale in exams, which is not a very high point. Besides exam grades, students are

evaluated according to their performances in class which is mainly based on in class activities,

homework, and quizzes. The overall assessment and evaluation system allows our students to be

aware of the strengths and weaknesses of their performances. However, the curriculum in our high

school is above standards compared to public and other private schools in Ankara but it helps to

equip students through a really challenging preparation program for the university entrance exam at

the end of high school.

These circumstances bring about the effect that most of the students that are at risk of school failure

leave our school by the end of 9th grade. So, by considering these facts, teachers involved in first

pilot decided that the most appropriate group of students will be 9th graders as their performance

in STEM related subjects is a good indicator of their future achievement. 11th and 12th graders

would not be feasible groups as they are mainly concentrated on their "university entrance exam

preparation courses", likewise the 10th graders will do during the second year of MAKEITREAL

(during the second pilot). The other reason that we have decided on this group is that we could

work with these 9th graders during an hour called "Student Club Hour" on every Wednesday for 60

min., which is not unlikely would happen with higher graders.

Teachers started the selection process by identifying 30 students among 9th graders who were the

ones with the lowest average performance in school. They prepared a presentation with pictures

and videos that described 3D modelling/printing while explaining the meaning of STEAM and

maker movement. They brought some art crafts that were printed beforehand to the classroom and

encourage students through prompt questions to realize how 3D printing can lead us to a new way

of thinking.

The first part of thematic tables related to “Perceptions and attitudes towards STEM” in eligibility

cards were filled out by the students themselves and the other three tables were filled by the

teachers, consulting both to the 9th grade psychological counsellor and the assistant principal. Based

on the information on these cards, they selected the 16 that are more eligible by giving priority to

those that demonstrate low performance in STEM related subjects and feel uncomfortable with

STEM and think that school curriculum does not allow them to be creative.

When they first formed the group, the number of girls was 7 and the number of boys was 10, but

after the second session with the students, a girl and a boy decided to quit therefore we had to

complete the pilot with 15 students (6girls/9boys). One of the boys in the group was repeating 9th

grade.

Before implementing the projects of first pilot, the group came together asked to fill in pre-test

questions online via Google docs. Then to establish a good relationship between members, they

were asked whether they want to work individually on their laptops or with another team member.

Most of the students were coupled up without the need of any icebreaking activity. Teachers

created a set of rules that helped better implementation of learning intervention, which included


5

classroom behaviour, safety rules and submission of designs to the teachers via email. Students

were also informed about keeping a record of entire process.

For the following sessions of intervention between March 2017-May2017, students and teachers

got together on every Wednesday during the “Student Club Hour” for 60 minutes. Before

implementing the first project, one of the teachers made a demonstration of FreeCAD software and

familiarized students with the basic features. Students were provided with worksheets of each task

in Turkish and with the necessary explanations of the teachers. Before introducing each task in the

class, teachers tried the task first to understand the challenges that the students may encounter and

they printed the object of each task before the session so the students were able to touch the real

object before they design it. While helping with the tasks, teachers asked driving questions to help

students build connections to the science and math curriculum and their everyday experiences.

When the designing and printing process of pre-determined tasks (cube, keychain and penguin) has

finished, some students were encouraged to design the BATMAN logo and some tried to design

gear models.

At the end of each project teams were encouraged to evaluate their work and to provide feedback to

the other teams as well. Besides, students presented their work to other students and teachers in an

exhibition at the common hall and gave each teacher a keychain with the school logo as a present

for the end of school year.


6

Chapter 2: Pre and Post Tests

Pre- and post-tests were translated into Turkish and shared with the students online via Google

docs. Since students might be uncomfortable using their names, each student has given a code.

Students are made to fill in pre-test after some clarifications provided by the teachers. After the last

session with the students, post-test is applied. Pre and post- test scores of students are compared to

measure knowledge gained from the learning intervention.

Comparing participants’ post-test scores to their pre-test scores enabled us to see that the first pilot

was successful in increasing participant interest in STEAM. When we looked at the students’

responses, we have observed that their perceptions regarding STEAM subjects and their

performance in STEAM related disciplines are increased.

2.1. General Interest and Stem Interest

When pre-and post-test results were compared, it was seen that student’s interest in science, arts,

engineering and technology has increased. The increase in the interest in technology and

engineering subject area is remarkable, whereas interest in math has remained same.

General Interests

Q1. Select the subjects that you are the most interested in (you can select up to 3 subjects):

PRE and POST Test answers average points

Q1.1 Literature

Q1.2 History

Q1.3 Arts

Q1.4 Algebra

Q1.5 Geometry

Q1.6 Physics

Q1.7 Chemistry

Q1.8 ICT

Q1.9 Foreign languages

Q1.10 Sports

Q1.11 Geography

Q1.12 Music

Q1.13 Biology

Q1.14 Other

pre-test 5 5 8 6 0 2 1 4 4 3 2 2 3

post-test 7 5 8 6 1 2 0 4 2 4 2 1 3

General Interests

Q2. Select the subjects that you are the least interested in (you can select up to 3 subjects):

PRE and POST Test answers avarage points

Q2.1 Literature

Q2.2 History

Q2.3 Arts

Q2.4 Algebra

Q2.5 Geometry

Q2.6 Physics

Q2.7 Chemistry

Q2.8 ICT

Q2.9 Foreign languages

Q2.10 Sports

Q2.11 Geography

Q2.12 Music

Q2.13 Biology

Q2.14 Other

pre-test 2 5 0 4 6 3 7 1 6 2 5 2 2

post-test 4 6 0 4 2 1 8 1 7 2 7 2 1

STEAM interest

Q3. To what extent are you interested in the following subject areas?


Q3.1 Science

Q3.2 Maths Q3.3 Arts Q3.4 Engineering and Technology

pre-test 3,07 2,87 3,80 3,20

post-test 3,33 2,87 4,00 3,73


7

2.2. Relevance and School Experience

Student’s awareness about the real-life examples given in classroom, the number of creative tasks

they get involved in and hands-on activities using math and science has increased. Going forward

through new subjects in classes during first pilot might have been one of the key factors that caused

this increase in their perception as they had so many student-centered experiences in different

lessons as the weeks pass by.

Relevance & School Experiences

Q4. How often do you do these things in your class/school?


Q4.1 You get examples of how the things you learn in Math & Science matter in the real world

Q4.2 When you study a mathematical topic, your teachers include also other subject areas or disciplines (i.e. Arts, History, etc.)

Q4.3 Get involved in creative tasks that are meaningful to you

Q4.4 You are given opportunities to express or replenish your creativity

Q4.5 You are allowed to use artistic methods to show your understanding of a concept/event/object

Q4.6 Get involved in hands-on and practical activities using also math and/or science

Q4.7 Talk about the types of jobs and careers that use Science and Math

pre-test 3,67 3,20 3,40 3,07 3,13 3,93 3,60

post-test 4,13 3,87 4,07 3,93 3,93 4,53 3,87

2.3. STEM Profiles

2.3.1.Maths: There is a dramatic increase in the responses to Q5.2 “I think learning math will

help me in my daily life”, where there is decrease in the perception of math is boring. Overall, the

average points in most of the questions have increased.

STEAM PROFILES

Q5. MATH


Q5.1 I usually do well in Math

Q5.2 I think learning math will help me in my daily life

Q5.3 I enjoy doing Math

Q5.4 I am sure of myself when I do Math

Q5.5 Math is hard for me

Q5.6 Math is boring

pre-test 2,93 2,73 3,33 2,73 3,07 2,60

post-test 3,27 4,13 3,53 3,20 2,87 2,00

2.3.2.Science: When the answers given to the “science” section is compared between pre and

post-test, it is clearly seen that students has become more interested in science lessons. Students

started to find science lessons less boring and less challenging.

STEAM PROFILES

Q6. SCIENCE


Q6.1 I usually do well in Science

Q6.2 I think learning Science will help me in my daily life

Q6.3 I enjoy doing science

Q6.4 I am sure of myself when I do science

Q6.5 Science is hard for me

Q6.6 Science is boring

pre-test 2,87 3,87 3,60 3,27 3,27 2,53

post-test 3,13 4,33 4,07 3,20 3,07 1,47


8

2.3.3.Technology and Engineering: At the end of the MakeITReal learning

intervention, students seemed to be more aware of knowledge from math and science can be used

to design everyday objects and their interest in creating new objects has increased.

STEAM PROFILES

Q7. TECHNOLOGY/ ENGINEERING


Q7.1 I like to imagine creating new products.

Q7.2 I can use knowledge from Math and Science to design everyday objects

Q7.3 I usually do well in ICT class

Q7.4 I enjoy exploring what makes machines work

Q7.5 I am good at building and fixing things

pre-test 4,13 3,73 4,00 3,87 3,60

post-test 4,60 4,33 4,20 3,93 4,00

2.3.4. 21st Century Skills: The answers in pre-test related to this section had already high

scores and it is observed that this positive attitude of students has increased by the end of first pilot.

STEAM PROFILES

Q8. 21ST CENTURY SKILLS


Q8.1 I am confident that I can set goals and work towards achieving them

Q8.2 I am confident that I can work well in a team

Q8.3 I am confident I can include others’ perspectives when making decisions

Q8.4 I am confident that I can find solutions when things do not go as planned.

Q8.5 I am confident that I can present my work in public and communicate my ideas

Q8.6 I am confident that I can take initiative to plan and execute tasks that I have never performed before.

pre-test 3,47 4,00 4,40 4,20 3,60 3,93

post-test 4,00 4,40 4,67 4,27 4,47 4,40

2.3.4. Arts: Before the learning intervention has started, some students were thinking that art

has nothing to do with science, technology and engineering. However, after they have completed

the projects, more of them were convinced that arts, science and technology are definitely related.

All of them seem to be motivated when they are given opportunities to participate in arts activities

especially in STEM field subject areas.

STEAM PROFILES

Q9. ARTS


Q9.1 Arts has nothing to do with Science, Technology, Engineering and Maths

Q9.2 I feel motivated when through STEM subjects I am given opportunities to participate in Arts activities

Q9.3 I am interested in exploring science concepts when combined with Arts

pre-test 2,47 4,20 4,07

post-test 1,73 4,60 4,53


9

2.4. Perceptions and Future: When the pilot ended, it is obvious that the importance of

becoming knowledgeable in science has increased for the students. They were more aware of that

math and science can help them understand how the world works. More of them started thinking

positively about considering a job that involves using science and math.

Perceptions & Future

Q.10 To what extent do you agree with the following?


Q10.1 I'm planning to work harder in my math and science courses

Q10.2 Knowing how to use math and science together can allow me to make useful and creative things.

Q10.3 It is important for me to become knowledgeable in science and science concepts

Q10.4 Knowledge of math and science can help me understand how the world works

Q10.5 If I learn engineering, then I can improve things that people use every day.

Q10.6 I expect to use science and math when I get out of school.

Q10.7 I would consider a job that involves using science and/or maths

Q10.8 My education will create many future opportunities for me

pre-test 3,80 3,53 3,67 3,67 4,13 3,40 2,93 4,40

post-test 4,27 4,33 4,20 4,47 4,53 4,07 3,73 4,60

2.5. Conclusionary Remarks:

When pre and post - test results were analyzed, the following results were found to be remarkable;

Student’s interest in arts and engineering fields has increased.

Student’s perception on math and science will help them in daily life has changed

positively.

Students feel more confident about doing maths and science.

Students feel more confident about working in team/presenting their work in public.

Number of students that expect to use science and math when they get out of school has

increased.

All of them feel motivated when they are given opportunities to participate in arts activities

through STEM subjects.

The ones that preferred to work individually in the first place were actually started to work

with different students, which was a good indicator of a motivating education environment.

During the first pilot, most of the students acted as multipliers and helped other students, which led

them to gain self- esteem. They were eager to complete their projects within the given time. Some

of them that weren’t able to complete their design during the session, completed their work after

they got home and e-mailed their work to teachers. When the project is over, the students presented

their work in the classroom in teams within 5 minute presentations. Most of them talked about the

challenges they have faced during using the software and how they overcome these challenges

Most of them were very confident in front of other people. Students also presented their work to

other students and teachers in an exhibition at the common hall of our school. It was surprising that


10

most of them seemed to be very knowledgeable when talking about 3D modelling and printing and

how their designs enhanced their spatial thinking skills.

All these positive changes in attitudes and perception of students indicate that the learning

intervention was successful in increasing participant interest in STEAM and enhancing their

performance in STEAM related disciplines. Overall, this project provided several opportunities for

the students for collaboration, problem solving and critical thinking.

Chapter 3: Feedback generated by the students When feedback forms generated by students were analyzed, it is observed that;

11 students described the projects as fun, 10 students as interesting and 8 as creative out of

15 students. These three adjectives were the most popular ones they chose to describe the

workshop

They did like to print 3D designs most

They did like to engage with technical problems the least

They suggested to work for longer sessions

Most of the them said that they would like to participate in additional projects

The average point of the answers given to the “Would you like to participate in additional

projects like the ones you were recently involved in?” question is 4.33

The average point of the answers given to the “Would you propose this workshop to a

friend/ classmate?” is 4.60

New ideas for the following projects generated by students were: Iphone case/chess

pieces/human figures/shoes/objects with moving parts/fidget spinner/pen holder/headphone

stand/jewelry/toys/puzzle pieces/stamps/animal figures

Chapter 4: Feedback generated by the teachers Throughout the project, participant teachers worked in the same classroom with 15 students but

each teacher was responsible of 5 students. Each project is implemented by one of the teachers and

other two were in the classroom to help students.

When feedback forms generated by students were analyzed, it is observed that;

The educational resources provided were satisfying

The support from the project implementation team was useful

They are all satisfied with the way this 3-months workshop was carried out

They are confident that they can continue carrying out the MakeITReal project in real

classroom settings

They think that MakeITReal learning intervention offered them opportunities for

professional development and skill-building, MakeITReal practices can set a basis

whereupon underachievement in STEM can be tackled, they can help linking STEM to

creative, artistic hands-on practices and they can help linking STEAM to real life and

research/entrepreneurial practices

Chapter 5: Conclusion Analysis of the results gained by the evaluative data, it was possible to monitor the development of

our students from the beginning to the end of the learning intervention. It was clearly seen that

students have developed spatial thinking skills and their interest in STEM related subjects has

significantly increased. The main concern of the participant teachers was to help students identify


11

connections between their projects and the school curriculum. In order to make the project relevant,

teachers have spent a great effort in establishing links between what students are learning at school

and the nature of the assigned projects.

At the beginning of the pilot, students were thinking that they will do only some fun and easy

activities, but when they have realized that they have the use their background knowledge

especially the ones related to math and geometry, the projects became a little bit challenging. The

participant teachers’ role as facilitators, who provided frequent feedback on students’ performance,

encouraged them to deal with failure and created a positive classroom environment. By the end of

the project teachers were satisfied that meaningful learning experiences have occurred as students

were able to evaluate both their own and their peers learning process, and became more committed

to the tasks. The whole MakeITReal learning intervention has provided several opportunities for

students to develop skills through making and constructing. The project helped our students

actively engage in teamwork that emphasized collaborative learning and increased their problem-

solving abilities. Teaching in a technology rich environment was definitely rewarding as seeing

student’s excitement and ambition is a clear indicator of how this environment is very powerful.


12

Appendices

Appendix A: Database Evaluation sheet for pre and post-test results, feedbacks generated by

participants

first pilot - makeitrealmakeitreal.info/.../intevaluationreport-2016-1-pl01... · q4.5 you are...

Documents