talking talons youth leadership evaluation report · talking talons youth leadership evaluation...

LEIDEN LLC

Talking Talons Youth Leadership Evaluation

Report School Year 2015

Dr. Carmen Sorge

[email protected]

1

Table of Contents

Guide to basic interpretation ........................................................................................................... 7

Summary of results ......................................................................................................................... 7

Data Collection and Research ....................................................................................................... 10

Research Design ........................................................................................................................ 10

Composite instrument ............................................................................................................ 10

The Composite instrument scoring .................................................................................... 12

Reliability of the instrument .............................................................................................. 13

Quizzes .................................................................................................................................. 14

Reliability and Scale development..................................................................................... 15

Talking Talons Quiz Knowledge ....................................................................................... 16

Talking Talons Quiz Attitudes ........................................................................................... 17

Teacher Feedback Form ........................................................................................................ 19

Qualitative Data Collection ................................................................................................... 19

Participants ................................................................................................................................ 19

Explanation of GLM Repeated Measures ................................................................................. 21

Examples ............................................................................................................................ 23

Science Attitudes .......................................................................................................................... 27

Research on Science Attitudes .................................................................................................. 27

Science attitude change during program for participants .......................................................... 29

Science Attitude change from pretest to posttest by group ....................................................... 32

Student Qualitative Feedback on Science Attitudes ................................................................. 37

Classroom Teacher feedback about science Attitudes .............................................................. 40

Hands on science experiments and Animals in the classroom ..................................................... 40

Research on Hands on Science and Animals in classroom ....................................................... 40

Classroom Teacher feedback .................................................................................................... 42

Student Feedback on Animals in classroom and Hands on science .......................................... 42

Buddy class ................................................................................................................................... 46

Research on peer tutoring .......................................................................................................... 46

2

Classroom teacher assessment of Buddy class .......................................................................... 48

Student feedback on Buddy class .............................................................................................. 49

Role Model for younger students .......................................................................................... 53

Science Knowledge ....................................................................................................................... 54

Change in Science Knowledge by Group ................................................................................. 54

Quiz Knowledge ........................................................................................................................ 57

Self-perceived science grade in school ..................................................................................... 65

Quiz Attitude subscales ............................................................................................................. 68

Feedback on Educator ................................................................................................................... 70

Student Quantitative Feedback.................................................................................................. 70

Student Qualitative Feedback.................................................................................................... 74

Classroom teacher feedback for Educator ................................................................................. 75

Classroom teacher feedback for Talking Talons Program ............................................................ 78

Teacher Attitude ........................................................................................................................ 78

Effectiveness by student ability: feedback by classroom teachers .......................................... 79

Classroom teacher subscales ..................................................................................................... 81

Other Results ................................................................................................................................. 83

Significant changes in pre posttest Composite .......................................................................... 83

Moral Attitude ........................................................................................................................... 83

Research on Moral Attitude ................................................................................................... 83

Overall significant Composite results ....................................................................................... 87

Overall Non significant composite results ................................................................................ 90

Summary ..................................................................................................................................... 101

References ................................................................................................................................... 103

3

Tables

Table 1 Pretest and Posttest Cronbach’s alpha for subscales ....................................................... 13

Table 2 Participants by Group ...................................................................................................... 20

Table 3 Participants by Gender .................................................................................................... 20

Table 4 Group by Class................................................................................................................. 21

Table 5 Science Attitude Change over Time ................................................................................ 30

Table 6 Science Attitude Change over Time Multivariate Tests .................................................. 30

Table 7 Science Attitude Change over Time within Subjects Effects .......................................... 30

Table 8 Science Attitude Change over Time Within subjects Contrasts ...................................... 31

Table 9 Science Attitude Change over Time between Subject Effects ........................................ 32

Table 10 Change in Science Attitude by Group ........................................................................... 33

Table 11 Change in Science Attitude by Group Multivariate Tests ............................................. 34

Table 12 Change in Science Attitude by Group Within subjects Effects ..................................... 34

Table 13 Change in Science Attitude by Group within Subjects Contrasts ................................. 35

Table 14 Change in Science Attitude by Group between Subject Effects .................................... 36

Table 15 Change in Science Attitude by Group Estimated Means ............................................... 36

Table 16 Difference Change in Science Attitude by Group ........................................................ 37

Table 17 Attitude toward Science from Quizzes Mean ............................................................... 37

Table 18 Classroom Teacher assessment of Science Attitude change ......................................... 40

Table 19 Classroom teacher assessment of program components ................................................ 42

Table 20 Classroom teacher assessment of buddy class impact .................................................. 49

Table 21 Student answers by group to "I am a good role model for younger students" .............. 54

4

Table 22 I am a good role model significance tests ...................................................................... 54

Table 23 Change in Science Knowledge by Group ...................................................................... 56

Table 24 Change in Science Knowledge by Group Multivariate tests ......................................... 56

Table 25 Change in Science Knowledge by Group Means ......................................................... 57

Table 26 Mean scores on Knowledge Quiz by topic and class ..................................................... 59

Table 27 ANOVA comparison on Quiz Knowledge by Class ..................................................... 60

Table 28 Quiz Knowledge Means by Gender .............................................................................. 62

Table 29 ANOVA comparison for Talking Talons Quizzes by Gender ....................................... 63

Table 30 Group Comparison for I will get a good grade in science class this year ...................... 66

Table 31 Group Comparison for I will get a good grade in science class this year Multivariate

tests ............................................................................................................................................... 67

Table 32 Quiz Attitude subscale descriptive statistics ................................................................. 69

Table 33 Student feedback on Educator Subscales ....................................................................... 71

Table 34 Student feedback on Educator from Quiz results .......................................................... 72

Table 35 Student Feedback on Individual Educator questions ..................................................... 73

Table 36 Classroom Teacher Feedback on Talking Talons Educator .......................................... 76

Table 37 Classroom Teacher Feedback Personal ......................................................................... 79

Table 38 Classroom teacher feedback program effectiveness by student ability ......................... 80

Table 39: Classroom teacher subscales for Talking Talons program ........................................... 82

Table 40 Change in Moral Attitude by Group .............................................................................. 85

Table 41 Change in Moral Attitude by Group Multivariate tests ................................................. 85

Table 42 Change in Moral Attitude by Group Within subjects .................................................... 86

Table 43 Change in Moral Attitude by Group Between subjects ................................................. 86

Table 44 Statistically significant Pre Post change by group ........................................................ 87

Table 45 ANOVA Statistically significant Pre Post change by group ......................................... 88

5

Table 46 Effect Size Statistically significant Pre Post change by group ...................................... 89

Table 47 Non Significant Pre Post change by Group .................................................................. 89

Table 48 Means Non Significant Pre Post change by Group........................................................ 90

Table 49 ANOVA Non Significant Pre Post change by Group .................................................... 91

Table 50 Effect size Non Significant Pre Post change by Group ................................................. 92

Table 51 Pre and Posttest means by group ................................................................................... 92

Table 52 Pre and Posttest multivariate tests by group .................................................................. 95

Table 53 Pre and Posttest Univariate tests by group..................................................................... 96

Table 54 Within Subjects contrasts Pre and Posttest means by group ......................................... 98

Table 55 Pre and Posttest means by group ................................................................................... 99

Table 56 Posttest Attitudes for Treatment .................................................................................. 100

Table 57 Student overall perception of program ........................................................................ 100

6

Figures

Figure 1 Longitudinal change in Science Attitude ....................................................................... 29

Figure 2 Change in Science Attitude from Pretesting to Posttesting by Group............................ 32

Figure 3 I am a good role model for younger students ................................................................. 53

Figure 4 Change in Science Knowledge by Group ....................................................................... 55

Figure 5 Quiz Knowledge section means ..................................................................................... 58

Figure 6 Self perceived change in science grade by group ........................................................... 65

Figure 7 Quiz Attitude subscales .................................................................................................. 69

Figure 8 Student Perception of Educator ...................................................................................... 70

Figure 9 Student Perception of Educator ...................................................................................... 72

Figure 10 Student Feedback on Educator questions ..................................................................... 73

Figure 11 Classroom teacher feedback on Talking Talons Educator ........................................... 75

Figure 12 Classroom Teacher subscales on Educator ................................................................... 77

Figure 13 Classroom teacher feedback on Talking Talons program ............................................ 78

Figure 14 Classroom teacher assessment of Effectiveness of program by student ability ........... 80

Figure 15 Classroom teacher perception ....................................................................................... 81

Figure 16 Change in Moral Attitude by group.............................................................................. 84

Figure 17 Statistically significant Pre Post change by group ....................................................... 87

Figure 18 Non Significant changes for Composite ...................................................................... 90

Figure 19 Pretest and Posttest Multivariate Tests by group ......................................................... 96

7

Guide to basic interpretation

The p value tells the researcher the probability that this data would be observed simply by

chance. A p value of .05 (5 percent chance that the difference was just due to random

fluctuation) is the statistical standard for the maximum while a p value of .01 (less than 1 percent

chance the difference is due to random fluctuation) is considered more conservative and thus a

better indication of actual results. All p values considered statistically significant for the Talking

Talons program are less than .01.

The effect size (η2) examines the magnitude of change or how MUCH difference

occurred. The standard in use for repeated measures is .02 is a small effect size, .13 is a medium

effect size and .26 is a large effect (Bakeman).

Summary of results

A statistically significant change in Attitude toward Science was seen in the treatment

group and not for the control group from pretesting to post testing. F(1,58)=12.25. p<.001

η2=.17 (medium effect size)

8

A statistically significant change in Knowledge was seen in the treatment group and not

for the control group from pretesting to post testing. F(1,55)=11.01. p<.01 η2=.45 (very

large effect size)

A statistically significant change in Moral Attitude was seen in the treatment group and

not for the control group from pretesting to post testing. F(1,58)=8.00. p<.01 η2=.12

(small/medium effect size)

A statistically significant change in self-reported anticipated grade in science was seen in

the treatment group and not for the control group from pretesting to post testing

F(1,57)=19.83. p<.001 η2=.26 (large effect size)

The treatment group exhibited a statistically significant positive change in attitude toward

science as the program progressed. F(3,9)=5.15. p<.01 η2=.15 (medium effect size). This

statistic uses only the treatment group, as the control group does not take the Talking

Talon’s quizzes.

Classroom teacher feedback was extremely positive and both teachers indicated the

program increased student science knowledge and attitudes, that the buddy class was

worthwhile and also indicated a strong willingness to have the program again. Mean

scores for all subscales measuring teacher feedback were all above 6 on a 7 point scale (a

9

higher number indicates more positive feedback) with a mean of 6.44 and a standard

deviation of .37.

Qualitative feedback from students was overwhelmingly positive, mentioning that the

program was a positive experience, that they enjoyed presenting to the buddy class and

found it both rewarding and educational. Many expressed that they relished “being the

teacher” stating that “It’s like my grownup mode just turns on.” The students also were

enthusiastic about that both the hands on and presentation sections of the program and

found them enjoyable and educational. As a student stated during interview “Talking

Talons is like unicorns, it’s amazing!”. The only suggestion for improvements by the

students was the inclusion of even more animals. Means scores for all student treatment

subscales were above 5.35 on a 6 point scale with a mean of 5.54 and a standard

deviation of .34.

10

Data Collection and Research

The Talking Talons program evaluation data collection consists of:

A control/treatment pretest and posttest called the Talking Talons Composite

A series of 10 quizzes measuring knowledge and attitude change for the

participants

Online teacher feedback

Qualitative interviews of a sample of the Talking Talons participants

Research Design

Composite instrument

Talking Talons program is evaluated using a pretest and posttest quasi- experimental

design with control and treatment groups for the composite instrument. As the program is

offered by classroom, true random assignment is not feasible. However, the control groups are

selected from the same school using teachers who are not currently receiving the program.

Although ideally a Solomon design would be used, the effort and expense involved in collecting

four sets of data points is prohibitive.

The use of a control group is imperative. Without control groups any changes cannot be

attributed to the program. Factors such as maturation, the effect of testing and other outside

factors cannot be eliminated as agents of change without the use of a control group.

11

Improvements seen in the treatment group therefore cannot be ascribed to the program unless a

control group is utilized.

For the composite instrument a General Linear Model Repeated measures utilizing SPSS

18 was run. The GLM repeated measures improves on the multivariate regression model by

allowing for linear transformations or linear combinations of multiple variables. This expansion

means that the GLM has important advantages over the multiple and the purported multivariate

regression models which are inherently univariate methods. The first advantage is that

multivariate test of significance may be use if the responses on multiple dependent variables are

correlated. This is helpful as separate univariate tests of significance for correlated dependent

variables (such as used in multiple t tests) are not independent and may not be appropriate.

Multivariate tests of significance of independent linear combinations of multiple dependent

variables may also yield information about which response variables are, and are not, actually

related to the predictor variables. A second advantage is the ability to analyze effects of repeated

measure factors. Linear combinations of responses reflecting a repeated measure effect can be

constructed and tested for significance using either the univariate or multivariate approach to

analyzing repeated measures in the general linear model. In this research, with pretest and

posttest measurements of control and treatment groups the GLM repeated measures is the most

appropriate test.

Effect sizes based on recent research on repeated measure designs based on Cohen (1988,

pp. 413–414), who did not consider repeated measures designs explicitly, defined an η2 of .02 as

small, one of .13 as medium, and one of .26 as large. “It seems appropriate

to apply the same guidelines to η2 as well.” (Bakeman)

12

The Composite instrument scoring

All composite questions are scored on a Likert scale from 1 to 6 ranging from Very False

to Very True. All questions and scales are renormed so that a higher value indicates a more

desirable outcome. Therefore a value of 5 indicates better self-control or less violent tendencies

than a score of 3. For all subscales a higher score is more desirable.

The Locus of Control measure reports a student’s self-reported ability to direct and

manage their own behavior. Ten questions addressing these issues such as “I do things I know

are wrong because my friends are doing them.” and “I am easily distracted.” are included.

The School Attitude scale consists of five self-reported feelings about school such as

“The work I do in school is important to me” and “I would skip school a lot if I knew that I

would not get caught”.

The Self-esteem attitude scale consists of twelve self-assessments of the student’s self-

value and perception of ability such as “I am proud of myself.” And “If I want to learn to do

something new I usually can learn it.”

The Attitude toward Violence scale consists of five self-reported actions or intentions of

violence such as “I fight a lot.” and “If people will not do what you want it is okay to threaten

that you will hurt them.” Questions 9, 11, 25, 42 and 20 from the composite instrument are

included in this subscale.

13

The Moral Judgment scale consists of eleven questions evaluating student attitudes

toward various ethical and unethical behaviors such as “It is okay to cheat on schoolwork if you

do not get caught.” And “It is wrong to lie in order to get what you want.”

Seventeen questions for the treatment group only are asked of the treatment

group. These questions address the participant’s evaluation of the program and the educator and

include questions such as “I looked forward to the days that Talking Talons came to my

classroom” and “My Talking Talons educator was easy to understand”. A full report of these

questions is included in the educator report.

Reliability of the instrument

Cronbach’s alpha was calculated for each subscale using the overall sample. Reliability

greater than .7 is considered good (Nunnaly, 1978). All subscales of the instrument have good to

excellent reliability. The 2015 evaluation has a much smaller sample size than previous years,

which does impact reliability. School Attitude reliability fell below the reasonable interpretation

threshold and thus any significant results are disregarded.

Table 1 Pretest and Posttest Cronbach’s alpha for subscales

2003-

2004

2003-

2004

2004-

2005

2004-

2005

2005-

2006

2005-

2006

2006-

2007

2006-

2007

2015* 2015

Pre Post Pre Post Pre Post Pre Post Pre Post

Locus of

Control

.797 .807 .78 .83 .79 .81 .81 .82 .78 .76

School

Attitude

.750 .779 .75 .75 .79 .78 .71 .74 .65 .52

14

2003-

2004

2003-

2004

2004-

2005

2004-

2005

2005-

2006

2005-

2006

2006-

2007

2006-

2007

2015* 2015

Self

Esteem

.666 .798 .81 .84 .83 .86 .82 .86 .91 .87

Attitude

toward

Violence

.717 .747 .73 .73 .80 .85 .77 .83 .77 .71

Moral

Scale

.776 .799 .83 .84 .86 .86 .80 .86 .70 .87

Science

Attitude

Scale

.957 .965 .95 .97 .96 .97 .94 .95 .90 .94

Talking

Talons

Scale

.680 .782 .76 .80 .71 .89 .77 .76 .77 .71

Posttest

only

Treatment

section

.94 .92 .76 .73

*note: Sample size for 2015 was much smaller than previous years, this tends to reduce

reliability

Quizzes

The treatment group students are also given a series of ten quizzes. These quizzes consist

of five knowledge questions and five attitude questions. The knowledge questions pertain to the

information taught by the program and the attitude questions cover attitudes towards the

program, the educator and science in general. Some science attitude questions are repeated

throughout the attitude section of the quizzes over the course of the program in order to measure

longitudinal changes in attitude scores.

15

These quizzes are used to determine both the changes in science knowledge and in attitudes

over the course of the program.

Reliability and Scale development

The quizzes were used to generate several subscales. For each subsection the internal

reliability was examined. Internal reliability is how much consistency is apparent in the answers

of one subtest. Internal consistency is measured with a scale called “Cronbach's Alpha”.

Technically speaking, Cronbach's alpha is not a statistical test - it is a coefficient of reliability (or

consistency). Alpha is calculated by one split-half reliability and then randomly dividing the

items into another set of split halves and recomputing until, all possible split half estimates of

reliability have been computed. In other words, each item is compared to the group in all

possible combinations of items. Cronbach's alpha measures how well a set of items (or variables)

measures a single unidimensional latent construct. When data have a multidimensional structure

(they are not all measuring the same construct) then Cronbach's alpha will usually be low. If the

number of items is increased and the consistency remains the same the Cronbach’s alpha will

increase. Additionally, if the average inter-item correlation is low, alpha will be low. As the

average inter-item correlation increases, Cronbach's alpha will also increase. This effect makes

sense intuitively - if the inter-item correlations are high, and then there is evidence that the items

are measuring the same underlying construct.

Recall that an alpha value of .70 is considered acceptable; however we would like it to be

higher. Reliabilities are presented in each section of the report.

16

Talking Talons Quiz Knowledge

All of the scales are mean scores. Thus if an educator did not give one of the quizzes

containing a question for that subset it is not entered into the mean. This can impact the results

because the score is then based on fewer data points. All attitude scales are recoded so that 6

represents the highest and best score and 1 is the lowest and worst score. For the quizzes 5 is the

highest possible score and 0 is the lowest. Remember that this is a very simple way at looking at

scores. With simple means one participant who has a negative change of 6 point will mask the

positive change of 1 point for 6 participants. This is especially true for small sample sizes (the

smaller classrooms). An educators with a score of 5.5 on the scale could have had half the kids

answer “6” and half answer “5” OR almost all kids answering 6 with a few answering “1”. Keep

this in mind when interpreting results.

Bar charts are good visual presentations of information. However, they may lead to over

interpretation. As the standard deviations are not listed on a bar chart it is difficult to ascertain

true significant differences. One must ask “Does that “3” on a bar chart mean that almost

everyone answered 3 or that half the participants answered “5” and half answered “2”? “

Significance tests indicate actually differences by utilizing the standard deviation as well.

Therefore two bars on a chart may look very different and not be statistically different or may

look similar but actually represent a statistical difference.

The knowledge sections relate to the topics studied in the program and includes the following

topics Introduction, Hawks, Raptors, Bats 1, Bats 2, Owls, Reptiles,Vocabulary General and

Energy. Five questions related to these topics are asked on each of the quizzes.

17

Talking Talons Quiz Attitudes

The back of each quiz includes an attitude section; these questions are then utilized for the

following subscales.

Overall perception of educator (Reliability α=.90)

My Talking Talons educator (teacher) is easy to understand.

My Talking Talons educator (teacher) understands how kids think.

My Talking Talons educator (teacher) knows a lot about the animals.

My Talking Talons educator (teacher) speaks clearly and loudly enough for me to hear

and understand.

My Talking Talons educator (teacher) pays attention to all of the kids in my class.

My Talking Talons educator (teacher) treats me kindly.

My Talking Talons educator (teacher) is prepared to teach us when he/she comes to class.

My Talking Talons educator (teacher) was fair to everyone in my class.

Bonding with Animals (Reliability α=.68)

In the future, I would like to be a Talking Talons Educator and hold a bird.

I would like the chance to hold a Talking Talons reptile (like a snake or lizard).

I think about the Talking Talons animals when I am not in science class

18

I feel comfortable around the Talking Talons animals (eliminated due to small sample

size)

Seeing the Talking Talons animals makes me feel happy.

Perceived Enjoyment of Talking Talons program (Reliability α=.78)

I like learning about animals.

I look forward to the days that Talking Talons comes to the classroom

I talk to kids who are not in the Talking Talons program about the program.

I would like the Talking Talons program to come to my science class next year.

Perceived Ability (Reliability α=.72)

The quizzes for Talking Talons were very hard.

I try my hardest to do well in school.

I am good at science.

I think I will get a better grade in science class this year than last year.

I understood most of the information in the Talking Talons program.

Attitude toward science (Reliability α=.82)

I like science class.

Science class is boring

19

I like science class.

Attitude toward environment (Reliability α=.68)

It is ok to shoot hawks because they kill smaller birds.

It is ok to throw trash out the window because the highway department picks it up.

I think teaching other kids about the environment is important

Teacher Feedback Form

At the end of the program an online form (to preserve teacher anonymity) is used to

collect data on the classroom teachers’ perceptions of the program, of changes in attitude of

his/her students and of the educator.

Qualitative Data Collection

A sample of five to six students from each group was interviewed by the evaluator for

feedback about the Talking Talons program. These interviews were recorded and transcribed.

Student names were removed from the transcription to preserve anonymity.

Participants

20

Participant demographic information is included below. For each subscale the total

number of student varies slightly due to outliers and participants who did not finish a

particular subscale. The total number can be found in the table of means for each subsection.

Because of the small sample size for the 2015 program, missing posttest data for the control

group was imputed from representative previously collected control test data for four cases in

order to reach a statistically useable sample size. Students were all between 10 and 11 years

old at post testing.

Table 2 Participants by Group

Frequency Percent Valid Percent Cumulative Percent

Valid Control 23 29.9 29.9 29.9

Treatment 54 70.1 70.1 100.0

Total 77 100.0 100.0

One student did not provide gender.

Table 3 Participants by Gender

Unknown Female Male

group Control 0 13 10 23

Treatment 1 21 32 54

Total 1 34 42 77

21

Table 4 Group by Class

class

Total Control Teacher A Teacher B

group Control 23 0 0 23

Treatment 0 29 25 54

Total 23 29 25 77

Explanation of GLM Repeated Measures

The data for the composite file were analyzed using SPSS General Linear Model

Repeated Measures (GLM). The GLM Repeated Measures procedure is based on the general

linear model, in which factors and covariates are assumed to have linear relationships to the

dependent variables.

This section explains how to interpret the graphs and data from the analysis of the

composite section. For simplicity of explanation sample graphs and data are used which make

the differences clear for interpretation.

A GLM repeated measures with control groups is used to examine the following possibilities:

22

QUESTION: Are the control and treatment groups similar to each other?

VISUAL ANSWER: Are the two lines far apart (be sure to look at the scale of the graph)

from each other, especially at pretesting?

STATISTICAL ANSWER: If the two groups are not similar to each other than the test

of between groups will be significant.

QUESTION: Does the score on the instrument change over time from pretesting to post testing

for one or both of the groups?

VISUAL ANSWER: Is the line flat or sloped for the groups?

STATISTICAL ANSWER: If the scores are changing overall in the same direction and

magnitude then the “Time” variable will be significant.

QUESTION: Is the change over time different for the two groups? (Either one group goes up

and one goes down or the magnitude of the change is different.)

VISUAL ANSWER: Is the slope of the line different (in direction or magnitude) for

each group?

STATISTICAL ANSWER: If the scores are changing, but differently for each group

then the “Time by Group” variable will significant overall. The difference can be either

in different directions or in different magnitudes.

23

These questions are answered statistically in the tables. However, for ease of interpretation the

statistical significance is listed under the graph showing change.

Examples

Groups are different but with no change over time

The first question is answered statistically in a table called “Between groups’ differences”

but can also be seen by the distance between the score values of the control and treatment

groups. Note the scale on the graph as well when interpreting the data.

The first graph shows the following

No change over time for either group (the slope of the lines are flat)

Groups are different from each other (there is a big gap between the Treatment and

Control Group lines)

24

No Change over Time

Groups are different

TIME

21

Estim

ate

d M

arg

ina

l M

ea

ns

4.5

4.0

3.5

3.0

2.5

2.0

1.5

1.0

.5

GROUP

Treatm ent

Control

Groups are similar and both change over time in the same way

The second graph shows the following

both change from pretesting to post testing (the slope of the line is not flat)

Both change in the same manner (the slope is in the same direction and at the same

angle).

25

Change over Time

TIME

21

Estim

ate

d M

arg

ina

l M

ea

ns

7.0

6.5

6.0

5.5

5.0

4.5

4.0

3.5

GROUP

Treatm ent

Control

Groups are the same but change differently over time

The third graph shows

change over time for both groups

Change is not in the same direction.

26

Different Changes Over Time

TIME

21

Estim

ate

d M

arg

ina

l M

ea

ns

6

5

4

3

2

1

0

GROUP

Treatm ent

Control

Obviously with real data the differences are not always so clear. However, this

explanation should make visual interpretation of the results simpler. Statistical results are

presented in the tables below the charts for those who are interested in the means, levels of

significance, effect size and power for the results. Type III Sums of squares were used as the

hypotheses being tested involve only marginal averages of population cell means. Greenhouse-

Geisser epsilon was used to adjust degrees of freedom if sphericity was violated as per

assessment with Mauchly's test. These statistics are available from the researcher if desired.

Estimated marginal means of the dependent variables (with covariates held at their mean value)

for specified between- or within-subjects factors in the model are provided. In this research these

predicted means are equivalent to observed means as the covariates are categorical.

27

Outlying scores due to extreme responses may influence results. In order to identify

outliers the score distributions for the participants for all observed variables were examined

univariately. Scores that were more than three standard deviations from cell means and were

also discontinuous from their closest neighboring scores were considered univariate outliers and

were removed on a subscale by subscale basis.

For each subsection the results were examined by GLM repeated measures using the

following group memberships.

Overall

By gender

The results are then presented for results that yielded significant or interesting result for

the difference between the groups if there was no difference then overall results are presented

unless a trend that merits consideration was present. Recall that as the group is subdivided the

membership within that group becomes smaller and significance is more difficult to detect.

Science Attitudes

Research on Science Attitudes

Attitudes toward science have long been examined as an important factor in student

science education. Researchers have found that attitudes toward science impact future course

selection (Farenga and Joyce), motivation to study science (Slate and Jones), achievement in

28

science (Oliver and Simpson; Kose et al.; Smith, Pasero, and McKenna; Baker; Oliver;

Reynolds and Walberg; Warburton, Jenkins, and Coxhead; Willson) and general attitudes toward

school (Jarvis and Pell) ). Improving student’s attitudes toward science could therefore be said

to have many positive outcomes.

However as students enter the middle school years, their generally positive attitudes

about science decrease (Mattern and Schau; Desy, Peterson, and Brockman; Catsambis;

Mattern; Sorge) especially for girls (Backes; Lee and Burkam; Oliver; Papanastasiou and

Zembylas; Warburton, Jenkins, and Coxhead) .

Of particular relevance is research examining changes in attitudes exhibited after

exposure to “hands on” or inquiry based science. These programs have been found to improve

attitudes toward science with increased frequency of hands on experiments leading to most

positive attitudes (Ornstein) and with hands on laboratory experiment increasing knowledge

(Freedman). Interestingly, girls were found to be impacted more (Teshome, Maushak, and

Athreya) by hands on experimentation than boys.

Specific research on involving students in environmental science projects in order to

increase both knowledge and attitudes toward science has statistically positive outcomes on both

factors (Al-Balushi and Al-Aamri).

Science attitudes for the Talking Talons students were examined in multiple forms.

Pretest and posttest attitudes were compared for control and treatment groups. For the treatment

groups the same question about attitudes towards science was asked throughout the program in

29

order to measure longitudinal change. Information was collected from the classroom teacher and

from interviews.

Science attitude change during program for participants

In order to examine longitudinal changes in science attitude for the treatment group

(those students who participated in the Talking Talons program), science attitude questions were

asked at pretesting, on quiz 3, on quiz 7 and at posttesting. The chart below delineates the

change over time. This change was statistically significant (see table for results)

Figure 1 Longitudinal change in Science Attitude

30

Table 5 Science Attitude Change over Time

Descriptive Statistics

Mean Std. Deviation N

Science Attitude Post. 4.9032 1.35043 31

Science Attitude Quiz 3 5.32 .945 31

Science Attitude Quiz 7 5.48 .811 31

Science Attitude Post. 5.6452 .70938 31

Table 6 Science Attitude Change over Time Multivariate Tests

Multivariate Testsc

Effect

Value F

Hypothesis

df

Error

df Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powerb

linsc Pillai's

Trace

.252 3.148a 3.000 28.000 .041 .252 9.443 .667

a. Exact statistic

b. Computed using alpha = .05

c. Design: Intercept

Within Subjects Design: linsc

Table 7 Science Attitude Change over Time within Subjects Effects

Tests of Within-Subjects Effects

Measure:MEASURE_1

31

Source Type III Sum

of Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

linsc Sphericity

Assumed

9.452 3 3.151 5.151 .002 .147 15.453 .913

Error(linsc) Sphericity

Assumed

55.048 90 .612

Lower-

bound

55.048 30.000 1.835

a. Computed using alpha = .05

Table 8 Science Attitude Change over Time Within subjects Contrasts

Tests of Within-Subjects Contrasts

Measure:MEASURE_1

Source linsc Type III Sum of

Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

linsc Linear 8.832 1 8.832 9.647 .004 .243 9.647 .852

Error(linsc) Linear 27.468 30 .916


32

Table 9 Science Attitude Change over Time between Subject Effects

Tests of Between-Subjects Effects

Measure:MEASURE_1

Transformed Variable:Average

Source Type III Sum of

Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

Intercept 3534.226 1 3534.226 1730.366 .000 .983 1730.366 1.000

Error 61.274 30 2.042


Science Attitude change from pretest to posttest by group

Both the treatment and control group were assessed for science attitude changes from

pretesting to postttesting. A statistically significant change in Attitude toward Science was seen

in the treatment group and not for the control group from pretesting to post testing.

F(1,58)=12.25. p<.001 η2=.17. This represents a medium effect size.

Figure 2 Change in Science Attitude from Pretesting to Posttesting by Group

33

Table 10 Change in Science Attitude by Group


group Mean Std. Deviation N

sci

dimension1

Control 5.2609 .87684 23

Treatment 5.0246 1.01946 37

Total 5.1152 .96657 60

psci

dimension1

Control 4.4314 1.49023 23

Treatment 5.2713 .96420 37

34



Total 4.9494 1.25098 60

Table 11 Change in Science Attitude by Group Multivariate Tests

Multivariate Testsc

Effect

Value F

Hypothesis

df

Error

df Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powerb

science Pillai's

Trace

.058 3.592a 1.000 58.000 .063 .058 3.592 .462

science *

group

Pillai's

Trace

.174 12.251a 1.000 58.000 .001 .174 12.251 .931

a. Exact statistic


c. Design: Intercept + group

Within Subjects Design: science

Table 12 Change in Science Attitude by Group Within subjects Effects


Source Type III Sum

of Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

science Sphericity

Assumed

2.408 1 2.408 3.592 .063 .058 3.592 .462

35


Source Type III Sum

of Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

science *

group

Sphericity

Assumed

8.213 1 8.213 12.251 .001 .174 12.251 .931

Error(science) Sphericity

Assumed

38.885 58 .670


Table 13 Change in Science Attitude by Group within Subjects Contrasts


Measure:MEASURE_1

Source science Type III Sum

of Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

science dimension2

Linear 2.408 1 2.408 3.592 .063 .058 3.592 .462

science *

group dimension2

Linear 8.213 1 8.213 12.251 .001 .174 12.251 .931

Error(science) dimension2

Linear 38.885 58 .670


36

Table 14 Change in Science Attitude by Group between Subject Effects


Measure:MEASURE_1


Source Type III Sum of

Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

Intercept 2833.325 1 2833.325 1680.791 .000 .967 1680.791 1.000

group 2.584 1 2.584 1.533 .221 .026 1.533 .230

Error 97.771 58 1.686


Table 15 Change in Science Attitude by Group Estimated Means

group * science

group science

Mean Std. Error

95% Confidence Interval

Lower Bound Upper Bound

dimension1

Control

dimension2

Pre 5.261 .202 4.857 5.665

Post 4.431 .248 3.934 4.929

Treatment

dimension2

Pre 5.025 .159 4.706 5.343

Post 5.271 .196 4.879 5.663

37

Table 16 Difference Change in Science Attitude by Group

Change in Science Attitude

group Mean N Std. Deviation

Control -.8295 23 1.46536

Treatment .2467 37 .92088

Total -.1658 60 1.26354

Student data from quizzes.

Table 17 Attitude toward Science from Quizzes Mean


N Minimum Maximum Mean Std. Deviation

Attitude science 47 3.67 6.00 5.3936 .69368

Valid N (listwise) 47

Student Qualitative Feedback on Science Attitudes

Interview sections pertaining to science attitudes for the treatment group are excerpted

below.

38

Student feedback on science from interviews

(Note: I=Interviewer, Students are indicated by a letter)

Classroom B:

I: Ok, so what about the TT program compared to your regular science program? Which do

you prefer?

[All talking over each other]

Student B: I definitely prefer TT.

Student A: Me too.

Student B: Talking Talons.

Student C: Talking Talons, definitely. Definitely.

Student A: I mean it’s cool that you get to learn about animals and like things like that, I’m

an animal lover. I ADORE animals and um.. and also I get to see them, so that’s even better.

I: And they are kind of unusual animals, right?

Student A: Yeah

Student B: Animals you won’t find in New Mexico.

Student A: Well actually. You find them, but you don’t get to get that close to them. They are

usually trying to get away from you.

I:Do you look forward to the days Talking Talons comes?

39

[All talking over each other] Yeah, Yeah, Yes

Student A: I really do. When I had a (redacted due to identifying information) test in here I

was really sad because it was Talking Talons day and I was like WHAT! I missed out.

Classroom A:

I: Do you guys look forward to the days talking talons comes to the classroom?

yes, yeah [all nodding heads]

I:So you are happy when TT is coming?

Student E: Sometimes it gives us a break from normal class and we can learn something

new about animals.

Student F: We learn something new every time we go, mammals reptiles it’s really

interesting

Student G: I like to tell my dad what happened at a school

I: Yeah it gives you something interesting to talk to your parents about?

Student G: it’s really interesting

I: What about you, what do you think? Are you happy on the days Talking Talons is coming to

school …

Student H: I’m happy about it

40

Classroom Teacher feedback about science Attitudes

The classroom teachers indicated that they perceived that the student’s both learned a lot

of science from the program (mean score 6.5/7) and that the program had a positive impact on

student attitudes toward science (mean score 6./7).

Table 18 Classroom Teacher assessment of Science Attitude change

The students learn a lot of science

from the TT program.

The TT program had a positive impact on the

student’s attitudes about science.

Mean 6.5000 6.50

Std.

Deviation

.70711 .707

Hands on science experiments and Animals in the classroom

Research on Hands on Science and Animals in classroom

Significant contribution of hands on learning to science knowledge has been identified as

making a significant contribution to peer interaction through cooperative learning, object-

mediated learning and embodied experience. (Satterthwait). The more hands on experience, the

more learning occurs “Specifically, students who engaged in hands-on activities every day or

41

once a week scored significantly higher on a standardized test of science achievement than

students who engaged in hands-on activities once a month, less than once a month, or never.”

(Stohr-Hunt)

When compared to textbook learning, hands on (inquiry) also improves achievement

“Students in the inquiry-based group reached significantly higher levels of achievement than

students experiencing commonplace instruction. This effect was consistent across a range of

learning goals (knowledge, reasoning, and argumentation) and time frames (immediately

following the instruction and 4 weeks later).” (C. D. Wilson et al.)

Hands on learning is more effective for student comprehension even when compared to

teacher demonstration “students in the hands- on laboratory class performed significantly better

on the procedural knowledge test than did students in the teacher demonstration class. These

results were unrelated to reasoning ability.” (Glasson). This impact has been shown to influence

traditionally underrepresented minorities attitudes toward science and career plans. (Kanter and

Konstantopoulos). This is especially important as the impact of No Child Left behind has led to

less teaching of science in the classroom. (Milner et al.)

Furthermore, the use of animals in the classroom has been found to teach humane values

(Zasloff et al.) while improving young students attitudes toward animals (Ellery Samuels, Nicoll,

and Trifone) and towards other humans (Arkow) . The use of animals also improves

environmental education for young children (Margadant-Van Arcken) and knowledge about

animals specifically (Kimble).

42

The specific use of more exotic animals (zoo outreach) found that students recorded more

observations, made more use of science facts and use more science vocabulary when writing

about more exotic animals rather than more ordinary pets. (Trainin et al.) and also used more

advanced literary concepts (K. Wilson et al.)

Classroom Teacher feedback

Classroom teachers gave the maximum score possible (7/7) to all factors relating to the

effectiveness of the hands on and animal sections of the Talking Talons program.

Table 19 Classroom teacher assessment of program components

The animals are

an important part

of the TT

program for the

students.

The public

speaking

component of TT

is an important part

of the program.

The hands on

activities are an

important part of

the TT program.

Presenting to the Buddy

class made the students

take the responsibility

of TT more seriously.

Mean 7.00 7.00 7.00 7.00

Std.

Deviation

.000 .000 .000 .000

Student Feedback on Animals in classroom and Hands on science

43

Sections of the interview pertaining to the animals and hands on learning are excerpted

below.

I:Ok, now I am going to ask you about what the best things were about Talking Talons and what

things could they do better? So what were the best things and what things would you improve?

What do you think Student B?

Student B: Hm, can I pass

I:Can you pass and I can come back to you?

Student B: To be honest I see no problems.

I:Would there be anything you would like to add.

Student C: More animals?

Student A: That would be awesome but like I don’t see much problems and like the animals

and everything and getting to hold them like the snake I was amazed by how it felt because

when it would like move it was just like plastic moving around it was like, this is really soft

Student B: and cold!

I: Yeah, cold! You hear cold blooded but you don’t know what that’s like until you touch

them, right?

Student A Because of their shiny skin you would think they would be like wet or something

and it’s like really fascinating. So I don’t see like much problem, I don’t see any problem at

all because everyone’s really nice and everything about it so it’s pretty cool

44

I: What about you Student C?

Student C: Actually, I actually want more activities to do.

I: Hands on activities? So which is more fun, doing the animal presentations or doing the

animal activities like the eyeballs?

Student B: I say both.

Student C: both

Student A: That’s really hard because they are both really fun.

I: What about you Student D? You like the hands on, or the animal part?

Student D: I like both

I: Were any of you afraid of any of the animals?

[Chorus of voices] No, no, nope (all indicated no and Student B and Student A were

shaking their heads no as well)

Student A: I understand that like some people leave because of like religious reasons.

Pretty much everyone thought it was fun.

I: Do you guys talk about Talking Talons like when you are not in class or not? So, do you go

home and tell your mom and dad if you got to hold the snake?

Student A: Yes, I do because it’s interesting and we do like at the dinner table…like how

was your day but not like always.

45

Student B: Like if I go to parks and recs and my dad picks me up and asks me how my

day was and I’m like good and then we get home and then I’m like hey I learned this

today at Talking Talons and I’ll say like this animal does like this and he will say oh

that’s cool

I: to Student C: Do you talk to your parents about it?

Student C: Oh yeah, my dad’s very annoyed with it

I:Annoyed?

Student C: Because I am talking about it too much.

I: to Student D: Do you talk about it?

Student D: No not really.

I: Do you talk to anybody else about it?

Student D: Oh, I do talk to my friends in (Teacher A) class because they have it; it’s not in

too many classes.

Student A: or if there is something like really exciting, Like when Animal A came, like

when that happens like that incident.

Student B: He pooped on the carpet

Student C: Right.

[All of them laughing]

Student C: Yeah like he literally pooped on the carpet

I:So if you could be in Talking Talons again you would do it?

[All talking at once] yes, yeah, of course

46

I: So even if you were older, like if they had Talking Talons in high school?

Student A: Probably

Student B: Maybe

Student D: Yeah

Student B: Yeah. Actually, to be honest I like Talking Talons because we don’t have to

do math.

[All laughing.]

Student D: That’s a plus

Buddy class

Research on peer tutoring

One aspect of the Talking Talons program is the peer interaction. Students in the Talking

Talons program learn the material with the goal of presenting to a younger group of students in

the school.

A meta-analysis of peer assisted learning found increases in achievement. Peer tutoring is

most effective with “PAL interventions were most effective with younger, urban, low-income,

and minority students.”(Rohrbeck et al.).

Peer tutoring helps reading fluency (Kamps et al.), reading comprehension (McMaster,

Fuchs, and Fuchs) mathematics learning (Fantuzzo, Polite, and Grayson) both mathematics and

47

social skills (Pigott, Fantuzzo, and Clement) English (Greenwood et al.) and social interaction

for rejected boys (Gumpel and Frank).

Meta analyses on impact on math found the effect is particular strong for elementary aged

students (Kunsch, Jitendra, and Sood). A hands on gardening program found improvement in

social relationships in the classroom (Kim, Park, and Son).

Specifically buddy lesson plans for healthy eating and exercise physical activity, healthy

eating, and self-esteem and body image where lesson plans were delivered by 9-12 year olds for

6 to 8 year olds found that “Reductions in waist circumference were particularly significant for

children who were younger, overweight or obese, or attending First Nations schools. No

difference in body mass index score was observed between groups. Self-efficacy, healthy living

knowledge, and dietary intake significantly improved in younger peers who received the

intervention compared with students from control schools.” (Santos et al.).

Meta-analysis also shows that this impact is positive for both the “teacher” and the

“student” in peer tutoring situations. “This review of the literature on peer and cross-age tutoring

emphasizes programs in mathematics and suggests that such programs have positive academic

outcomes for African American and other minority students as well as for White students who

participate as tutors, as tutees, or both”. This results was shown even with short programs

“tutors show academic gains even when they do not receive additional subject matter instruction,

why longer and/or more substantial tutoring programs may not foster greater immediate

academic gains than shorter programs,” (Robinson, Schofield, and Steers-Wentzell)

48

Teaching benefits the tutor specifically in science classes “Mental rehearsal of peer-tutoring

episodes helped them appreciate weaknesses in their own subject knowledge.” (Galbraith and

Winterbottom)

Meta analyses also supports that peer tutoring is effective regardless of many factors.

“This meta-analysis examined effects of peer tutoring across 26 single-case research experiments

for 938 students in Grades 1–12. The Tau effect size for 195 phase contrasts was 0.75 with a

confidence interval of 0.71 to 0.78, indicating that moderate to large academic benefits can be

attributed to peer tutoring. Five potential moderators of these effects were examined: dosage,

grade level, reward, disability status, and content area. This is the first peer tutoring meta-

analysis in nearly thirty years to examine outcomes for elementary and secondary students, and

extends previous peer tutoring meta-analyses by examining disability as a potential moderator.

”Findings suggest that peer tutoring is an effective intervention regardless of dosage, grade level,

or disability status. Among students with disabilities, those with emotional and behavioral

disorders benefitted most.” (Bowman-Perrott et al.)

Classroom teacher assessment of Buddy class

The classroom teachers all agreed very strongly (mean of 7/7) that having the buddy

class had a positive impact on their students.

49

Table 20 Classroom teacher assessment of buddy class impact

Presenting to the Buddy class made the students take the responsibility of TT more seriously.

N Valid 2

Missing 0

Mean 7.00

Std. Deviation .000

Student feedback on Buddy class

Students universally expressed that they enjoyed “being the teacher” and presenting to

younger students. They mentioned leadership directly as well. Sections related the buddy class

are excerpted from interviews below.

I: Did you like doing the presentations or not?

[All jump in and speak at once] yeah; I liked it [nodding heads]

Student E: I like doing it to the other class.

Student G: I like teaching the little kids.

Student G:Cause you like see (Talking Talons educator) come in and do it, and I always

wanted to do it like that

Student F: It’s like you are the teacher

50

Student E: So you get to be like them, it’s really good.

I: What do you think?

Student H: It teaches us how to be leaders, it’s kind of like nerve-wracking to talk to kids

though but its pretty fun, they get to see animals.

I: What about you, did you like doing presentations?

Student K: I have not done one yet.

2:05

I:You have not done one yet?

Student F: He goes somewhere during class sometimes.

I: What are the best things about Talking Talons?

Student H: You learn about new things and all sorts of animals.

I:What did you guys think about the buddy class?

Student E and F speaking alternatively: I love it because they are really focused…they are

quiet, they defiantly chose it…they defiantly chose a good class.

Student E: Both my cousins are in there

I: Are your cousins in there. So do you like having them come in?

Student E: I think it’s a good idea, it’s like Ms. Betsy and Ms. Laurie teaching us and then we

are teaching the buddy class.

51

Student F: It’s like my grownup mode just turns on.

[Laughter]

Student G: It’s like we are the teacher

Class B:

I:Ok, what about the presentations, did you like doing the presentations?

Student A: Yeah that was really fun, I had the snake we did too,

Student B: we were like first we had the snake and it was really cool experience that we got

to teach. : I like the presentations because we get to hold the animals.

Student A: Also its cool being like everyone watching you and they are like hi.

I: Oh, you liked that, you liked being in front of the class? What about you, did you like being in

front of the class?

Student B: Yeah, me and (pointing to Student A) got to be in front of the class, in front of

four thousand people before.

Student A: Yeah we were singing.

Student B: We went to a choir, like together, and my like there’s like 4000 people

Student A: and then I’ve also like done dancing

I: So you are used to being in front of the class. What about you?

Student D: I’m fine with people

I: You didn’t have any problem doing the presentations?

52

I: Did you liked doing them, what did you think of the having the buddy class? Was that a

good thing or a bad thing?

Student A” Well it’s pretty cool because you get to hear other people’s questions and

things. And like that and you know are like, like the teacher.

I:What did you think?

[Student B Laughs]

I: So Student C did you like having a buddy class come in or would you rather that there was not

a buddy class, be honest this does not affect the funding it just tells us whether you think it was

useful.

Student C : I would like the buddy class, I feel like it is in addition to our class so it’s kinda

cool how having third graders, seeing how third graders take on what we are learning.

Student A yeah Plus it’s cool because it gives other people a chance to like, experience

everything because I used to be in (lower grade teachers) and at one point in time she was

actually doing the buddy class kind of thing so it was really cool

I: Oh so you been a buddy class?

Student A: Yeah I’ve been a buddy class and I am doing that now

I: That is really cool, and so what do you remember from being a buddy class?

Student A: I remember Banana, she stood out a lot.

I:[ laughing] Banana?

Student A: I miss Banana! And it was like really cool, I just went in the classroom and I was

like, oh my gosh there’s animals in here. Cause you can’t even have a dog most of the time.

It’s like you guys get to bring in birds and snakes and bats and you can’t even bring in a dog!

I: That’s true. So you guys were good with the buddy class? You would keep the buddy class?

53

[All four]: Yeah, yes

Student B: It was really awesome.

Role Model for younger students

The change in role model was not significant, the power was low due to sample size.

However, the direction of change was positive for the treatment group and negative for the

control group. This information is included for trends, rather than statistical significance.

Figure 3 I am a good role model for younger students

54

Table 21 Student answers by group to "I am a good role model for younger students"


I am a good role model for younger

students.

dimension1

Control 4.9130 1.50493 23

Treatment 5.0750 1.32795 40

Total 5.0159 1.38532 63

POST I am a good role model for

younger students.

dimension1

Control 4.4783 1.83079 23

Treatment 5.2750 1.06187 40

Total 4.9841 1.43113 63

Table 22 I am a good role model significance tests

Multivariate Testsc

Effect

Value F

Hypothesis

df

Error

df Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powerb

rolem Pillai's Trace .004 .264a 1.000 61.000 .609 .004 .264 .080

rolem *

group

Pillai's Trace .031 1.929a 1.000 61.000 .170 .031 1.929 .277

a. Exact statistic



Within Subjects Design: rolem

Science Knowledge

Change in Science Knowledge by Group

55

A statistically significant change in Knowledge was seen in the treatment group and not

for the control group from pretesting to post testing. F(1,55)=11.01. p<.01 η2=.45. This

represents a very large effect size.

Figure 4 Change in Science Knowledge by Group

56

Table 23 Change in Science Knowledge by Group



know

Control 4.3533 .71249 23

Treatment 3.7493 .51974 34

Total 3.9930 .66932 57

pknow

Control 4.0550 .43830 23

Treatment 4.6221 .61254 34

Total 4.3933 .61266 57

Table 24 Change in Science Knowledge by Group Multivariate tests

Multivariate Testsc

Effect

Value F

Hypothesis

df

Error

df Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powerb

knowled Pillai's

Trace

.167 11.010a 1.000 55.000 .002 .167 11.010 .903

knowled *

group

Pillai's

Trace

.454 45.737a 1.000 55.000 .000 .454 45.737 1.000

57

a. Exact statistic


Control. Design: Intercept + group

Within Subjects Design: knowled

Table 25 Change in Science Knowledge by Group Means

group knowled

Mean Std. Error

Control

Pretest 4.353 .126

Posttest 4.055 .115

Treatment

Pretest 3.749 .104

Posttest 4.622 .094

Quiz Knowledge

The information below indicates the mean score for the treatment group on the

knowledge sections of the quizzes. A table with the standard deviation is also included. The

educators have been given the results by question in order to examine areas which need more

58

attention. This document is available upon request and is included on the CD. A comparison

was also done by class. For all quizzes except the Reptile quiz, the results were not different by

class. For the Reptile quiz one teacher had slightly lower scores than the other teacher. There

were no significant differences by gender.

Figure 5 Quiz Knowledge section means

59

Table 26 Mean scores on Knowledge Quiz by topic and class


N Range Minimum Maximum Mean Std. Deviation

Q1 Intro. 43 4.00 1.00 5.00 3.8605 1.08192

Q2 Hawks. 39 3.00 2.00 5.00 3.4359 1.04617

Q3 Raptors. 37 2.00 1.00 3.00 1.9459 .40455

Q4 Bats 1. 46 4.00 1.00 5.00 4.2174 1.05226

Q5 Bats 2. 42 4.00 1.00 5.00 4.0238 1.09295

Q6 Owls. 42 4.00 1.00 5.00 3.7619 1.05483

Q7 Reptiles . 40 4.00 1.00 5.00 4.1000 1.29694

Q8 Vocab. 39 3.00 2.00 5.00 3.9744 .98641

Q9 General. 47 4.00 1.00 5.00 4.0426 1.14127

Q10 Energy . 43 4.00 1.00 5.00 3.8605 1.24559


class Q1

Intro.

Q2

Hawks.

Q3

Raptors.

Q4 Bats

1.

Q5 Bats

2.

Q6

Owls.

Q7

Reptiles .

Q8

Vocab.

Q9

General.

Teacher B Mean 3.8696 3.4211 2.0000 4.3333 4.2381 4.1500 4.5714 4.0909 3.9615

N 23 19 25 24 21 20 21 22 26

Std.

Deviation

1.01374 .96124 .40825 1.04950 .99523 .87509 .81064 .92113 1.31090

Teacher A Mean 3.8500 3.4737 1.8333 4.0909 3.8095 3.4286 3.5789 3.8235 4.1429

N 20 19 12 22 21 21 19 17 21

60

Std.

Deviation

1.18210 1.17229 .38925 1.06499 1.16701 1.12122 1.53897 1.07444 .91026

Total Mean 3.8605 3.4359 1.9459 4.2174 4.0238 3.7619 4.1000 3.9744 4.0426

N 43 39 37 46 42 42 40 39 47

Std.

Deviation

1.08192 1.04617 .40455 1.05226 1.09295 1.05483 1.29694 .98641 1.14127

Table 27 ANOVA comparison on Quiz Knowledge by Class

ANOVA Table

Sum of

Squares df

Mean

Square F Sig.

Q1 Intro. * class Between

Groups

(Combined) .004 1 .004 .003 .954

Within Groups 49.159 41 1.199

Total 49.163 42

Q2 Hawks. * class Between

Groups

(Combined) .221 2 .111 .096 .908


Total 41.590 38

Q3 Raptors. *

class

Between

Groups

(Combined) .225 1 .225 1.391 .246

Within Groups 5.667 35 .162

Total 5.892 36

Q4 Bats 1. * class Between

Groups

(Combined) .675 1 .675 .604 .441


61

ANOVA Table

Sum of

Squares df

Mean

Square F Sig.

Total 49.826 45

Q5 Bats 2. * class Between

Groups

(Combined) 1.929 1 1.929 1.640 .208


Total 48.976 41

Q6 Owls. * class Between

Groups

(Combined) 5.926 2 2.963 2.911 .066


Total 45.619 41

Q7 Reptiles . *

class

Between

Groups

(Combined) 9.826 1 9.826 6.694 .014


Total 65.600 39

Q8 Vocab. * class Between

Groups

(Combined) .686 1 .686 .699 .408


Total 36.974 38

Q9 General. * class Between

Groups

(Combined) .382 1 .382 .289 .594


Total 59.915 46

Q10 Energy . *

class

Between

Groups

(Combined) 1.058 1 1.058 .676 .416

62

ANOVA Table

Sum of

Squares df

Mean

Square F Sig.


Total 65.163 42

No significance difference was found by gender, males and females had equivalent

scores on the knowledge sections of the test.

Table 28 Quiz Knowledge Means by Gender

Report

gender

Q1

Intro.

Q2

Hawks.

Q3

Raptors.

Q4 Bats

1.

Q5 Bats

2.

Q6

Owls.

Q7

Reptiles

.

Q8

Vocab.

Q9

General.

Q10

Energy

.

Mean 3.0000 3.0000 2.0000

N 1 1 1

Std.

Deviation

. . .

f Mean 4.1579 3.3333 1.8333 4.0500 4.1579 4.0556 4.3889 4.1250 3.8500 3.5500

N 19 18 18 20 19 18 18 16 20 20

Std.

Deviation

.89834 1.08465 .38348 1.09904 .89834 .80237 1.09216 .95743 1.03999 1.19097

m Mean 3.6250 3.5500 2.0526 4.3462 3.9130 3.5652 3.8636 3.8696 4.2692 4.1304

N 24 20 19 26 23 23 22 23 26 23

Std.

Deviation

1.17260 1.05006 .40465 1.01754 1.23998 1.19947 1.42413 1.01374 1.15092 1.25424

63

Report

gender

Q1

Intro.

Q2

Hawks.

Q3

Raptors.

Q4 Bats

1.

Q5 Bats

2.

Q6

Owls.

Q7

Reptiles

.

Q8

Vocab.

Q9

General.

Q10

Energy

.

Total Mean 3.8605 3.4359 1.9459 4.2174 4.0238 3.7619 4.1000 3.9744 4.0426 3.8605

N 43 39 37 46 42 42 40 39 47 43

Std.

Deviation

1.08192 1.04617 .40455 1.05226 1.09295 1.05483 1.29694 .98641 1.14127 1.24559

Table 29 ANOVA comparison for Talking Talons Quizzes by Gender

ANOVA Table

Sum of

Squares df

Mean

Square F Sig.

Q1 Intro. * gender Between

Groups

(Combined) 3.011 1 3.011 2.675 .110


Total 49.163 42

Q2 Hawks. * gender Between

Groups

(Combined) .640 2 .320 .281 .757


Total 41.590 38

Q3 Raptors. *

gender

Between

Groups

(Combined) .445 1 .445 2.856 .100


Total 5.892 36

64

ANOVA Table

Sum of

Squares df

Mean

Square F Sig.

Q4 Bats 1. * gender Between

Groups

(Combined) .991 1 .991 .893 .350


Total 49.826 45

Q5 Bats 2. * gender Between

Groups

(Combined) .624 1 .624 .516 .477


Total 48.976 41

Q6 Owls. * gender Between

Groups

(Combined) 3.022 2 1.511 1.384 .263


Total 45.619 41

Q7 Reptiles . *

gender

Between

Groups

(Combined) 2.731 1 2.731 1.651 .207


Total 65.600 39

Q8 Vocab. * gender Between

Groups

(Combined) .616 1 .616 .627 .434


Total 36.974 38

Q9 General. *

gender

Between

Groups

(Combined) 6.250 2 3.125 2.562 .089


65

ANOVA Table

Sum of

Squares df

Mean

Square F Sig.

Total 59.915 46

Q10 Energy . *

gender

Between

Groups

(Combined) 3.604 1 3.604 2.400 .129


Total 65.163 42

Self-perceived science grade in school

A statistically significant change in self-reported anticipated grade in science was

seen in the treatment group and not for the control group from pretesting to post testing

F(1,57)=19.83. p<.001 η2=.26. This represents a large effect size.

Figure 6 Self perceived change in science grade by group

66

Table 30 Group Comparison for I will get a good grade in science class this year



I will get a good grade in science class

this year

dimsion1

dimension1

Control 5.2609 1.28691 23

Treatment 5.1111 1.00791 36

Total 5.1695 1.11653 59

POST I will get a good grade in

science class this year

Control 4.1304 1.91417 23

Treatment 5.5833 .64918 36

67



I will get a good grade in science class

this year

dimsion1

dimension1

Control 5.2609 1.28691 23

Treatment 5.1111 1.00791 36

Total 5.1695 1.11653 59

POST I will get a good grade in

science class this year

Control 4.1304 1.91417 23

Treatment 5.5833 .64918 36

Total 5.0169 1.46795 59

Table 31 Group Comparison for I will get a good grade in science class this year Multivariate tests

Multivariate Testsc

Effect

Value F

Hypothesis

df

Error

df Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powerb

gradesci Pillai's Trace .055 3.346a 1.000 57.000 .073 .055 3.346 .436

gradesci *

group

Pillai's Trace .258 19.836a 1.000 57.000 .000 .258 19.836 .992

a. Exact statistic



Within Subjects Design: gradesci

68

Group Comparison for I will get a good grade in science class this year Within Subjects Contrasts


Measure:MEASURE_1

Source gradesci Type III

Sum of

Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

gradesci dimension2

Linear 3.040 1 3.040 3.346 .073 .055 3.346 .436

gradesci *

group dimension2

Linear 18.023 1 18.023 19.836 .000 .258 19.836 .992

Error(gradesci) dimension2

Linear 51.790 57 .909


Quiz Attitude subscales

Student mean attitudes about the program are presented below. The maximum possible

score is a 6/6 which indicates Strongly Agree.

69

Figure 7 Quiz Attitude subscales

Table 32 Quiz Attitude subscale descriptive statistics


Program enjoyment 48 3.00 6.00 5.3646 .76098

Reported understanding 49 4.25 6.00 5.3581 .46657


70


Attitude Environment 46 4.00 6.00 5.7717 .42362

Feedback on Educator

Student mean attitudes about the Talking Talon’s educators are presented below. The

maximum possible score is a 6/6 which indicates Strongly Agree.

Student Quantitative Feedback Figure 8 Student Perception of Educator

71

Student Feedback

Table 33 Student feedback on Educator Subscales


Perception of Educator 52 4.36 6.00 5.6282 .39387

Bonding with Educator 50 4.00 6.00 5.5850 .53066

Educator Teaching Skills 51 4.38 6.00 5.6880 .37678

72

Figure 9 Student Perception of Educator

Table 34 Student feedback on Educator from Quiz results


POST My Talking Talons

educator (teacher) cared a lot

about what I learned.

45 1 6 5.56 1.119


educator (teacher) was interested

in what I had to say.

45 3 6 5.69 .668


educator (teacher) was fair about

giving everyone an equal chance

to do things.

45 1 6 5.58 1.215

73



educator knew a lot about the

animals.

45 5 6 5.89 .318


educator (teacher) was easy to

understand.

45 3 6 5.67 .739

Figure 10 Student Feedback on Educator questions

Table 35 Student Feedback on Individual Educator questions

74



POST I think my Talking Talons

educator (teacher) liked coming

to teach my class.

45 4 6 5.78 .471


educator understands how kids

my age think.

45 1 6 5.38 1.211


educator treated boys and girls

equally.

45 3 6 5.87 .505


educator helped me to learn how

to give speeches.

45 1 6 5.64 1.026


educator was usually on time for

my class.

45 4 6 5.71 .626


Student Qualitative Feedback

Portions of the interview relating to the Talking Talon’s educators are excerpted below.

Student E: I like our teachers Ms. Betsy and Ms. Laurie because they always call on everybody

and they are always like…

Student F: focused

Student E: they are always like focused and they know what to say, they are always prepared

75

Student F ( over the top of Student E) always prepared

Ok

Student G: They are not always serious and uh…they joke around times.

[Laughing]

Classroom teacher feedback for Educator

The classroom teachers mean perception of the educator is presented below. These

questions are on a scale with a maximum positive score of seven.

Figure 11 Classroom teacher feedback on Talking Talons Educator

76

Table 36 Classroom Teacher Feedback on Talking Talons Educator


The TT educator was generally

prepared to teach the class.

2 5 7 6.00 1.414

The TT educator worked with me

to make the program run

smoothly.

2 6 7 6.50 .707

The TT educator used classroom

time well.

2 7 7 7.00 .000

77


The TT educator appeared

generally knowledgeable about

content presented

2 6 7 6.50 .707

The TT educator appeared to

enjoy teaching the students.

2 7 7 7.00 .000

The students appeared

comfortable with the educator.

2 7 7 7.00 .000


Figure 12 Classroom Teacher subscales on Educator

78

Classroom teacher feedback for Talking Talons Program

The classroom teacher’s mean perceptions of the Talking Talon’s program are presented

below. These questions are on a scale with a maximum positive score of seven.

Teacher Attitude Figure 13 Classroom teacher feedback on Talking Talons program

79

Table 37 Classroom Teacher Feedback Personal


I was able to tie the TT program

into my curriculum.

2 7 7 7.00 .000

I was frustrated with the TT

program.

2 1 1 1.00 .000

I found the material presented by

the TT program to be interesting

to me personally.

2 6 7 6.50 .707

I would be willing to have the TT

program in my classroom again.

2 6 7 6.50 .707

Effectiveness by student ability: feedback by classroom teachers

The classroom teachers mean perception of the impact of the program by student ability

is presented below. These questions are on a scale with a maximum positive score of seven. The

teachers found the program generally very effective for average students and very effective to

effective for advanced students. The program was found to be effective for those with learning

disabilities. These results mirror previous years feedback and indicate that the program is

reaching the majority of the student’s abilities at the correct level.

80

Figure 14 Classroom teacher assessment of Effectiveness of program by student ability

Table 38 Classroom teacher feedback program effectiveness by student ability


The TT program was

effective for above average

students.

2 6 6 6.00 .000

81


The TT program was

effective for average

students.

2 6 7 6.50 .707

The program was effective

for students with learning

disabilities.

2 4 6 5.00 1.414


The teachers found the program generally very effective for average students and

effective for advanced students and those with learning disabilities.

Classroom teacher subscales

The classroom teacher’s mean perceptions for subscales are presented below. These

questions are on a scale with a maximum positive score of seven.

Figure 15 Classroom teacher perception

82

Table 39: Classroom teacher subscales for Talking Talons program


Teacher science for students 2 5.67 6.33 6.0000 .47140

Teacher effectiveness by

student ability

2 5.33 6.33 5.8333 .70711

Teacher components of TT 2 7.00 7.00 7.0000 .00000

Teachers personal feelings

about TT

2 6.20 6.80 6.5000 .42426

83


Teachers perception of

educator

2 6.33 7.00 6.6667 .47140

Mean overall teacher

assessment of educators

2 6.18 6.71 6.4412 .37435


Other Results

Significant changes in pre posttest Composite

Moral Attitude

Research on Moral Attitude

A higher moral attitude has been linked to the ability to judge aggressive tendencies in

oneself (Berkowitz et al.)(). It also has an impact on moral behavior (Aquino and Reed) and

serves to decrease delinquency( (Raaijmakers, Engels, and Van Hoof; Tarry and Emler^;

Arbuthnot and Gordon)). Increases in moral attitude are linked with increases in academic

performance (Maguin and Loeber; Arbuthnot and Gordon) and decreases in actual aggressive

tendencies (Guivernau and Duda). Moral attitude has also been found to impact moral behavior

and empathy (Tangney, Stuewig, and Mashek; Malti and Krettenauer; Reynolds and Ceranic)

84

A statistically significant change in Moral Attitude was seen in the treatment group and

not for the control group from pretesting to post testing. F(1,58)=8.00. p<.01 η2=.12. This

represents a small/medium to effect size.

Figure 16 Change in Moral Attitude by group

85

Table 40 Change in Moral Attitude by Group


moral

dimension1

Control 5.5797 .46415 23

Treatment 5.3008 .60219 37

Total 5.4077 .56595 60

pmoral

dimension1

Control 5.2754 .99486 23

Treatment 5.5218 .55699 37

Total 5.4273 .75694 60

Table 41 Change in Moral Attitude by Group Multivariate tests

Multivariate Testsc

Effect

Value F

Hypothesis

df

Error

df Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powerb

dmoral1 Pillai's Trace .003 .202a 1.000 58.000 .655 .003 .202 .073

dmoral1 *

group

Pillai's Trace .121 8.006a 1.000 58.000 .006 .121 8.006 .795

a. Exact statistic



Within Subjects Design: dmoral1

86

Table 42 Change in Moral Attitude by Group Within subjects


Measure:MEASURE_1

Source Type III

Sum of

Squares df

Mean

Square F Sig.

Partial

Eta

Squared

Noncent.

Parameter

Observed

Powera

dmoral1 Sphericity

Assumed

.049 1 .049 .202 .655 .003 .202 .073

dmoral1 *

group

Sphericity

Assumed

1.957 1 1.957 8.006 .006 .121 8.006 .795

Error(dmoral1) Sphericity

Assumed

14.175 58 .244


Table 43 Change in Moral Attitude by Group Between subjects


Measure:MEASURE_1


Source Type III Sum

of Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

Intercept 3332.529 1 3332.529 5286.484 .000 .989 5286.484 1.000

group .007 1 .007 .012 .914 .000 .012 .051

Error 36.562 58 .630


87

Overall significant Composite results

Figure 17 Statistically significant Pre Post change by group

Table 44 Statistically significant Pre Post change by group

group Change in Moral

Attitude

Change in Science

Attitude Change in Knowledge

Control Mean -.3043 -.8295 -.2982

N 23 23 23

88

group Change in Moral

Attitude

Change in Science

Attitude Change in Knowledge

Std. Deviation .79717 1.46536 .73762

Treatment Mean .2209 .2467 .8728

N 37 37 34

Std. Deviation .63178 .92088 .56822

Total Mean .0196 -.1658 .4003

N 60 60 57

Std. Deviation .73948 1.26354 .86023

Table 45 ANOVA Statistically significant Pre Post change by group

ANOVA Table

Sum of

Squares df

Mean

Square F Sig.

Change in Moral

Attitude * group

Between

Groups

(Combined) 3.913 1 3.913 8.006 .006


Total 32.263 59

Change in Science

Attitude * group

Between

Groups

(Combined) 16.426 1 16.426 12.251 .001


Total 94.195 59

Change in Knowledge

* group

Between

Groups

(Combined) 18.814 1 18.814 45.737 .000


89

ANOVA Table

Sum of

Squares df

Mean

Square F Sig.

Change in Moral

Attitude * group

Between

Groups

(Combined) 3.913 1 3.913 8.006 .006


Total 32.263 59

Change in Science

Attitude * group

Between

Groups

(Combined) 16.426 1 16.426 12.251 .001


Total 94.195 59

Change in Knowledge

* group

Between

Groups

(Combined) 18.814 1 18.814 45.737 .000


Total 41.439 56

Table 46 Effect Size Statistically significant Pre Post change by group

Measures of Association

Eta Eta Squared

Change in Moral Attitude * group .348 .121

Change in Science Attitude * group .418 .174

Change in Knowledge * group .674 .454

Table 47 Non Significant Pre Post change by Group

90

Overall Non significant composite results

Note that the scale is small, and these bar charts represent small differences. These

results are not statistically significant and are presented to examine direction of change.

Figure 18 Non Significant changes for Composite

Table 48 Means Non Significant Pre Post change by Group

91

group Change in Self

Esteem

Change in Locus of

Control

Change in Attitude

toward School

Change in Attitude

toward Violence

Control Mean -.0755 -.4290 -.0413 -.3413

N 23 23 23 23

Std. Deviation 1.03937 .79424 .75388 .99780

Treatment Mean .1644 -.0626 .1194 .0635

N 37 37 37 37

Std. Deviation .69467 .74604 .63479 .90605

Total Mean .0725 -.2031 .0578 -.0917

N 60 60 60 60

Std. Deviation .84327 .77916 .68117 .95475

Table 49 ANOVA Non Significant Pre Post change by Group

ANOVA Table

Sum of

Squares df

Mean

Square F Sig.

Change in Self Esteem

* group

Between

Groups

(Combined) .816 1 .816 1.151 .288


Total 41.955 59

Change in Locus of

Control * group

Between

Groups

(Combined) 1.904 1 1.904 3.256 .076


Total 35.818 59

92

ANOVA Table

Sum of

Squares df

Mean

Square F Sig.

Change in Attitude

toward School * group

Between

Groups

(Combined) .366 1 .366 .786 .379


Total 27.376 59

Change in Attitude

toward Violence *

group

Between

Groups

(Combined) 2.324 1 2.324 2.620 .111


Total 53.781 59

Table 50 Effect size Non Significant Pre Post change by Group

Measures of Association

Eta Eta Squared

Change in Self Esteem * group .139 .019

Change in Locus of Control * group .231 .053

Change in Attitude toward School * group .116 .013

Change in Attitude toward Violence * group .208 .043

Table 51 Pre and Posttest means by group



sci dimension1

Control 5.2609 .87684 23

93



Treatment 5.1052 .91377 34

Total 5.1680 .89444 57

psci

dimension1

Control 4.4314 1.49023 23

Treatment 5.3379 .93918 34

Total 4.9721 1.26236 57

self

dimension1

Control 5.0242 1.22679 23

Treatment 5.1683 .70116 34

Total 5.1101 .94130 57

pself

dimension1

Control 4.9487 1.00337 23

Treatment 5.3276 .69226 34

Total 5.1747 .84445 57

moral

dimension1

Control 5.5797 .46415 23

Treatment 5.3652 .57605 34

Total 5.4518 .53986 57

pmoral

dimension1

Control 5.2754 .99486 23

Treatment 5.5645 .48619 34

Total 5.4479 .74068 57

viol

dimension1

Control 5.6261 .53361 23

Treatment 5.4235 .82941 34

Total 5.5053 .72615 57

pviol dimension1

Control 5.2848 .87431 23

94



Treatment 5.5279 .68503 34

Total 5.4298 .76898 57

attsch

dimension1

Control 5.4239 .83583 23

Treatment 5.3926 .47068 34

Total 5.4053 .63658 57

pattsch

dimension1

Control 5.3826 .75297 23

Treatment 5.4235 .52746 34

Total 5.4070 .62217 57

know

dimension1

Control 4.3533 .71249 23

Treatment 3.7493 .51974 34

Total 3.9930 .66932 57

pknow

dimension1

Control 4.0550 .43830 23

Treatment 4.6221 .61254 34

Total 4.3933 .61266 57

locus

dimension1

Control 5.0362 .84560 23

Treatment 4.6961 .79463 34

Total 4.8333 .82544 57

plocus

dimension1

Control 4.6072 .96740 23

Treatment 4.6000 .89680 34

Total 4.6029 .91739 57

95

Table 52 Pre and Posttest multivariate tests by group

Multivariate Testsc

Effect

Value F

Hypothesis

df

Error

df Sig.

Partial

Eta

Squared

Noncent.

Parameter

Observed

Powerb

Between

Subjects

Intercept Pillai's

Trace

.993 1057.283a 7.000 49.000 .000 .993 7400.980 1.000

group Pillai's

Trace

.163 1.359a 7.000 49.000 .244 .163 9.514 .519

Within

Subjects

time Pillai's

Trace

.381 4.312a 7.000 49.000 .001 .381 30.186 .980

time *

group

Pillai's

Trace

.528 7.841a 7.000 49.000 .000 .528 54.887 1.000

a. Exact statistic



Within Subjects Design: time

96

Figure 19 Pretest and Posttest Multivariate Tests by group

Multivariatec,d

Within Subjects Effect

Value F

Hypothesis

df

Error

df Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powerb

time Pillai's Trace .381 4.312a 7.000 49.000 .001 .381 30.186 .980

time *

group

Pillai's Trace .528 7.841a 7.000 49.000 .000 .528 54.887 1.000

a. Exact statistic



Within Subjects Design: time

d. Tests are based on averaged variables.

Table 53 Pre and Posttest Univariate tests by group

Univariate Tests

Source Measure

Type III Sum of

Squares df

Mean

Square F Sig.

Partial

Eta

Squared

Noncent.

Parameter

Observed

Powera

time scig 2.443 1 2.443 3.655 .061 .062 3.655 .468

selfg .048 1 .048 .144 .706 .003 .144 .066

moralg .076 1 .076 .314 .578 .006 .314 .085

violg .385 1 .385 .850 .360 .015 .850 .148

schoolg .001 1 .001 .003 .953 .000 .003 .050

knowg 2.265 1 2.265 11.010 .002 .167 11.010 .903

97

Univariate Tests

Source Measure

Type III Sum of

Squares df

Mean

Square F Sig.

Partial

Eta

Squared

Noncent.

Parameter

Observed

Powera

locusg 1.891 1 1.891 6.442 .014 .105 6.442 .703

time *

group

scig 7.739 1 7.739 11.578 .001 .174 11.578 .917

selfg .378 1 .378 1.131 .292 .020 1.131 .181

moralg 1.740 1 1.740 7.219 .010 .116 7.219 .752

violg 1.363 1 1.363 3.010 .088 .052 3.010 .399

schoolg .036 1 .036 .167 .685 .003 .167 .069

knowg 9.407 1 9.407 45.737 .000 .454 45.737 1.000

locusg .760 1 .760 2.590 .113 .045 2.590 .353

Error(time) scig 36.764 55 .668

selfg 18.394 55 .334

moralg 13.260 55 .241

violg 24.898 55 .453

schoolg 11.792 55 .214

knowg 11.312 55 .206

locusg 16.147 55 .294

98

Table 54 Within Subjects contrasts Pre and Posttest means by group


Source Measure time Type III

Sum of

Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

time

dimension1

scig Linear 2.443 1 2.443 3.655 .061 .062 3.655 .468

selfg Linear .048 1 .048 .144 .706 .003 .144 .066

moralg Linear .076 1 .076 .314 .578 .006 .314 .085

violg Linear .385 1 .385 .850 .360 .015 .850 .148

schoolg Linear .001 1 .001 .003 .953 .000 .003 .050

knowg Linear 2.265 1 2.265 11.010 .002 .167 11.010 .903

locusg Linear 1.891 1 1.891 6.442 .014 .105 6.442 .703

time *

group

dimension1

scig Linear 7.739 1 7.739 11.578 .001 .174 11.578 .917

selfg Linear .378 1 .378 1.131 .292 .020 1.131 .181

moralg Linear 1.740 1 1.740 7.219 .010 .116 7.219 .752

violg Linear 1.363 1 1.363 3.010 .088 .052 3.010 .399

schoolg Linear .036 1 .036 .167 .685 .003 .167 .069

knowg Linear 9.407 1 9.407 45.737 .000 .454 45.737 1.000

locusg Linear .760 1 .760 2.590 .113 .045 2.590 .353

Error(time)

dimension1

scig Linear 36.764 55 .668

selfg Linear 18.394 55 .334

moralg Linear 13.260 55 .241

violg Linear 24.898 55 .453

schoolg Linear 11.792 55 .214

99


Source Measure time Type III

Sum of

Squares df

Mean

Square F Sig.

Partial Eta

Squared

Noncent.

Parameter

Observed

Powera

knowg Linear 11.312 55 .206

locusg Linear 16.147 55 .294


Table 55 Pre and Posttest means by group

time

Measure time

Mean Std. Error

95% Confidence Interval

Lower Bound Upper Bound

dimension0

scig 1 5.183 .121 4.940 5.426

2 4.885 .161 4.563 5.207

selfg 1 5.096 .128 4.840 5.352

2 5.138 .112 4.913 5.363

moralg 1 5.472 .072 5.328 5.617

2 5.420 .099 5.222 5.618

violg 1 5.525 .098 5.328 5.721

2 5.406 .103 5.199 5.614

schoolg 1 5.408 .087 5.235 5.582

2 5.403 .085 5.233 5.573

knowg 1 4.051 .082 3.888 4.215

2 4.339 .074 4.190 4.487

100

locusg 1 4.866 .110 4.646 5.087

2 4.604 .125 4.353 4.854

Table 56 Posttest Attitudes for Treatment



Perception of Educator 51 4.36 6.00 5.6340 .39556

Animal Bonding 51 4.00 6.00 5.4627 .50595

Program enjoyment 47 3.00 6.00 5.3723 .76728

Reported understanding 49 4.25 6.00 5.3581 .46657


Bonding with Educator 50 4.00 6.00 5.5850 .53066

Educator Teaching Skills 50 4.38 6.00 5.6951 .37715

Attitude Environment 45 4.00 6.00 5.7667 .42699


Table 57 Student overall perception of program


studenover 52 4.66 6.00 5.5366 .33736


101

Summary

A statistically significant change in Attitude toward Science was seen in the treatment group

and not for the control group from pretesting to post testing. F(1,58)=12.25. p<.001 η2=.17. This

represents a medium effect size. In order to examine the longitudinal change, the science

attitudes of the treatment group were examined twice during the program as well as during

pretesting and posttesting. The treatment group exhibited a statistically significant positive

change in attitude toward science as the program progressed. F(3,9)=5.15. p<.01 η2=.15

exhibiting a medium effect size. This statistic uses only the treatment group, as the control group

does not take the Talking Talon’s quizzes. Students in the program also exhibited a statistically

significant change in self-reported anticipated grade in science. This change was seen in the

treatment group and not for the control group from pretesting to post testing F(1,57)=19.83.

p<.001 η2=.26 which is a large effect size.

A statistically significant change in Knowledge was seen in the treatment group and not for

the control group from pretesting to post testing. F(1,55)=11.01. p<.01 η2=.45. This represents a

very large effect size. A breakdown of the treatment group Knowledge scores on quizzes also

was provided.

102

A statistically significant change in Moral Attitude was seen in the treatment group and not

for the control group from pretesting to post testing. F(1,58)=8.00. p<.01 η2=.12 with a small to

medium effect size.

Classroom teacher feedback was extremely positive and both teachers indicated the program

increased student science knowledge and attitudes, that the buddy class was worthwhile and also

indicated a strong willingness to have the program again. Mean scores for all subscales

measuring teacher feedback were all above 6 on a 7 point scale (a higher number indicates more

positive feedback) with a mean of 6.44 and a standard deviation of .37. The classroom teachers

rate the program as most effective for average students, with effectiveness for above average and

delayed students slightly lower.

Qualitative feedback from students was overwhelmingly positive, with students mentioning

that the program was a positive experience, that they enjoyed presenting to the buddy class and

found it both rewarding and educational. Many expressed that they relished “being the teacher”

stating that “It’s like my grownup mode just turns on”. The students also were enthusiastic about

that both the hands on and presentation sections of the program and found them enjoyable and

educational. As a student stated during interview “Talking Talons is like unicorns, it’s

amazing!”. The only suggestion for improvements by the students was the inclusion of even

more animals (they would like a llama and turtles). Means scores for all student treatment

subscales were above 5.35 on a 6 point scale with a mean of 5.54 and a standard deviation of .34.

Even with a small sample size, and thus less statistical power, it is clear that the program

positivity impacted the participants. Science attitudes and knowledge are increased and

classroom teachers find the program effective and enjoyable for their students.

103

References

Al-Balushi, Sulaiman M, and Shamsa S Al-Aamri. “The Effect of Environmental Science

Projects on Students’ Environmental Knowledge and Science Attitudes.” International

Research in Geographical and Environmental Education 2014: 213–227. Web.

Aquino, Karl, and Americus Reed. “The Self-Importance of Moral Identity.” Journal of

personality and social psychology 83.6 (2002): 1423–1440. Web.

Arbuthnot, J, and D a Gordon. “Behavioral and Cognitive Effects of a Moral Reasoning

Development Intervention for High-Risk Behavior-Disordered Adolescents.” Journal of

consulting and clinical psychology 54.2 (1986): 208–216. Web.

Arkow, Phil. “Animals as Facilitators of Social Capital and Character Education.” Society for the

Study of Social Problems 2006. Web.

Backes, John S. “Bridging the Gender Gap: Self-Concept in the Middle Grades.” Schools in the

Middle 3.3 (1994): 19–23. Print.

Bakeman, Roger. “Recommended Effect Size Statistics for Repeated Measures Designs.”

Behavior research methods 37.3 (2005): 379–384. Web.

Baker, Dale R. “Predictive Value of Attitude, Cognitive Ability, and Personality to Science

Achievement in the Middle School.” Journal of Research in Science Teaching 22.2 (1985):

103–113. Print.

Berkowitz, Marvin W. et al. “Moral Reasoning and Judgments of Aggression.” Journal of

104

Personality and Social Psychology 51.4 (1986): 885–891. Web.

Bowman-Perrott, Lisa et al. “Academic Benefits of Peer Tutoring : A Meta-Analytic Review of

Single-Case Research.” School Psychology Review 42.1 (2013): 39–55. Print.

Catsambis, Sophia. “Gender, Race, Ethnicity, and Science Education in the Middle Grades.”

Journal of Research in Science Teaching 32.3 (1995): 243–257. Print.

Desy, Elizabeth A, Scott A Peterson, and Vicky Brockman. “Gender Differences in Science-

Related Attitudes and Interests among Middle School and High School Students.” Science

Educator 20.2 (2011): 23–30. Web.

Ellery Samuels, William, Kate Nicoll, and Cindy Trifone. “An In-Class, Humane Education

Program Can Improve Young Students’ Attitudes Toward Animals.” Society & Animals

16.1 (2008): 45–60. Web.

Fantuzzo, John W., Kenneth Polite, and Natalie Grayson. “An Evaluation of Reciprocal Peer

Tutoring across Elementary School Settings.” Journal of School Psychology 28.4 (1990):

309–323. Web.

Farenga, Stephen, and Beverly Joyce. “Science-Related Attitudes and Science Course

Selection...” Roeper Review 20 (1998): 247. Print.

Freedman, Michael P. “The Influence of Laboratory Instruction on Science Achievement and

Attitude toward Science across Gender Differences.” Journal of Women and Minorities in

Science and Engineering 8.2 (2002): 191–200. Web.

Galbraith, Jonathan, and Mark Winterbottom. “Peer‐tutoring: What’s in It for the Tutor?”

105

Educational Studies 37.3 (2011): 321–332. Web.

Glasson, George E. “The Effects of Hands-on and Teacher Demonstration Laboratory Methods

on Science Achievement in Relation to Reasoning Ability and Prior Knowledge.” Journal

of Research in Science Teaching 26.2 (1989): 121–131. Web.

Greenwood, Charles R et al. “Classwide Peer Tutoring Learning Management System:

Applications with Elementary-Level English Language Learners.” Remedial and Special

Education 22.1 (2001): 34–47. Print.

Guivernau, Marta, and Joan L. Duda. “Moral Atmosphere and Athletic Aggressive Tendencies in

Young Soccer Players.” Journal of Moral Education 31.1 (2002): 67–85. Web.

Gumpel, T P, and R Frank. “An Expansion of the Peer-Tutoring Paradigm: Cross-Age Peer

Tutoring of Social Skills among Socially Rejected Boys.” Journal of applied behavior

analysis 32.1 (1999): 115–118. Web.

Jarvis, Tina, and Anthony Pell. “Changes in Primary Boys’ and Girls' Attitudes to School and

Science during a Two-Year Science in-Service Programme.” The Curriculum Journal 13.1

(2002): 43–69. Print.

Kamps, D M et al. “Classwide Peer Tutoring: An Integration Strategy to Improve Reading Skills

and Promote Peer Interactions among Students with Autism and General Education Peers.”

Journal of applied behavior analysis 27.1 (1994): 49–61. Web.

Kanter, David E., and Spyros Konstantopoulos. “The Impact of a Project-Based Science

Curriculum on Minority Student Achievement, Attitudes, and Careers: The Effects of

106

Teacher Content and Pedagogical Content Knowledge and Inquiry-Based Practices.”

Science Education 94.5 (2010): 855–887. Web.

Kim, SeongSil, SinAe Park, and KiCheol Son. “Improving Peer Relations of Elementary School

Students through a School Gardening Program.” HortTechnology 24.2 (2014): 181–187.

Web.

Kimble, G. “Children Learning about Biodiversity at an Environment Centre, a Museum and at

Live Animal Shows.” Studies in Educational Evaluation 41 (2014): 48–57. Web.

Kose, Sacit et al. “The Effects of Cooperative Learning Experience on Eighth Grade Students’

Achievement and Attitude toward Science.” Education 131.1 (2010): 169–180. Print.

Kunsch, Catherine A, Asha K Jitendra, and Sheetal Sood. “The Effects of Peer-Mediated

Instruction in Mathematics for Students with Learning Problems: A Research Synthesis.”

Learning Disabilities Research & Practice 22.1 (2007): 1–12. Web.

Lee, Valerie E, and David T Burkam. “Gender Differences in Middle Grade Science

Achievement: Subject Domain, Ability Level, and Course Emphasis.” Science Education

80.6 (1996): 613–650. Print.

Maguin, Eugene, and Rolf Loeber. “Academic Performance and Delinquency.” Crime and

Justice 20 (1996): 145–264. Web.

Malti, Tina, and Tobias Krettenauer. “The Relation of Moral Emotion Attributions to Prosocial

and Antisocial Behavior: A Meta-Analysis.” Child development 84.2 (2013): 397–412.

Web.

107

Margadant-Van Arcken, Marjan. “Environmental Education, Children, and Animals.”

Anthrozoos 3:1 (1989): 14–19. Print.

Mattern, Nancy Page Garland. “Relationships between Attitudes toward and Achievement in

Science for Rural Middle School Students: Patterns across Gender.” Dissertation Abstracts

International Section A: Humanities & Social Sciences 60.12-A (2000): 4319. Print.

Mattern, Nancy, and Candace Schau. “Gender Differences in Science Attitude-Achievement

Relationships over Time among White Middle-School Students.” Journal of Research in

Science Teaching 39.4 (2002): 324–340. Web.

McMaster, Kristen L., Douglas Fuchs, and Lynn S. Fuchs. “Research on Peer-Assisted Learning

Strategies: The Promise and Limitations of Peer-Mediated Instruction.” Reading & Writing

Quarterly 22.1 (2006): 5–25. Web.

Milner, Andrea R. et al. “Elementary Teachers’ Beliefs About Teaching Science and Classroom

Practice: An Examination of Pre/Post NCLB Testing in Science.” Journal of Science

Teacher Education 23.2 (2012): 111–132. Web.

Oliver, Joseph S. “A Longitudinal Study of Attitude, Motivation and Self Concept as Predictors

of Achievement in and Commitment to Science among Adolescent Students.” Dissertation

Abstracts International 47.8-A (1987): 2983. Print.

Oliver, Steve J, and Ronald D Simpson. “Influence of Attitude toward Science, Achievement

Motivation, and Science Self Concept on Achievement in Science: A Longitudinal Study.”

Science Education 72.2 (1988): 143–155. Web.

108

Ornstein, Avi. “The Frequency of Hands-on Experimentation and Student Attitudes toward

Science: A Statistically Significant Relation.” Journal of Science Education and

Technology 15.3/4 (2006): 285–297. Web.

Papanastasiou, Elena C, and Michalinos Zembylas. “The Effect of Attitudes on Science

Achievement: A Study Conducted Among High School Pupils in Cyprus.” International

review of education 48.6 (2002): 469–484. Print.

Pigott, H E, J W Fantuzzo, and P W Clement. “The Effects of Reciprocal Peer Tutoring and

Group Contingencies on the Academic Performance of Elementary School Children.”

Journal of applied behavior analysis 19.1 (1986): 93–8. Web.

Raaijmakers, Quinten, Rutger Engels, and Anne Van Hoof. “Delinquency and Moral Reasoning

in Adolescence and Young Adulthood.” International Journal of Behavioral Development

29 (2005): 247–258. Web.

Reynolds, Arthur J, and Herbert J Walberg. “A Structural Model of Science Achievement and

Attitude: An Extension to High School.” Journal of Educational Psychology 84.3 (1992):

371–382. Print.

Reynolds, Scott J, and Tara L Ceranic. “The Effects of Moral Judgment and Moral Identity on

Moral Behavior: An Empirical Examination of the Moral Individual.” The Journal of

applied psychology 92.6 (2007): 1610–1624. Web.

Robinson, Debbie R., Janet Ward Schofield, and Katrina L. Steers-Wentzell. “Peer and Cross-

Age Tutoring in Math: Outcomes and Their Design Implications.” Educational Psychology

Review 17.4 (2005): 327–362. Web.

109

Rohrbeck, Cynthia a. et al. “Peer-Assisted Learning Interventions with Elementary School

Students: A Meta-Analytic Review.” Journal of Educational Psychology 95.2 (2003): 240–

257. Web.

Santos, Robert G et al. “Effectiveness of Peer-Based Healthy Living Lesson Plans on

Anthropometric Measures and Physical Activity in Elementary School Students: A Cluster

Randomized Trial.” JAMA pediatrics 168 (2014): 330–7. Web.

Satterthwait, Donna. “Why Are ‘Hands-on’ Science Activities so Effective for Student

Learning?” Teaching Science: The Journal of the Australian Science Teachers Association

56.2 (2010): 7–10. Web.

Slate, John R, and Craig H Jones. “Fourth and Fifth Grade Students’ Attitudes toward Science:

Science Motivation and Science Importance as a Function of Grade Level, Gender, and

Race.” Research in the Schools 5.1 (1998): 27–32. Print.

Smith, T. J., S. L. Pasero, and C. M. McKenna. “Gender Effects on Student Attitude Toward

Science.” Bulletin of Science, Technology & Society 34.1-2 (2014): 7–12. Web.

Sorge, Carmen. “The Relationship Between Bonding with Nonhuman Animals and Students’

Attitudes Toward Science.” Society & Animals 16 (2008): 171–184. Web.

Stohr-Hunt, Patricia M. “An Analysis of Frequency of Hands-on Experience and Science

Achievement.” Journal of Research in Science Teaching 33.1 (1996): 101–109. Web.

Tangney, June Price, Jeff Stuewig, and Debra J Mashek. “Moral Emotions and Moral Behavior.”

Annual review of psychology 58 (2007): 345–372. Web.

110

Tarry, Hammond, and Nicholas Emler^. “Attitudes, Values and Moral Reasoning as Predictors

of Delinquency.” British Journal of Developmental Psychology 25 (2007): 169–183. Web.

Teshome, Yalem, Nancy Maushak, and Krishna Athreya. “Attitude toward Informal Science and

Math: A Survey of Boys and Girls Participating in Hands-On Science and Math

(FUNTIVITIES).” Journal of Women and Minorities in Science and Engineering 7.1

(2001): 59–74. Web.

Trainin, Guy et al. “Extraordinary Animals and Expository Writing: Zoo in the Classroom.”

Journal of Science Education and Technology 14.3 (2005): 299–304. Web.

Warburton, S J, W L Jenkins, and P Coxhead. “‘Science Achievement and Attitudes’ and the

Age of Transfer to Secondary School.” Educational Research 25.3 (1983): 177–183. Print.

Willson, Victor L. “A Meta-Analysis of the Relationship between Science Achievement and

Science Attitude: Kindergarten through College.” Journal of Research in Science Teaching

20.9 (1983): 839–850. Print.

Wilson, Christopher D. et al. “The Relative Effects and Equity of Inquiry-Based and

Commonplace Science Teaching on Students’ Knowledge, Reasoning, and Argumentation.”

Journal of Research in Science Teaching 47.3 (2010): 276–301. Web.

Wilson, K. et al. “Our Zoo To You: The Link between Zoo Animals in the Classroom and

Science and Literacy Concepts in First-Grade Journal Writing.” Journal of Early Childhood

Literacy 11.3 (2011): 275–306. Web.

Zasloff, R L et al. “Animals in Elementary School Education in California.” Journal of Applied

111

Animal Welfare Science 2.4 (1999): 347–357. Web.

talking talons youth leadership evaluation report · talking talons youth leadership evaluation...

Documents