first robotics as a model for experiential problem-based learning: a comparison of student attitudes...
TRANSCRIPT
FIRST ROBOTICS AS A MODEL
FOR EXPERIENTIAL
PROBLEM-BASED LEARNING:
A COMPARISON OF STUDENT
ATTITUDES AND INTERESTS IN
SCIENCE, MATHEMATICS,
ENGINEERING, AND
TECHNOLOGYDonald Sanford Griffith, Jr.
May 2005
INTRODUCTIONSignificance of the Study
• As reported by the Third International Mathematics and Science Study (TIMSS), students in the United States are lagging behind other nations in the areas of mathematics and science (U.S. Department of Education, 1999).
• Undergraduate admissions requirements for engineering schools require students to have a solid background in mathematics and science (U.S. Department of Labor, 2002).
• United States Department of Labor statistics also indicate that there will be the need for 20 percent more engineers by the end of this decade (Southern Regional Education Board, 2001).
INTRODUCTIONSignificance of the Study
• The number of students who plan to major in engineering upon college entrance has continued to decrease, ACT Policy Report (Noeth, Cruce & Harmston, 2003).
• Students enrolled in engineering programs hit a 17-year low in 1999, according to the National Society of Professional Engineers.
• Enrollment in engineering technology programs in 2000 was about half what it was in 1990 (Southern Regional Education Board, 2001).
Statement of the Problem
The problem addressed in this study was whether student
participation in the six-week challenge of the FIRST
Robotics Competition had an effect on high school
students’ attitudes and interest in science, mathematics,
engineering, and technology. The possible relationships
between the six-week challenge of the FIRST Robotics
Competition and high school students’ attitudes and
interest in science, mathematics, engineering, and
technology is uncertain and may be important to the
overall success of students in high school courses, post-
secondary education and related careers.
Purpose of the Study
The purpose of this study was to investigate whether
student participation in the six-week challenge of the FIRST
Robotics Competition, positively changed high school
students’ attitudes and interest in science, mathematics,
engineering, and technology after their participation. It
was anticipated that gender and race of the high school
students, in the context of the FIRST Robotics experience,
would influence attitudes and interest in science,
mathematics, engineering and technology in either a
positive or negative manner.
Research Questions
Primary Questions1. Do high school students that participate in the six-
week challenge of the FIRST Robotics Competition have more positive changes in their attitudes regarding science, mathematics, engineering, and technology after their participation than students who do not participate in the program?
1. Do high school students that participate in the six-week challenge of the FIRST Robotics Competition have more interest in science, mathematics, engineering, and technology after their participation than students who do not participate in the program?
Subsidiary Questions
1. Does the gender of high school students have an effect on their interest in science, mathematics, engineering, and technology?
1. Does the race of high school students have an effect on their interest in science, mathematics, engineering, and technology?
Null Hypothesis
1. There is no difference in the attitudes of high school students that participate in the six-week challenge of the FIRST Robotics Competition regarding science, mathematics, engineering, and technology after their participation, than students who do not participate in the program.
1. There is no difference in the interest level of high school students that participate in the six-week challenge of the FIRST Robotics Competition regarding science, mathematics, engineering, and technology for students after their participation, than students that do not participate in the program.
Assumptions
For this study the following was assumed:
1. high school students participating in the research study would provide accurate and valid responses on the pre- and post- survey instruments.
2. the variation of science, mathematics, engineering, and technology teachers students have experienced, and courses students have completed was equally distributed in the population.
3. the variation of science, mathematics, engineering, and technology extra- curricular activities students have experienced were equally distributed in the population.
4. the variation of students with previous FIRST Robotics Competition experience were equally distributed in the experimental group sample.
Literature Review
FIRST has participated in 7 research studies since 1996
seeking to answer the question, “Is the program
working?” In each study, data were collected from
participants involved in FIRST. In each of these studies,
FIRST students were not compared to other student
populations; therefore there was not conclusive evidence
that the change in students’ attitudes or interests was due
to their participation in FIRST.
Literature Review
Educational philosophies, theories, and methodologies
supporting the FIRST Robotics Competition were
considered as the rationale for identifying topics of inquiry
in this review. The topics of inquiry identified were as
follows: philosophies of education, experiential learning,
constructivism, situated cognition, problem-based
learning, and project-based learning within the context of
high school students’ attitudes and interests in science,
mathematics, engineering, and technology.
Research Locations
• High school sites were located in urban and rural locations throughout the state of South Carolina.
• High schools sites had students participating as team members with a FIRST Robotics team that was registered to participate in the 2005 FIRST Robotics Competition.
• High schools and FIRST Robotics teams selected had participated in the 2004 FIRST Robotics Competition.
343
415
845
342
1026
804
1319
1051
1102
1320
1398 128
7
Distribution of FIRST Robotics Teams Participating in the Study
Research Locations
During the six-week period of the study, students
participating in FIRST Robotics were involved designing,
developing, and producing a robot for the 2005 FIRST
Robotics Competition while students at the sponsoring
high schools were enrolled in a science, mathematics,
engineering, or technology course.
Sampling
• A convenience sample was drawn from the assessable population of 2005 FIRST Robotics teams and sponsoring high schools in the state for this study.
• Students selected were attending public high schools during the 2004-2005 school year.
• Students participating in the study represented twelve of the nineteen 2004 FIRST Robotics teams and twenty of the thirty- five sponsoring high schools.
• The twelve FIRST Robotics teams had a total of one hundred eighty- three participating students.
• The sponsoring high schools had a total of five hundred fifty participating students.
Instrumentation
• Copied segments of the What Do YOU Think? survey designed by the Goodman Research Group in 2000 for FIRST, to investigate attitudes and interest in science, mathematics, engineering, and technology of the study participants.
• Administered pre- and post-surveys to participants based conclusions and recommendations included in the Goodman Research Group 2000 Final Report To FIRST :
“…any data being collected from students as part of a program evaluation be accomplished via written mail surveys not web- based surveys.”
“…future research in the direction of tracking students over time and comparing them to students who do not participate in the program.”
• Administered the What Do YOU Think? pre- and post-surveys using Optical Mark Reader (OMR) forms to record participant responses.
Procedures
• Public high schools in a southeastern state identified.
• Letters of support received.
• Procedures for data collection detailed.
• Approval to conduct study received from Clemson University IRB.
• Pilot study performed.
• Study pre- and post- surveys administered.
• Statistical procedures performed.
• Findings compared to findings reported by the Goodman Research Group, Inc. 2000 Final Report to FIRST.
Statistical Procedures(OMR OS/390 Data Set Analysis: 9.1 SAS Software©)
•the frequency for responses to each question was examined.
•chi-square procedure was used to examine the distribution patterns of the independent demographic characteristics.
•ANOVA was conducted to determine if differences existed between the pre- and post-survey responses to each question within each group, and if differences existed between these two groups.
•ANOVA was conducted to determine if differences existed between the pre- and post-survey response averages within each group, and if differences existed between these two groups.
Microsoft Word Document
Participants
Group Pre-survey Post Survey Loss Participant %
Experimental 181/183 131/132 (52) 72.38(FIRST)
Control 546/550 373/375 (177) 68.32(SMET)
Totals 727/733 504/507 (229) 70.35
Microsoft Word Document
Gender and Race Pre-survey Post-survey
GenderPercentages
Group Female Male Female MaleExperimental 24 76 30 70(FIRST)Control 37 63 40 60 (SMET)
RacePercentages
Minority White Minority White
Experimental 23 77 30 70(FIRST)Control 31 69 33 66 (SMET)
Microsoft Word Document
Microsoft Word Document
Survey Instrument
7 Attitudinal Categoriesteamwork
science and mathematics knowledge of science and math careers,
persist in science, mathematics, engineering, and technology enjoyment of high school science and math classes
the working world self-image
3 Interest Categoriescareer field interests
students’ expected activities for the year after high school graduation
students’ highest level of education
Survey Response SelectionFour items are based on a 5 point Likert type scale from (1) a lot to (5) doesn’t apply (to the item). Twenty-three items of the sixty survey items are based on a 5 point Likert type scale from (1) strongly agree to (5) strongly disagree (with the item).
A B C D E
Strongly Agree Neither Agree Disagree Strong Agree nor Disagree Disagree 14. I enjoy participating in team- 1 2 3 4 5 orientated activities.Ten items are based on a 4 point Likert type scale from (1) very likely to(4) very unlikely (with the item). Two items are based on a 4 point Likert type scale from (1) a lot to (4) nothing (with the item).
A B C D
A lot Some A little Nothing
33. Science? 1 2 3 4
The remaining twenty-one items required the student to select one from multiple choices.
Microsoft Word Document
Research Question 1
Do high school students that participate in the six-week challenge of the FIRST Robotics Competition have more
positive changes in their attitudes regarding science, mathematics, engineering, and technology after their
participation than students who do not participate in the program?
A one-way analysis of variance (ANOVA) was conducted to examine differences between pre- and post-survey dependent variables within the experimental group (FIRST), and the control group (SMET).
A two-way ANOVA was conducted to examine findings specific to the participants by comparing between the FIRST and SMET groups.
Test statistic used to determine level of significance for each dependent variable in experimental and control group ( p-value α=.05 ).
Microsoft Word Document
One-way ANOVAExamine differences between pre- and post-survey
dependent variables within each group
Dependent variable 54: Managing your time effectively is an important part of being successful in your job. Source df Sum of Squares Mean Square F Value Pr > FModel 1 4.5989011 4.5989011 7.54 0.0062Error 649 395.7820513 0.6098337
Corrected Total 650 400.3809524
Two-way ANOVA
Examine differences between pre- and post-survey dependent variables between each group
Control * Test Group (SMET 546 / FIRST 181)
Examine category means, pre- and post-survey question averages in each group
FIRST Group Category 6: Statements about the working world.
Source df Sum of Squares Mean Square F Value Pr > FModel 1 1.6037269 1.6037269 3.67 0.0557Error 648 282.8072133 0.4364309Uncorrected Total 649 284.4109402
Research Question 2
Do high school students that participate in the six-week challenge of the FIRST Robotics Competition have more
interest in science, mathematics, engineering, and technology after their participation than students who do
not participate in the program?
Response frequencies for the experimental group (FIRST) and control group (SMET), pre- and post survey responses were explored in order to address research question 2.
The response frequency for each question on the pre- and post-survey was calculated as a percentage for each group.
Increases in career field interests between pre-survey and post-survey, FIRST experimental group, reported by gender.
Percentage
Field Gender Gender Percentage Boy Girl Pre-Survey Boy Girl Post-survey Change
Architecture 6.47 1.90 8.37 8.46 4.76
13.22 +4.85
Business, Finance 1.67 2.53 4.20 3.08 5.56
8.64 +4.44
Math 12.32 4.43 16.75 10.38 9.52
9.90 +3.15
Engineering 18.37 10.76 29.13 22.31 7.94 30.25
+1.12
Decreases in career field interests between pre-survey and post-survey, SMET control group, reported by gender.
Percentage
Field Gender Gender Percentage Boy Girl Pre-Survey Boy Girl Post-survey Change
Architecture 6.80 2.55 9.35 6.71 1.72
8.43 -0.92
Business, Finance 3.80 6.75 10.55 4.52 5.15
9.67 -0.88
Math 8.26 5.10 13.36 8.63 2.94
11.57 -1.79
Engineering 11.34 5.10 16.44 11.37 4.66
16.03 -0.41
Increases in career field interests between pre-survey and post-survey Gender differences of the SMET group
Percentage
Field Gender Gender Percentage Boy Girl Pre-Survey Boy Girl Post-survey Change
Arts, Design 4.45 6.75 11.20 5.07 10.05 15.12 +3.92Social Science 1.38 2.70 4.08 1.78 4.90 6.68 +2.60Medicine 2.27 6.75 9.02 3.70 7.84 11.54 +2.52Law 3.08 5.40 8.48 3.84 7.11 10.95 +2.47
Comparison of FIRST and SMET post-survey career field interests between students, positive difference of interests for FIRST students. Gender differences are reported for each group.
Percentage
Field Gender FIRST Gender SMET Percentage Boy Girl Post-Survey Boy Girl Post-survey Change
Engineering 22.31 7.94 30.25 11.37 4.66 16.03
+14.22
Math 10.38 9.52 19.90 8.63 2.94 11.57
+ 8.33
Computers 13.08 9.52 22.60 9.04 7.35 16.39
+ 6.21
Architecture 8.46 4.76 13.22 6.71 1.72 8.43
+ 4.79
Science 4.62 7.14 12.03 3.97 6.37
10.37 + 1.69
Comparison of the FIRST and SMET groups, pre-survey and post-surveyinterest in attending a 4-year college, service academy, or university after leaving high school. Gender differences are reported for each group.
Percentage
Pre-Survey Post-survey Net ChangeGroup Gender Gender Gender Boy Girl Boy Girl Boy Girl
FIRST 50.47 51.35 59.06 61.02
+8.59 +9.67
SMET 31.14 44.62 32.78 45.50
+1.64 +0.88
Comparison of the FIRST and SMET groups, pre-survey and post-survey interest in attending a 2-year junior or community college after leaving high school. Gender differences are reported for each group.
Percentage
Pre-Survey Post-survey Net Change Group Gender Gender Gender Boy Girl Boy Girl Boy Girl
FIRST 14.62 25.68 18.11 23.73
+3.49 -1.95
SMET 18.32 20.25 17.78 18.48
-0.54 -1.77
Although there was a decrease of 1.95% indicated by girls in the FIRST group regarding their interest to attend a 2-year junior or community college after leaving high school there was an increase of 9.67% by girls in the FIRST group between pre- and post-survey responses, regarding their expectation to be attend a 4-year college, service academy, or university.
Comparison of the FIRST and SMET groups, post-survey responses, students’ highest expected level of education. Gender differences are reported for each group.
Percentage Post-survey
Expectation Group Gender Total Boy Girl
4-year college degree FIRST 19.55 20.34 39.89 SMET 27.54 18.69 46.23 Master degree FIRST 22.56 22.03 44.59 SMET 13.37 19.63 33.00
Ph.D. or M.D. FIRST 12.78 13.56 26.34 SMET 6.42 10.28 16.70
Comparison of the FIRST and SMET groups, post-survey responses, students’ expectation of a 4-year college degree. Gender differences are reported for each group.
Percentage
Expectation: Race Gender 4-year college degree White Minority Boy Girl Pre Post Pre Post Pre Post Pre Post
FIRST 17.41 13.59 6.30 6.52 22.22
19.55 16.44 20.34
SMET 6.29 16.05 5.72 7.77 22.95
27.54 18.27 18.69
Null Hypothesis 1
There is no difference in the attitudes of high school students that participate in the six-week challenge of the FIRST Robotics
Competition regarding science, mathematics, engineering, and technology after their participation, than students who do not
participate in the program.
The null hypothesis concerning attitude differences of high school students that participate in the FIRST Robotics six-week challenge as compared to students that do not participate in the program was not rejected.
High pre-survey dependent variable scores provided by students in the FIRST group did not allow for significant gain in each of the seven-attitudinal categories.
Findings indicated that there were significant attitudinal differences between students in the experimental group (FIRST), and students the control group (SMET).
Students in the FIRST group had statistically significant higher attitude means than students in the SMET group on both pre- and post-survey’s in the seven-attitudinal categories.
Null Hypothesis 2
There is no difference in the interest level of high school students that participate in the six-week challenge of the FIRST Robotics Competition regarding science, mathematics, engineering, and technology for students after their participation, than students
that do not participate in the program.
The null hypothesis concerning interest differences of high school students that participate in the FIRST Robotics six-week challenge as compared to students that do not participate in the program was rejected.
The frequency for responses to each question in the three interest categories of the pre- and post-survey were calculated for the experimental and control group to evaluate differences.
The results were expressed in percentages and indicated that there were significant differences in the respondent scores for the pre-survey versus the post-survey in the FIRST group.
Subsidiary Question 1 and 2
Does the gender of high school students have an effect on their interest in science, mathematics, engineering, and
technology?
Does the race of high school students have an effect on their interest in science, mathematics, engineering, and
technology?The chi-square procedure was used to examine the distribution patterns of the independent demographic
characteristics along the dependent variables.
Descriptive analysis was conducted for the data on race and gender. Subsidiary questions 1 and 2 were addresses
within the independent samples t test analysis demonstrating no significant differences on pre-survey or
post-survey scores by either gender or race.
The Importance of This Study
The findings of this study suggest that FIRST students’ have, and continue to have, high positive attitudes and
interests in science, mathematics, engineering, and technology.
Positive attitudes and interest in these subjects lead to higher academic achievement.
The consequences are that students tend to enroll in advanced courses and pursue careers related to science,
mathematics, engineering, and technology.
Enjoyment Science and Math
The post-survey reported increases in enjoyment of science and mathematics for students in the experimental group
(FIRST).
The post-survey indicated that more than 79% of the FIRST respondents enjoyed science, and 81% enjoyed math a lot,
compared to 54% and 59% respectively reported by students in the control group.
Develop Interest in Science and Math
FIRST respondents, 84% were either likely or very likely to have plans for taking advanced high school or college level
classes in each of the four fields.
Students indicated the greatest likelihood of taking college level classes in science, mathematics, and computers.
In contrast, the control group respondents reported on the pre- and post-survey that 63% were either likely or very likely to have plans for taking advanced high school or
college level classes in each of the four fields.
Immediately After High School
The post-survey reported that over 60% of the students in the FIRST group planned on attending a four-year college,
service academy, or university immediately after graduating high school, an increase of
9% from the pre-survey.
These post-survey results are compared to 39% for the SMET group of students, an increase of 1.5%.
Over 60% of FIRST students responded on the post-survey, that over the course of their educational careers, they would earn at least a Bachelor’s degree, 22%
expected a Master’s degree, and 21% an advanced professional or terminal degree.
Educational Expectations
The study indicated that there was a difference in white and minority students’ expectations about their highest
level of education.
Minority students in both groups indicated on the pre- and post-surveys that they were less likely than white students
to expect a 4-year college degree.
The same trend was observed regarding girls with one exception;
girls in the FIRST group responded on the post-survey that they were more likely than boys to expect a 4-year college
degree after participating in the FIRST Robotics Competition six-week challenge
experience.
Consistent With Previously Reported Findings
Study findings that were consistent with findings reported in the Goodman Research Group’s Final Report to FIRST
(Goodman Research Group, 2000)
Differences in study findings were reported in two areas.
students’ perceived knowledge of science and mathematics careers, and students’ intentions to persist in science, mathematics engineering, and technology courses
and related careers.
Summary
Results of the study offer some evidence that the FIRST Robotics Competition six-week challenge experience has
short-term attitudinal and interest impact on students.
Students in the FIRST group:
reported high positive attitudes regarding science, mathematics, teamwork, and the working world.
were more interested and felt positive about their abilities, in science and mathematics compared to students in the control group.
attitudes were more positive regarding teamwork and the world of work compared to students in the control group.
likely to have plans for taking advanced high school or college level classes in science, mathematics, engineering, technology.
Recommendations for Future Study
Investigate the long-term outcomes of student participation in FIRST Robotics.
An investigation of student participation in FIRST Robotics Competition regional and championship events. FIRST Robotics Competition event experiences involve students in the performance phase of the competition.
Expanding the study to include FIRST Robotics Competition student participants enrolled in other than public high schools, e.g., private, parochial, charter, etc., may provide very useful insights.
Follow-up studies in various geographic areas of the country may provide additional information and prove useful in forming the basis for future inquiry regarding the study variables.