shane wheeler final project 1

8/3/2019 Shane Wheeler Final Project 1

1/20

Shane WheelerEDTech 505Section 4172Professor PerkinAugust 1st, 2011

08Fall

An Evaluation Plan for an iPadEntrepreneur Programming Class


2/20

Table of ContentsLEARNING REFLECTION 3

EXECUTIVE SUMMARY 4

PURPOSES OF EVALUATION 5

Q UESTIONS TO A DDRESS 5

S TAKEHOLDERS 5

BACKGROUND INFORMATION 6

O RIGINS 6

P RIOR P ROGRAMS 6

P EOPLE INVOLVED IN THE P ROGRAM 6C HARACTERISTICS 7

G OALS 7

OBJECTIVE 1 7

OBJECTIVE 2 7

OBJECTIVE 3 8

OBJECTIVE 4 8

DESCRIPTION OF EVALUATION DESIGN 8

CONCLUSIONS 9

SOURCES 10

APPENDIX A: EVALUATOR PROGRAM DESCRIPTION 11

APPENDIX B: EVALUATOR PROGRAM TIMELINE 12

APPENDIX C: BEGINNING OF THE YEAR STUDENT SURVEY 14

APPENDIX D: END OF THE YEAR STUDENT SURVEY 17

APPENDIX E: HARMONY STUDENT PORTFOLIO RUBRIC 19

APPENDIX F: INTERVIEW QUESTIONS FOR TEACHER 20

APPENDIX G: INTERVIEW QUESTIONS FOR STUDENTS 21

2


3/20

Learning Reflection

EDTech 505 has taught me a great many things about evaluation.Evaluation is used to make decisions, compare items objectively, refine processes,

or eliminate processes. Evaluation is not research and research is not evaluation.Research is stricter in its application. The parameters that need to be set up forresearch concern keeping the outside world from affecting variables in theprogram, where evaluation is not so much concerned with outside influence.Evaluation is more concerned with the key questions and decisions regarding theprogram. Evaluation is more practical in the real world application of improvingeducation at the level I teach at because conducting research at my level is notpractical for a working educator. Im not discounting research, because it is vitaland not nearly read or debated enough by educators. Im actually surprised by how much we try to change our systems all the time, but there is little real debateon the research or evaluations about research.

I have learned the proper steps for conducting an evaluation, but it seemsthat every evaluation is going to be unique given that each evaluation has its ownunique goals, processes, funding, and variables. I found reading the differentevaluations interesting, because they read so differently. While there is a setprocess, the authors voice is present in the material.

The last thing that I hope to take away from the process of taking EDTech505 is a good, solid core evaluation process for my grant program. It is going toprobably be more than what the Iowa School Board Foundation is going toexpect, but I personally am more interested in evaluating the success and failuresof the program. I look forward to the feedback I will receive of my evaluationsetup, but Im also nervous because my project will be different from everyoneelses.

Having worked on this final project, I have come to realize that I shouldhave sought more clarity on how to change the final project (totally my fault). Irealized too late that I have research, but feels clumsy to inject into the project. Ialso realize that it is impossible to be objective in a program evaluation if your arethe major figure that needs to be evaluated.

3


4/20

Executive Summary

This report is a justification of the evaluation plan for the iPadEntrepreneur Programming Class (IEPC) during the 2011-2012 school year.Since the program cannot be evaluated during the time frame of summer session

of EDTech 505, my report is a defense of my chosen methods for evaluation. As a requirement of receiving the grant for Qwest and the Iowa School

Board Foundation, I have to report the outcomes of the program to the IowaSchool Board Foundation by the end of May of 2012. The grant was received based on the four guiding principles:

Improve student learning through the innovative use of mobiletechnology Enhance the development of students higher order thinking andproblem solving skills Increase educators collaborative study of their content, practice,and student effects Increase educators understanding of best practices related tointegration of technology for improving student learning.

The evaluation will follow the goal-based model (Boulmetis & Dutwin,2005). The chosen methods for data collection will be surveys and interviews with students in the class, co-teachers of the class, and administration. A checklist of student accomplishments on a pass/fail basis is also one of datasources. The surveys will be administrated through Google Docs, and both thecurriculum director and myself will conduct the interviews. Student portfolios will be collected and analyzed by the curriculum director based on apredetermined rubric.

Based on the results of the evaluation, the class will be refined to best meet

the intended goals. The results for the evaluation will be made public in the IowaSchool Foundation newsletter in May and also published in the Cardinalnewsletter.

4


5/20

Purposes of Evaluation

The reason for this evaluation is to fulfill the requirements of the QwestTechnology grant as judged by the Iowa School Board Foundation (ISBF). Thegrant requires that author of the grant report the results they found during the

first year of the program. The evaluation is to hold the district, administration,and author accountable for fulfilling the grant, and also the research branch of ISBF are interested at looking at the results of the program.

Questions to Address

After having conferred with Mary Delagardelle, the executive director of ISBF,and discussing the goals of the grant and of the course as it relates to the Earlhamcurriculum, the following questions are the main area of concern for both theISBF and the author of the grant:

Has student learning improved through the innovative use of mobiletechnology? Have students higher order thinking and problem solving skills improved

because of this course? How has the co-teachers collaborative study of their content, practice, and

student effects been affected? How has understanding of best practices related to integration of

technology for improving student learning changed for the co-teachers?

Stakeholders

The students are the more invested stakeholders of this program because they will be receiving the instruction in a newly designed class by instructors who arepracticing a new style of instruction. Future students of the class will beimpacted because through the evaluation the class should see improvement in thedelivery of instruction and content. The co-teachers are the second mostimpacted stakeholders in this new program. This is a departure into a new content area, and a new style of delivery for both teachers. The administration isalso affected by this program because they are responsible for helping evaluatethe program, but more importantly they are also responsible for the learning of the district which the course takes place in. They too are responsibility to makesure the program is carried out in a way that benefits the students who take the

class.

5


6/20

Background Information

Origins

In the fall of 2010, I applied for a 2011 Teachers and Technology GrantProgram. After being one of 6 accepted grants, the program received funding inMay of 2011. The Earlham School Board then approved the addition of the new class called Entrepreneur Programming into the business section of thecurriculum.

The offering of the new class in business marketing section of thecurriculum in the high school that will focus on creating, marketing, and testingapplications for the Apple iPad 2. The class is an attempt to start to bring back more computer-based programming classes, which had been eliminated the year before within the high school because of downsizing. While the class deals with both marketing and programming content, programming will take the majority of content instruction within the program.

Prior Programs

While Earlham Community School District has not had a previousprogram like this in the past, the idea for the program came to me when I wasreading about research of programs that are similar in nature to this one or aboutthe use of iPads. Numerous articles point to the future of hand held devicerevolutionizing the world student live in (Dale, 2008; Johnson, 2005; TheConsortium for School Networking, 2006). The research on the use of show thatthe technology itself makes now difference in learning, but shows that it canreduce barriers to education (Vesisenaho et al.; Hussain & Aslam, 2009). The

most interesting research came from Crispin Dales article iPods and Creativityin Learning and Teaching: An Instructional Perpective. In the article statesthat while using mobile devices is risky there are perceived benefits in thegrowth of creativity in the students work. Most research shows that technology with not cause growth in students learning, so the improvement must come fromthe changes in instruction of this new program.

People Involved in the Program

The program will be co-taught by the business teacher, Mr. Rogers, andmyself (Shane Wheeler). It will be a yearlong course and will meet during one43-minute class period a day. A curriculum will be developed based on the IowaCores 21st Century Skills and the International Society for Technology inEducation Standards. The curriculum director, Kristin Sheffield, will oversee thedevelopment by the two co-teachers, and will play a key role in evaluating theclass as an outside observer. The principle, Jean Fletcher was responsible forchanging teacher schedule and the master class schedule to make room for thenew class. The superintendent, Michael Wright, was responsible for approving

6


7/20

the purchases made with the grant money and any additional money that isauthorized to the program.

Fifteen students will make up the first class of students to take the class.The students range in age from 14 to 17 year of age. The students need to havetaken a math class of Algebra 1 or higher to be accepted into the class.

Characteristics

The students from the entrepreneurial/programming class will use the iPadsto replace traditional textbooks with the enhanced offerings of the e-reader withInternet access, which will serve as a platform to create original content related toclass. The class will use iPhone Programming: The Big Nerd Ranch Guide by JoeConway and Aaron Hillegass, and The Business of iPhone App Development:Making and Marketing Apps that Succeed by Dave Wooldridge and MichaelSchneider as e-textbooks.

Students will learn C+ language. In addition, students will create applicationsfor the school district, such as a school district calendar application or simplelearning games for elementary students. Teachers opinions will be sought onapplications that would best address the learning needs of their students.

For the summation of the course, students will be required to have for sale atleast one application of his/her own original design. To develop applications foriPads, iPhones, and iPods, the students will be grouped in teams of three that best balance the skill sets needed to take their products to completion.

Goals

Objective 1: Improve student learning through the innovative use of mobiletechnology

The grant was rewarded based on trying something new and different forthe school district. The Earlham community School District will not be the firsthigh school to teach application programming for mobile devices, but it is new forthe school. Does these new class and change instruction improve learning?

Objective 2: Enhance the development of students higher order thinking andproblem solving skills

The idea of the program is to face students with the real life problem of marketing, programming, testing, and getting people to download an app from Apple iTunes store. Do these tasks required by students to accomplish this forcethem to develop higher order thinking skills and know how to face complexproblems?

Objective 3: Increase educators collaborative study of their content, practice,and student effects

7


8/20

The grant requires that collaboration between teachers be a big point of concern. Does the creation of this course cause the co-teachers to collaboratemore? Does this collaboration benefit their ability to understand their contentand instructional delivery? What effect does their collaboration have on the

students learning?Objective 4: Increase educators understanding of best practices related tointegration of technology for improving student learning.

The class will use technology, but does the integration of technology enhance the delivery of instruction? Does the class help the co-teachers developa better understanding of how best to integrate technology to enhance and not just because it is there?

Description of Evaluation DesignThe design used for this evaluation will be a goal-based model

(Boulmetis & Dutwin, 2005). The goals for this evaluation are quitestraightforward and laid out by the ISBF and the grant application process. TheEarlham Community School administration and co-teachers have all signeddocuments to fulfill and carry out the intentions of the goals. The goals areaddressed in the Background Information under the subsection on Goals. While the goals are straightforward in their approach, the design of methods andmeasurements of these goals are more difficult given the limited number of staff responsible for the program and the fact that the main evaluator of the program

is also the person most responsible for the program. Certain features need to bedesigned into the evaluation to make sure objectivity is maintained as much aspossible.

This evaluation is both qualitative and quantitative in its collection of data.The quantitative portion will be a pre and post survey given to students in theclass. The qualitative portion is a pre and post interview with both students andco-teachers and an evaluation of student portfolios using a rubric.

Two surveys were designed to address the use of mobile technology of students daily life and for educational purposes. The first survey will be given tostudents at the beginning of the school year, with the second survey with similarquestions given at the end of the year. The survey utilizes the Likert scale (never,

rarely, sometimes, most of time, all the time) and open-end questions.The surveys are intended to track the changes in use of mobile devices foreducational purposes and compare changes in regular daily use. If there are nochanges in regular daily use, but changes in educational use, a correlation couldpossibly be drawn from the data to indicate the impact the program is having onhow students use mobile devices for education. The survey is not designed tomeasure the improvement of learning by using mobile devices inside and outsideof class, but to monitor habitual changes in student use.

8


9/20

The curriculum director at the beginning and the end of the school year will interview students. Because the instructor is also responsible for theprogram and evaluation, it is in the best interest of the program that an outsideperson conducts interviews for the program. This person is also able to observethe classroom on a regular basis to further collect data on the overriding

objectives of the program. The curriculum director will ask open-endedquestions to students about how they perceive the class, their ability to deal withcomplex problems, how well they deal with higher order thinking processes, andquestions regarding how the different instructional strategies the co-teacher uses benefits or hinders their learning.

The curriculum director will also interview both co-teachers and ask thema series of open-ended questions with prompts. The questions will be focused onthe topic of their experiences teaching, learning new content, developing co-teaching instruction strategies, and integrating technology into the program for best practice.

The curriculum director will also rate the students portfolios of work forthe school based on a rubric used by the Harmon High School of the Kansas City School District. The curriculum director will analyze work form the beginning of the school year to that of the end of the school year using a rubric, and report outthe results.

The curriculum director will also be responsible for observing every classroom once a week within the district. The curriculum director has acceptedthe responsibility of reporting back observations of the class to the leader of theprogram to include in the report.

Conclusions

There is no way to be completely objective if Im in charge of every evaluative measure. I have enlisted the help of the curriculum director, in aneffort to increase objectivity into the evaluation. I would also recommend theprincipal be more involved, but that is not realistic in this case because he has just been introduced to the school district 10 days before the start of the year andknows very little about the district, let alone one specific class. He has too muchon his plate to be involved in this process. It hard not to think about how toanswer the questions that you have written about the evaluation, because youstart to address your own concerns about the program. Thus, you have already basis the evaluation. I believe now that the best course of action would have been to adopt a goal-free model, and had the principal and curriculum directordo all the observations, not knowing the goals of the grant.

9


10/20

Sources

Dale, C. (2008). IPods and creativity in learning and teaching: aninstructional perspective. International Journal of Teaching and Learning in Higher Education , 20 (1), 1-9.

Johnson, D. (2005, October). A vision for the net generation mediacenter. International Society for Technology and Education .Retrieved July 19, 2011, fromhttp://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=EJ719951

Hussain, I., & Aslam, M. (2009). Role of mobile technology inpromoting campus-wide learning environment. The TurkishOnline Journal of Educational Technology , 8 (3), 48-140. Retrieved

July 13, 2011, fromhttp://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=EJ859492

The consortium for school networking. (2006). Hot technologies foreducation: what's happening now and later. Middle Ground , 9 (4),12-15.

Vesisenaho, M., Valtonen, T., Kukkonen, J., Havu-Vuutinen, S.,Hartikainen, A., & Karkkainen, S. (2010). Blended learning witheveryday technologies to activate students' collaborativelearning. Science Education International , 21 (4), 272-283.

10
http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=EJ719951http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=EJ719951http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=EJ719951http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=EJ719951


11/20

Appendix A Evaluator's Program Description

EvaluationQuestion

ProgramObjective/Goals

Activity(ies) toObserve

Data Source Population/Sample

Design Data Analysis Audience

Has studentlearning improvedthrough theinnovative use of technology?

Student willdemonstrate anincrease in quality of work from the beginning of the year to the end of the school year.

survey responses,interview responses,review of students'portfolio,analysis of students'success rate of placing an appon iTunes

surveys,interviews,portfolio,observation

students in the business/appprogrammingclass

surveys createdand distributed by Google forms;interviews pre, andpost; creating acomparison rubricto collect data onstudent portfolios;observation datacollection onsubmitted andaccepted apps

Descriptiveanalysis

Administration,curriculum direcschool board, anIowa School BoFoundation

Have studentshigher orderthinking andproblem solvingskills improvedbecause of thiscourse?

Students will beable to analyze andplan for complexproblems in order toreach a resolutionpoint. Students willdemonstrate growthin higher orderthinking skills andproblem solving.

survey responses,interview responses,review of students'portfolio,analysis of students'success rate of placing an appon iTunes

surveys,interviews,portfolio,observation

students in the business/appprogrammingclass

surveys createdand distributed by Google forms;interviews pre, andpost; creating acomparison rubricto collect data onstudent portfolios;observation datacollection onsubmitted andaccepted apps

Descriptiveanalysis andchart


How has the co-teacherscollaborativestudy of theircontent, practice,and studenteffects beenaffected?

Co-teacher willpresent their viewson a collaborative work environmentand the correlatingchanges to theirteaching practices.

interview responses,survey responses

surveys,interviews

co-teachers interviews pre, andpost by co-teachers,administrators,and curriculumdirectors;

Descriptiveanalysis


How has theunderstanding of best practicesrelated tointegration of technology forimprovinglearning changedfor the co-

teachers?

Co-teacher willdemonstrate the best-practicelearned through theprocess of teachingthe class for anentire school year.Co-teachers willpresent student

works as examplesof how to apply best-practices foruse of technology inthe classroom.

interview responses,

interviews co-teachers interviews pre, andpost by co-teachers,administrators,and curriculumdirectors;

Descriptiveanalysis


11


12/20

Appendix B

EDP Schedule

Date Activity 6/27/2011 Create EDP and timeline6/29/2011 Gather research on best practices for technology 6/30/2011 Gather research on best practices for technology 7/1/2011 Gather research on collaborating teachers in a classroom

7/2/2011 Gather research on best practices for technology 7/3/2011 Gather research on higher order thinking skills and problem solving

for high school students7/5/2011 Gather research on higher order thinking skills and problem solving

for high school students7/6/2011 Revise EDP if necessary 7/7/2011 Work on survey questions, interview questions7/8/2011 Work on survey questions, interview questions7/9/2011 Work on survey questions, interview questions

7/10/2011 Work on survey questions, interview questions7/11/2011 Work on rubrics for portfolios7/12/2011 Work on rubrics for portfolios7/13/2011 Work on rubrics for portfolios7/14/2011 Make spreadsheet for recording submitted apps, and approvals -

share with co-teacher7/15/2011 Review survey questions, and interview questions7/16/2011 Start rough draft of final project. Focus on the research proving the

effectiveness of technology in the classroom compared to the

objectives. The methodology section and background section.7/17/2011 Work on rough draft, and search for additional research7/18/2011 Work on rough draft, and search for additional research7/19/2011 Finish up purpose and background section of the final report7/20/2011 Revise the background and background section

12


13/20

7/21/2011 Finish up the methodology section. This should be the researchpiece that proves the methods chosen will work.

7/22/2011 Revise methodology section7/23/2011 Result section? Will not have any

7/24/2011 Write Discussion and conclusion section. Focus on the research andmethods.7/25/2011 Revise the discussion and conclusion sections.7/26/2011 Write the summary section.7/27/2011 Revise the summary section.7/28/2011 Work on revisions and corrections.7/29/2011 Work on revisions and corrections.8/1/2011 Hand in Final Project to Dr. Perkins

13


14/20

Appendix C

14


15/20

15


16/20

Appendix D

16


17/20

17


18/20

Appendix E

Appendix FOverall Evaluation Questions:

1. Has student learning improved through the innovative use of mobile technology?2. Have students higher order thinking and problem solving skills improved

because of this course?3. How has the co-teachers collaborative study of their content, practice, and

student effects been affected?4. How has understanding of best practices related to integration of technology for

improving student learning changed for the co-teachers?

Interview Protocol for Teachers1. What is your name?2. What is your previous experience co-teaching?3. What was your experience like co-teaching this year?4. What was your experience level with the different content of this class at the

beginning of the year?a. What was your experience level with the specific content (app

programming & marketing) of this class when you started teaching?

18


19/20

b. How has that changed throughout the year?2. How did you choose the materials for the class?a. What would you change about the2. How do you communicate with your co-teacher?3. Have you explored other ways to work together?4. What things have you done to try to promote higher order thinking skills in

students?5. How do you think that this class has innovated using mobile technology?a. What proof do you have that it has been a positive in the classroom?2. How have you worked with your co-teacher to understand and develop your

content?3. How have you worked with your co-teacher to improve your practice?4. How do you think your co-teaching has affected student achievement?a. Can you give explains?

b. What data tells you that this is happening?2. How has co-teaching affected your understanding of best practices?3. What ways have you worked to understand what best practices work for

technology?4. What data do you have that your best practices affected student learning?5. How have you integrated technology into your co-teaching?6. How have the student reacted to the integration?7. How have you shared your experience with other teachers outside of your co-

teacher?a. What impact do you think that it has had on other teachers?b. What impact do you think it had on the district as a whole?

1. Having co-taught for a year, what are the positives and negatives of co-teaching?2. What would be your recommendations to others that might start co-teaching?3. After teaching this course for a year, what changes would you make in

curriculum, instruction, assessment and/or technology integration?

Appendix GOverall Evaluation Questions:

1. Has student learning improved through the innovative use of mobile technology?2. Have students higher order thinking and problem solving skills improved

because of this course?3. How has the co-teachers collaborative study of their content, practice, and

student effects been affected?4. How has understanding of best practices related to integration of technology for

improving student learning changed for the co-teachers?

Interview Questions for Students at the End of the Year 1. What background knowledge did you have on the subject area?

a. Can you give specific examples for both programming and marketing?2. What experience do you have with mobile devices?a. Have you used them for educational purposes?

i. If so how?b. How has mobile technology changed your daily life?

c. How has mobile technology changed your education?

19


20/20

2. Has the use of an iPad changed from the beginning of the year to the end?1. If so how?

2. Has it benefited you learning?2. Has your outlook at mobile devices changed at all because of this class?3. Have you ever taken a class with either teacher before?1. If so, have they or both changed in the delivery of their instruction?

2. Do you think these changes have improved you learning?i. Could give a specific example?

2. Has the outlook on solving problems changed because of this class?1. If so, how and why?

2. How do you apply this to other classes, if at all?2. Do you feel that you have been challenged by this class to think at a higher

level?1. Can you give an example?

2. What challenges could the teacher presented to you to make you think ahigh level?

2. What were the biggest challenges or problems you faced in this class?1. How did you deal with it?

i. Did it work?2. Would you take this class again, if you had the chance?3. Did Apple accept your app?

shane wheeler final project 1

Documents