beginning spanish 1a summative evaluation

Course Design Document: Summative Evaluation Planning

1 Web-Based Learning Chapter 10

Table of Contents Revised Goal Statement .................................................................................................................. 2

Summative Evaluation .................................................................................................................... 2

Evaluation Planning Worksheet for: ............................................................................................... 4

Summative Evaluation Timeline ................................................................................................. 6

Appendices ...................................................................................................................................... 7

Appendix B: Summative Evaluation graphs and tables. (samples) ............................................. 7

Learner Exit Survey Results ..................................................................................................... 8

Appendix C: End Surveys and Questionnaires ............................................................................ 9



Revised Goal Statement Upon completion of the WBI course Beginning Spanish 1a, students will be able to read,

write, and communicate in Spanish using basic grammar and vocabulary.

Summative Evaluation The original questions that the Preliminary Summative Evaluation Planning intended to address

were:

Was the WBI effective?

Was the WBI appealing?

Was the WBI efficient?

Areas of Consideration

Questions

Instructional Goals And Content

Effectiveness:

Did the instructional content meet the intended goals?

Are learners able to use Spanish phrases to communicate at a basic level?

Appeal:

Are learners planning to take the next course in the WBI series, Beginning Spanish 1b?

Would learners like to take another WBI course at JB K-12 Public School?

Efficiency:

Did the actual time commitment reflect what was advertised in the course syllabus?

What was the average time spent on course for the learner? For the instructor? For the technology liaison?

Learners and Instructor

Effectiveness: Were the stakeholders pleased with the end results of instruction?

Did the instructor feel the instruction met her teaching expectations?

Appeal: Did learners feel the course met their needs in terms of age appropriateness, interest, intended use, and personal goals?

Would the instructor teach this course as is again?

Efficiency: Did learners find the WBI user-friendly?

Did the instructor find the WBI user-friendly and easy to modify if necessary?



Context

Effectiveness: Was the program compatible with equipment available and used by the learners and instructor?

Did the course lend itself well to the WBI format?

Was the technical support sufficient for the course?

If there were problems with the WBI, were they addressed and resolved in a reasonable amount of time?

Did the computer specifications and Web browser work well with the Website?

Appeal: Were course expectations clearly stated and followed?

Were the authors clearly identified? Were all references documented correctly and permissions obvious?

Efficiency: Was the Web site easily accessible for all participants?

Was course access limited to registered participants?



Evaluation Planning Worksheet for: Beginning Spanish 1a

by

Shannon D. Foster, JB K-12 Public School, Sunday, April 28, 2013

The Summative Evaluation Plan

Who are the stakeholders?

Primary: Shannon D. Foster, Mr. M.F. Patrick, school superintendent and principal, and : JB K-12 Public School board, the instructor

Secondary: Learners, parents, and Office of Public Instruction.

What is being evaluated?

Course: Beginning Spanish 1a

Products: content, directions for assignments, discussions, and group/paired/individual activities

Processes: assignments, learner participation and interaction, feedback from instructor

Who are the evaluators?

Evaluators

Mrs. Lynda Simmons has been hired as the external evaluator. Mrs. Simmons is an adjunct professor of Spanish at the local university and has been involved with developing and teaching online Spanish to college students and other adult learners.

Mr. Todd Harkins will serve as the technical evaluator. Mr. Harkins has been fully involved in the WBI development and has seen firsthand how the WBI Beginning Spanish 1a has worked. He is also the technical liaison for other WBI courses that are beginning to be adopted at the schools served by the Technology Co-op.

Mrs. Carri Leigh will be the subject matter expert (SME), replacing Mrs. Bernard who is now retired. Mrs. Leigh has been a high school Spanish teacher for the past 20 years and has taken online courses and so is familiar with the online classroom.

Mrs. Jillian Gleeson will serve as the final course evaluator. She is the school counselor at JB K-12 Public School and has been closely involved with overseeing the students’ educational requirements at JB K-12 Public School. Participants in the Evaluation

Parents of students who have taken the WBI course Beginning Spanish 1a have been given an evaluation form at the end of the course asking for feedback regarding their students participation, communication with the instructor, and if expectations were met.

Students who complete the WBI course Beginning Spanish 1a receive an exit survey regarding the effectiveness, appeal, and efficiency of the course.

Mrs. Young, administrative assistant, will be interviewed regarding record keeping



responsibilities and the hybrid model.

Selected members of the current teaching staff who daily interact with students enrolled in the course will be asked for feedback regarding student workload and expectations, attitude toward WBI course observed in participants, and the effectiveness of the hybrid model.

What are the evaluation methods and tools?

Because this is a new format for teaching Spanish in the JB K-12 Public School, test scores and semester grades will be used to determine the successfulness of the WBI format. Besides these data, other methods will be used:

Opinion surveys for information regarding user-friendliness and if expectations were met.

Interviews with selected participants.

When are data collected and analyzed?

Data will be collected from learner test scores and semester grades, surveys, and interview. The data will be analyzed by the instructor, Mrs. Foster, and she will compile the information into a report that will be distributed to the primary stakeholders during a school board meeting. Discussion will ensue during the meeting. Initial data will be collected, analyzed, reviewed, and discussed after the initial implementation of the WBI course Beginning Spanish 1a. Data will be collected as scheduled below in the Summative Evaluation Timeline. Henceforth, every three years the WBI course Beginning Spanish 1a will be reviewed and evaluated.

How are evaluation reports reported?

Data will be gathered from course test scores and course semester exam. Other data will be derived from end of course surveys given to course participants and parents, and interviews with selected staff that interacts with participating students

Data will be organized in a report comprising a written report, table and graphic representations of data gathered, and recommendations given by experts and evaluators.

Data analysis will be based on performance scores, survey responses from participant opinions regarding WBI course use, and recommendations from evaluators.



Summative Evaluation Timeline

Tasks

We

eks

1-3

We

ek

4-

6

We

eks

7-9

Qu

arte

r b

reak

We

eks

10

-12

We

eks

13

-15

We

eks

16

-18

End

of

Sem

est

er

End

of

sem

este

r

+2 w

ee

ks

Consent form given and returned

X

Unit tests X X X X X

Current learner survey

X X

Staff interviews X X

Parent survey X

Observe current learners

X

Semester test X

Semester grade X

Evaluators’ evaluation X

Analyze data X X

Write report X X

Board meeting and stakeholder discussion

X



Appendices

Appendix B: Summative Evaluation graphs and tables. (samples) The following graph represents the high and low scores of students enrolled in the WBI course

Beginning Spanish 1a.

Enrolled students: 25

Unit 1 Test Unit 2 Test Unit 3 Test Unit 4 Test Unit 5 TestSemester

Exam

High score 98 95 95 93 95 97

Median score 86.5 82 77 79.5 82 91

Low score 75 69 59 66 69 85

0

20

40

60

80

100

120

Gra

de

WBI Beginning Spanish 1a performance scores



Learner Exit Survey Results The following information was taken from student responses. Student responses: 22

1 –

Str

on

gly

agre

e

2 –

Agr

ee

3 –

Neu

tral

4 –

Dis

agre

e

5 –

Str

on

gly

dis

agre

e

1. Did the instructional content meet the course goals and objectives?

17 (77%)

5 (23%)

2. Are you able to use Spanish phrases to communicate at a basic level?

15 (68%)

5 (23%)

2 (9%)

3. Was the program compatible with your equipment? 19 (86%)

3 (14%)

4. Did the course lend itself well to the WBI format? 12 (54.5%)

9 (41%)

1 (4.5%)

5. Was the technical support sufficient for the course? 19 (86%)

3 (14%)

6. If there were problems with the WBI, were they addressed and resolved in a reasonable amount of time?

20 (91%)

1 (4.5%)

1 (4.5%)

7. Did the computer specifications and Web browser work well with the Website?

22 (100%)

8. Are you planning to take the next course in the WBI series, Beginning Spanish 1b?

19 (86%)

1 (4.5%)

2 (9%)

9. Would you like to take another WBI course at JB K-12 Public School?

20 (91%)

1 (4.5%)

1 (4.5%)

10. Did you feel the course met your needs in terms of age appropriateness, interest, intended use, and personal goals?

18 (82%)

2 (9%)

2 (9%)

11. Were course expectations clearly stated and followed?

21 (95.5%)

1 (4.5%)

12. Did the actual time commitment reflect what was advertised in the course syllabus?

How much time on the average did you spend on this course? _________

19 (86%)

1 (4.5%)

2 (9%)

13. Was the course Web site user-friendly? 20 (91%)

1 (4.5%)

1 (4.5%)

14. Was the Web site easily accessible? 21 (95.5%)

1 (4.5%)



Appendix C: End Surveys and Questionnaires

Learner Exit Survey

Please answer the following questions based on your perceptions. Put the number in the blank to the left of the question.

1 – Strongly agree 2 – Agree 3 – Neutral 4 – Disagree 5 – Strongly disagree Upon completion, please return the questionnaire to the administrative assistant, Mrs. Young.

Effectiveness:

1. Did the instructional content meet the course goals and objectives?

2. Are you able to use Spanish phrases to communicate at a basic level?

3. Was the program compatible with your equipment?

4. Did the course lend itself well to the WBI format?

5. Was the technical support sufficient for the course?

6. If there were problems with the WBI, were they addressed and resolved in a reasonable amount of time?

7. Did the computer specifications and Web browser work well with the Website?

Appeal:

8. Are you planning to take the next course in the WBI series, Beginning Spanish 1b?

9. Would you like to take another WBI course at JB K-12 Public School?

10. Did you feel the course met your needs in terms of age appropriateness, interest,

intended use, and personal goals?

11. Were course expectations clearly stated and followed?

Efficiency:

12. Did the actual time commitment reflect what was advertised in the course syllabus? How much time on the average did you spend on this course? _________

13. Was the course Web site user-friendly?

14. Was the Web site easily accessible?

Write Additional Comments on back of paper:



Parent Questionnaire

Please answer the following questions based on your perceptions. Put the number in the blank to the left of the question.

1 – Strongly agree 2 – Agree 3 – Neutral 4 – Disagree 5 – Strongly disagree Upon completion, please return the questionnaire to the administrative assistant, Mrs. Young.

Effectiveness:

1. Were your expectations of the course met?

2. Is your child able to use Spanish phrases to communicate at a basic level?

3. Do you think the WBI format was compatible with your child’s learning abilities?

Appeal:

4. Would your child like to take another WBI course at JB K-12 Public School?

5. Did the course meet your child’s needs in terms of age appropriateness, interest, intended use, and personal goals?

6. Were course expectations clearly stated?

Efficiency:

7. Was the communication with the instructor and school satisfactory regarding your students’ progress?

8. Did the actual time commitment reflect what was advertised in the course syllabus? How much time on the average did your child spend on this course? _________

9. Was the course Web site user-friendly?

10. Was the Web site easily accessible?




Interview Questions for Administrative Assistant

1. Did the instructor submit masters for needed materials in a timely fashion? Comments:

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

2. Were extra materials available to give to students when the instructor was not on

campus? Comments:

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

3. Did the instructor communicate student concerns and non-participation to the required

authority (Mrs. Gleeson) in an efficient manner? How were those concerns

communicated? Comments:

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

4. Did the schedule work well using the hybrid format for the class? What are some

concerns that you would like to see addressed? Comments:

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

5. Were grades submitted at regular intervals and on time? Comments:

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

6. Did the hybrid model fulfill the course expectations? How?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________




Interview Questions for Current Teaching Staff

1. Were you aware of students participating in the Web-Based Instruction (WBI) course

Beginning Spanish 1a?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

2. Did students discuss the workload and expectations of the course? What were their

comments?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

3. Did they voice any concerns regarding the WBI hybrid format of the course Beginning

Spanish 1a? What concerns did they discuss?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

4. Did students appear overwhelmed by a WBI hybrid course due to technology issues or

number of assignments?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

5. What attitudes did you sense in students regarding the WBI hybrid course Beginning

Spanish 1a?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________


beginning spanish 1a summative evaluation

Documents