Curriculum and Program Development

Curriculum and Program DevelopmentThe Theory1THE FIVE MOST SIGNIGICANT CURRICULUM EVENTS IN THE TWENTIETH CENTURYRalph W. Tyler

Edward Thorndikes refutation of the two long held beliefs

John Deweys Monograph on Interest and Effort

The 26th Yearbook of NSSE

The Society for Curriculum Study (NEA)

The Eight-Year Study2Thorndikes refutation

The study of geometry or algebra does not necessarily increase students logical ability.

The study of languages such as Latin does not necessarily develop students verbal skills.

Consequently, these subjects should not be considered core curriculum.

Curriculum should be designed to help students become successful in life.

3Deweys MonographStudents will put more effort in learning something that they find interesting.

Before his monograph it was thought that studies should be distasteful and not interesting in order for the subjects to be a challenge.

Students have to discipline themselves.4The 26th Yearbook of the National Society for the Study of EducationIn 1927 the NSSE published 2 volumes on curriculum. They concluded that curriculum development processes in American schools was unsatisfactory.

This was the stimulus for the beginning of the curriculum as a field of study that would produce curriculum specialists. 5The Society for Curriculum StudyIn 1930 at an annual NEA meeting a group of 20 educators met informally and started the Society of Curriculum Study. Ralph Tyler was one of the 20.

Later from this beginning, the ASCD was formed.

6The Curriculum Experiments of the 1930sThis was a time of a great depression. Jobs were scarce. The Progressive Education Association developed the 8 yr. study to challenge 30 schools to demonstrate a curriculum that met the needs of the students.

7Significant Outcomes of the 1930 ExperimentsShowed that schools were capable of designing programs that would be of interest and support the needs of the students.SAT (entrance exams) were developed as a way of meeting college requirements.

Faculty Inservice workshops were started.

8More advantages from the 1930 ExperimentsMultiple testing methods were being introduced such as questionnaires, observations, and projects, and products. (Tyler, 1987).

It was recognized that rote memory and drill activities were not especially effective learning strategies. Metacognitive learning strategies were better.

9Other significant eventsBrain research into how we learn.Standardized testingLegislation like NCLBEtc.

10The Power of Mindful LearningLanger, E.J. (1997) Mindful learning creates new categories; is open to new viewpoints; is aware of more that one perspectiveMindful learning practices sideways learning with looks for patterns, is flexible and looks for the best choice.

11Mindful Learning and MythsBasics must be over learned through drill.Need to pay attention to learn.Gratification must be delayed.Rote memory is the way to learn.There is a right or wrong answer.

12An Information Processing System

13TWO THINKING STYLES A COMPARISON CHART by Dann Walker

14A Creative mind isPlayfool: Playing with Ideas like a Child with Blocks (Childlike Not Childish)Spontaneous Enthusiastic, expressive, aware of the present.Able to enjoy not just the final product, but the process!Curious Interested, intrigued by things, an explorer of ideas:Inclined to Wonder--Wanting to know how thing work.Inclined to Wonder -- Attracted to a variety of things .RomuH gnisneS Whimsical, fun-loving, able to laugh, finding humor in things.Arranging Building, generating order, creating new relationships among things.Maginative Able to pretend, to make believe, to enjoy fantasy and enchantment.

Progressive: Producing a Prodigious Portion of Preposterous IdeasFluent - Flowing ideas freely, producing plenty of possibilities.Aware of alternatives and able to generate a large quantity of options.Flexible Open-minded, adaptive, ambiguity-tolerant, and risk-taking.First learning how to follow the rules, then learning when to break them.

Proactive: Projecting and Persistently Pursuing GoalsPlanning Setting goals, sowing the seeds, organizing the effort.Preserving Keeping track of plans and results, documenting ideas.Persevering Believing in ones self and ideas, tenaciously pursuing the dream.

Periodically Passive: Purposely Practicing Peaceful Pausing and Patient Postponing

Intuitive Drawing on experience, supplementing rational thinking by trusting ones inner feeling.Incubating Waiting for the flash of illumination, putting it on the back burner, mulling it over, sleeping on it, etc. Serendipity: Those wonderful surprises out of the blue almost always follow a great deal of hard work.Inscrutably Inspired (At least in certain cases) The intangible spark of genius is difficult to analyze.

15The Cycle of Learning

16The adult learnerThe adult learner is self-directing.

The adult learner enters the educational environment with more life experience and a greater variety of life experience.

The adult learner is ready to learn when there is a need to know something in order to perform more effectively in some aspect of life.

The adult learner enters an educational activity with a life-centered, task-centered, or problem-centered orientation.

The adult learner is primarily internally motivated.17Conditions for adult learning and principles of teachingConditions of Adult LearningPrinciples of TeachingLearners feel a need to learn

The learning environment is physically & interpersonally comfortable

The goals of the learning experience are compatible with the learners goals

Learners participate actively in the learning process

The learners past experience is utilized

Learners have a sense of progress towards their goalsThe teacher:

Helps students recognize need to learnHelps students set personal learning goalsEnsures physical comfortAccepts & respects studentsBuilds mutual trust & helpfulness among studentsActs as a co-learnerInvolves students in goal formulationInvolves students in joint decisions regarding designing & operating the learning experienceInvolves students in the inquiry processHelps students utilize their past experienceRelates learning activities & content to the students past experienceHelps students measure progress (including self-evaluation)18Definitions of learning styleLearning styles are characteristic ways of responding in learning situations (Gordon)Learning styles are cognitive, affective, and physiological traits that serve as relatively stable indicators of how learners perceive, interact with, and respond to the learning environment.(Keefe)Learning styles are characteristic ways of processing information and behaving in learning situations (Hiemstra and Sisco)Learning styles are the differences among people in the attitudes, values and approaches they bring to learning.(Erickson and Strommer)19Definitions of learning styleLearning styles are predispositions to adopt particular learning strategies.(Schmeck)Learning styles are stable ways of approaching tasks that are characteristic of individuals.(Briggs)Learning style is the most inclusive term It includes cognitive style which is the common model of thinking and information processing that people use. Cognitive style is concerned with how a person encodes and decodes information. Learning style is also concerned with thinking style which is the mode of functioning that governs an individuals perception and intellectual activities.(Messick)20KOLBSLEARNING Cycle

ActiveExperimentation

Observationand Reflection

Conceptualization and GeneralizationConcrete Experience213 MAJOR THEORIES OF LEARNING BEHAVIORISM(CONNECTIONISM- Thorndike, classical conditioning Pavlov Watson, operant conditioning Skinner, and modeling)COGNITIVE DEVELOPMENT(developmental stages Piaget, levels of thinking, multiple intelligence Gardner, creativity, critical thinking, reflective thinking Dewey, intuitive thinking, and discovery learning Bruner Phenix)HUMANISTIC PSYCHOLOGY(PHENOMENOLOGY)(Gestalt, human needs Maslow, freedom to learn Roger, value clarification Rath , cooperative learning Johnson Slavin, and motivation)1.2.3.22BEHAVIORISM is:based on observable changes in behavior. It focuses on a new behavior pattern being repeated until it becomes automatic.

It uses rewards and punishments in determining future behaviors.

Behaviors that produce positive results are likely to be repeated.

Skinner, Pavlov, Watson and Thorndike are some of the well-known researchers connected with this theory.

23COGNITIVE THEORYis based on the thought process behind the behavior. Changes in behavior are observed, and used as indicators as to what is happening inside the learners mind.

Cognitive theorists believe that learning involves associations established through contiguity and repetition and interaction between heredity and environment.

Wurth, J. (2009). Nova Southeastern University. Ft. Lauderdale, FL. 24COGNITIVE THEORY (CONT.)

Cognitive Theorists are also aware of the value of reinforcement, although they use it as providing feedback instead of as a motivator. (Good and Brophy, 1990).

25HUMANIST THEORY Is based on the belief that learning is viewed as a personal act to fulfill ones potential.Self-concept and self-esteem are essential for learning.There is a natural desire to learn.Learning occurs in a positive environment.

26Some other important theories:CONSTRUCTIVISM THEORY is based on the idea that learners construct their own meaning through individual experiences and schema.

Schema is our internal knowledge structure. All new information is compared, combined, extended or altered to our present schema according to constructivists.

27EXPERIENTIAL LEARNING THEORY was originated by David A. Kolb. It is a four-stage cyclical theory of learning that combines experience, perception, cognition, and behavior. This theory focuses on the idea that everyone has a natural desire to learn and that learners need to be empowered and have control over the learning process. The teacher becomes a facilitator. This theory could be seen as a foundation for the Learner-Centered Theory of learning.

28Learning Styles of Group ACCOMMODATORStrengths:Getting things doneLeadershipRisk-taking

ActiveDIVERGERStrengths:Imaginative abilityUnderstanding peopleRecognizing problemsBrainstorming ReflectiveExperimentationCONVERGERStrengths:Problem-solvingDecision-makingDeductive reasoningDefining problemsAssimilatorAssimilatorStrengths:PlanningCreating modelsDefining problemsDeveloping theoriesAbstract ConceptualizationConcrete Experience29Learning StyleWays of responding in learning situationsPhysiological traitsWays of processing informationDifferences in attitudes, values, approaches

30Learning Styles InventoriesPersonality Embedded FiguresMyers-BriggsFamiliar FiguresWitkinInformation ProcessingKolbGregorcMulti-IntelligenceSocial InteractionGrasha-Reichmann

31Learning Styles InventoriesEnvironmental and InstructionalPEPSDunn, Dunn PriceCanfieldHillMulti-Intelligence

32A FRAMEWORK OF LEARNING STYLE MODELS

33Curriculum and Program DevelopmentCurriculum Development

34Instructional Design is a process

of systematic development of instructional specifications using learning and instructional theory to ensure the quality of instruction.35Instructional Design is a processof analysis of learning needs and goals and the development of a delivery system to meet those needs.36

Instructional Design Models

Tyler ModelTaba ModelGagn ModelKemp ModelBackward Design ModelADDIE Model 37TYLERIAN MODEL OF CURRICULUM DESIGNNEEDOBJECTIVES ORGANIZATION OF CONTENTLEARNING EXPERIENCESEVALUATIONDISSEMINATIONState the need for proposed design.State objective: Indicate what is to be done and how objective will be accomplished State subject matter content and explain how it will be organized.Provide examples of learning experiences to be included in the design.State what will be evaluated and how evaluation will be conducted.Show how design will be communicated to others.38Elements of the Instructional Design Process Morrison, Ross, and Kemp; 4th EditionSupportServicesPlanningSupport ServicesSummative EvaluationProject ManagementFormative EvaluationRevisionEVALUATION INSTRUMENTSINSTRUCTIONAL RESOURCESINSTRUCTIONAL DELIVERYINSTRUCTIONAL STRATEGIESCONTENT SEQUENCINGINSTRUCTIONAL PROBLEMSLEARNER CHARACTERISTICSTASK ANALYSISINSTRUCTIONAL OBJECTIVE39EDUCATIONAL CHARTING Before beginning a new educational program or class when revising an existing one, it is helpful to chart your direction. This charting process can help you be more organized and effective in your presentations. It allows standardization of programs so that others can provide your classes with the same degree of quality. In order to chart your program, you should answer the following questions. Remember to provide enough detail so that a reader can have a clear picture of your program and how to conduct it.

Why is this program needed?What do I hope the learner can do (objective) as a result of this session? NOTE: Your answer should apply to learner actions and not to what you must so as an instructor.What specific content must be provided in this session? NOTE: This question does not mean that you must write the content out verbatim. An outline of key concepts is critical here.What activities will I use to assist the learner in the actions cited in question 2? NOTE: You should give a detailed outline of just how you intend to facilitate the leaning. A person reading your answer should be able to duplicate your efforts.What supplies, materials, resources, consultants, etc., will I need?How much time will this session take? You may specify by learning activity.How will I know that the learner can do the actions(evaluation methods) which have been selected for this session? NOTE: This is an often neglected, but critical step. Be sure to assess both the content and the process involved in providing the content. ALL EDUCATIONAL EFFORTS SHOULD INCLUDE THIS STEP FOR BOTH DOCUMENTATION AND QUALITY ASSURANCE REASONS.What are my projected costs for the total program? What is my actual cost for the program?What is my projected number of learners? What is my actual number of learners?How will you evaluate the effectiveness? What was the cost per student?

40SIX MAJOR PHASES OF A CURRICULUM MODELAnalysis Needs AnalysisTask AnalysisLearner Analysis

ObjectivesEvaluationInstructionImplementationRevision41Schools of PHILOSPHICAL THOUGHTIDEALISTS:External perception exists as ideas

PERENNIALISTS:Believe in liberal arts & sciences; favors cultivation of reasonREALISTS:Believe that theory & principles are basis for learningPRAGMATISTS:Emphasize thinking skills; relevance of curriculum is essential to learningEXISTENTIALISTS:Emphasize freedom of decision-making; foster self -development42PURPOSES AND ADVANTAGES OF BEHAVIORAL OBJECTIVES The definition of objectives in term of observable learner behavior -- the learner does what, given what, how well -- has the following purposes and advantages.

1. It helps the instructor and/or curriculum designer to make explicit his/her own educational aims.2. It communicates the intent of instruction to others.It provides a starting point for a behavioral analysis and construction of a hierarchy of knowledge.It furnishes a goal against which the success of instruction can be measured.Source: Gow, D.T. Design and Development of Curricular Materials, Volume I. University of Pittsburgh.

43 PURPOSES AND ADVANTAGES OF BEHAVIORAL OBJECTIVES The definition of objectives in terms of observable learner behavior -- the learner does what, given what, how well -- has the following purposes and advantages:

5. It provides a specific focus for involving others concerned with education in the discussion of educational goals. 6. It informs students of the purposes of their instruction so that they may know what they are expected to do.7. It facilitates the individualization of instruction.8. It makes possible the continuous reevaluation and reorganization of instructional objectives as needs changes.Source: Gow, D.T. Design and Development of Curricular Materials, Volume I. University of Pittsburgh.

44Gagn ModelSYSTEM LEVEL Needs Delivery Resources StructureObjectives

LESSON LEVEL Needs Media Resources AssessmentSYSTEM LEVEL Teacher Preparation Formative Evaluation Revision; Field Testing Summative Evaluation Diffusion

45Model for Curriculum Development Saylor, Alexander and Lewis

Formulate Goals and Objectives

Create Curriculum Design Specifications

Develop Implementation(Instruction) Plans

Select Evaluation Procedures (Students and Plan)

EXTERNAL FORCES:Community, Legal, Research, Professional KnowledgeSocietyLearnersKnowledgeSource: Adapted from J. Galen Saylor, William J. Alexander and Arthur J. Lewis.Curriculum Planning for Better Teaching and Learning (New York Holt, Rinehart and Winston, 1981) 46TEACHING LEARNING EVALUATION MODELDefine and Develop Instructional ObjectivesDevelop Instruments to Measure ObjectivesEmploy the Instruments to Obtain Student Evaluation of the Instructor and InstructionEmploy the Instruments to Measure Student AchievementProvide Feedback to the Instructor and to the StudentsProvide Feedback to the Instructor and to the StudentsDetermine if the Instructional Objectives have been AccomplishedSummarize Data During and at the End of the Term for Final Grade DeterminationStudent Learning Assessment ProcessFaculty Evaluation ProcessCourse Evaluation ProcessSource: Adapted from J. Galen Saylor, William J. Alexander and Arthur J. LewisCurriculum Planning for Better Teaching and Learning (New York; Holt, Rinehart and Winston, 1981)

47BRIGGS MODEL of Instructional Design1.2.3.4.5.STATE OBJECTIVES & PERFORMANCE STANDARDSPREPARE TESTS OVER THE OBJECTIVESANALYZE OBJECTIVES FOR STRUCTURE & SEQUENCEIDENTIFY ASSUMED ENTERING COMPETENCIESPREPARE PRETESTS, REMEDIAL INSTRUCTION, SCREENING, etc. 48BRIGGS MODEL of Instructional Design6.7.8.9.10.SELECT MEDIA & WRITE PRESCRIPTIONSDEVELOP FIRST DRAFT MATERIAL SMALL GROUP TRYOUTS & REVISIONSCLASSROOM TRYOUTS & REVISIONSPERFORMANCE EVALUATION49Process for Educational Program Development PROJECT SELECTION AND DESIGN

BASIC PLANNING INPUTS(Project-Specific)

Domain of knowledge Student knowledge, attitudes, and prioritiesSocietal needsResearchEducational priorities

PROJECT-SPECIFIC FACTORSResearchGoals TimeResources - Human - Materials - Facilities - FiscalStudents - Number - LocationPROJECT GENERATION AND SELECTIONTHE IDEAL SELECTIONOPERATIONAL SEQUENCEPHASE IInstructional Design Model by Robert Diamond50Process for Educational Program DevelopmentPRODUCTION, IMPLEMENTATION, AND EVALUATION FOR EACH UNIT

DESIGN EVALUATION INSTRUMENTS & PROCEDURESDETERMINE OBJECTIVES SELECT INSTRUCTIONAL FORMATSEVALUATE & SELECT EXISTING MATERIALSPRODUCE & FIELD TEST NEW & AVAILABLE MATERIALSCOORDINATE LOGISTICS FOR IMPLEMENTATIONIMPLEMENT EVALUATE & REVISEPHASE II Instructional Design Model Robert Diamond51Cognitive DomainKnowledgeComprehensionApplication AnalysisSynthesisEvaluation

52Affective DomainReceiving Responding ValuingOrganizationCharacterization

53Psychomotor DomainPerceptionPreparationOrientationPracticePatternPerformance

54

Three Basic Elements of OBJECTIVES To be useful, an objective must contain three basic elements:

1.A verb that describes observable action2.A description of the conditions under which this action takes place (when given x, the students will be able to) The level of acceptable performance that is, what percentage of correct answers, etc. will be considered acceptable. 55TYPES OF LEARNING OUTCOMES COMMON TO MANY AREAS AND LEVELS OF INSTRUCTIONLOWER LEVEL COGNITIVE PROCESSESRecallingComparing Knowledge TranslatingClassifying ComprehensionInterpretingApplying ApplicationEstimating

HIGHER LEVEL THINKING SKILLSIdentifyingFormulating AnalysisAnalyzingGenerating SynthesisInferringJudging EvaluationRelating

AFFECTIVE OUTCOMESListeningRelating AttitudesRespondingValuing InterestsParticipating AppreciationsSeeking AdjustmentsDemonstrating

PERFORMANCES OUTCOMESSpeakingConstructing ProcedureSingingDemonstrating ProductDrawingOperating Procedure and ProductComputingPerformingWritingOriginating{{{{56Strategies That Promote Active Learning In The ClassroomStudents are involved in more than listeningLess emphasis is placed on transmitting information and more on developing students skillsStudents are involved in higher order thinking (analysis, synthesis, evaluation)Students are engaged in activities (e.g., reading, discussing, writing)Greater emphasis is placed on students exploration of their own attitudes and values57Curriculum

ModelsKEMP 10-ELEMENTS

BANATHY MODEL

FARLS-STOWE 21- STEP MODEL

TABA MODEL

DICK & CRET 9-STAGE MODEL

GAGNE 14-STAGE MODEL

6 REVISIONSGENERAL SYSTEMS MODEL ANALYSIS OBJECTIVES IMPLEMENTATION INSTRUCTIONEVALUATION12345CURRICULUM DEVELOPMENT58Curriculum and Program DevelopmentAssessment

59Definition of Evaluationfrom American Evaluation Association Evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness.

60The Purposes of Evaluation is: * to provide a basis for making decisions * to make judgments about the worth or success of people, programs, lessons or projects* to determine if the objectives of the instruction are being mastered.

61Types of EvaluationsSUMMATIVE EVALUATION ASKS: How did we do? FORMATIVE EVALUATION ASKS: How are we doing?CONFIRMATIVE EVALUATION ASKS: How are we still doing?

You could change this slide to address assessment for students and courses. Formative: How are the students doing? Summative: How well did the students do? Or did the students learn what they were supposed to learn? Confirmative: Is the knowledge transferring and are the students applying the knowledge?62Definition of AssessmentAssessment is a process of gathering and documenting information about the achievement, skills, abilities, and personality variables of an individual.

63

There are five primary Purposes or uses of assessments

DiagnosticFormativeNeedsReactionSummative

Diagnostic assessment to ID needs and prior knowledge in order to direct them to the most appropriate learning experience.Formative assessment to provide practice for search and retrieval (memory) for a student to provide prescriptive feedback.Needs assessment to determine the knowledge, skills, abilities and attitudes of a group to assist with gap analysis and course developmentReaction assessment used to determine the satisfaction level with a learning or assessment experience. Ex. Course evaluations, smile or happy sheets completed at endSummative assessment that is usually quantitative with the purpose to give a grade or make a judgment about students achievement.64Purposes of Assessmentto identify specific problem areas and/or needs to evaluate the individual's performance in relation to others to evaluate the individual's performance in relation to a set of standards or goals to provide teachers with feedback on effectiveness of instruction to predict an individual's aptitudes or future capabilities

Examples: 1. prescriptive maybe use paper pencil test, observation, standard test 2. Standardized test Achievement test- FCAT 3. Criterion-referenced test 4. formative and summative 5. SAT GRE65

To assess knowledge effective evaluation methods include:

Fill in the blanks testsMultiple choice testsTrue/false testsMatching testsShort essaysLong essaysand problem-solving questionsInformal responses: one-minute papers, etc.

66Tylers Assessment GuideIDENTIFY THE OBJECTIVES OF THE EDUCATIONAL SYSTEM

DEFINE EACH OBJECTIVE IN TERMS OF BEHAVIOR & CONTENT

IDENTIFY SITUATIONS WHERE OBJECTIVES ARE UTILIZED IN INSTRUCTION

DEVISE WAYS TO PRESENT SITUATIONS

DEVISE WAYS TO OBTAIN A RECORD

DECIDE ON THE TERMS TO USE IN THE APPRAISAL

DEVISE MEANS TO GET A REPRESENTATIVE SAMPLE671. Determine if the purpose of your test is formative or summative.

2. Determine the domain(s) you are testing?

3. Determine the type of information you are trying to gather quantity or quality?

4. Determine the degree of realism you will have in your testing.Considerations in

TEST DEVELOPMENT

685. Determine the number of items needed.

6. Determine the types of items you will employ.

7. Determine the kinds of inferences you will draw from the test scores.

8. Write the test items.

9. Try to pilot for poor item analysis.

10. Revise items as needed.

Considerations in TEST DEVELOPMENT69 1. The assessment of student learning begins with educational values.

2. Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.

3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.

4. Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.

5. Assessment works best when it is ongoing, not episodic. principles of good practice for ASSESSING STUDENT LEARNING70 6. Assessment fosters wider improvement when representative from across the educational community are involved.

7. Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.

8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.

9. Through assessment, educators meet responsibilities to students and to the public.principles of good practice for ASSESSING STUDENT LEARNING71 I. Techniques for assessing academic skills and intellectual development.

II. Techniques for assessing students self-awareness as learners and self-assessments of learning skills.

III. Techniques for assessing student reactions to teachers and teaching methods, course material, activities, and assignments.CLASSROOMTechniquesASSESSMENT72Evaluation Is the Process of:Defining program standards,

Determining if a discrepancy exists between performance and standards, and

Using discrepancy information to change performance / standards.

Malcolm Provus, 1971

731. TO CONTRIBUTE TO DECISIONS RE: PROGRAM INSTALLATION

2. TO CONTRIBUTE TO DECISIONS RE: PROGRAM CONTINUATION, EXPANSION, OR CERTIFICATION

3. TO CONTRIBUTE TO DECISIONS RE: PROGRAM MODIFICATION

4. TO OBTAIN EVIDENCE TO RALLY SUPPORT FOR A PROGRAM

5. TO OBTAIN EVIDENCE TO RALLY OPPOSITION TO A PROGRAM

6. TO CONTRIBUTE TO THE UNDERSTANDING OF BASIC PSYCHOLOGICAL, SOCIAL, AND OTHER PROCESSES* From: Anderson, S.B. & Ball, S. (1980). The profession and practice of program evaluation. CA: Jossey-BassReasons To Evaluate 74GENERAL MODEL OF PROGRAM EVALUATIONNEEDS ASSESSMENTA need has been defined as the discrepancy between what is and what ought to be. Needs are the basis for setting program goals.PROGRAM PLANNING From program goals, specific, measurable objectives are derived and a plan containing the means to reach these objectives is formulated.(Formative Evaluation)

(Summative Evaluation)IMPLEMENTATION EVALUATIONSeeks out discrepancies between the plan & reality

PROGRESS EVALUATIONMonitors indicators of progress toward objectivesOUTCOME EVALUATIONDetermines whether or not the objectives have been attained. This phase includes analysis of program strengths & weaknesses, with recommendations for future modifications.75Objective Oriented EvaluationManagement- Oriented Evaluation

Consumer Oriented Evaluation

Expertise Oriented Evaluation

Adversary Oriented Evaluation

Naturalistic & Interpretive EvaluationEvaluation Models

76 1. Clarify the evaluation request

2. Set boundaries and analyze the evaluation context

3. Identify and select evaluation questions, criteria, & issues

4. Plan data collection, analysis & interpretation

5. Develop a management plan

6. Collect data

7. Analyze and interpret data

8. Report & use evaluation data

9. Evaluate the evaluationPractical Guidelines for PlanningImplementing & Using Evaluations(Worthen & Sanders)77

78ACTIVITY: SHARING ARTICLES

Anchoring Down the Data

Looking Deeper into the Data

Learning About and From Data

Data That Count

79

Evaluation is not something that occurs at the back end of Instructional Design.Evaluation is not something that only occurs at the backend of Instructional Design.My final word on evaluation and instructional design. 80THE EVALUATOR MUST BE ABLE TO:1. Describe the object of an evaluation;2. Describe the context of an evaluation;3. Conceptualize appropriate purposes and framework for evaluation4. Identify and select appropriate evaluation questions, information needs; sources of information;5. Identify, select, and apply appropriate techniques and procedures for information collection, processing, and analysis;6. Determine value of the object of an evaluation;7. Communicate evaluation plans and results effectively;8. Manage evaluations;9. Maintain ethical standards ;10. Adjust for external factors that affect an evaluation;11. Evaluate the evaluation (meta-evaluation.)

competencies of the EVALUATOR81 1. Conceptual clarity

2. Characterization of the object of the evaluation

3. Recognition and representation of legitimate audiences

4. Sensitivity to political problems

5. Specification of information needs and sources

6. Comprehensiveness / Inclusiveness

7. Technical adequacy

8. Consideration of costs

9. Explicit Standards / Criteria

10. Judgments and / or recommendations

11. Reports tailored to audiences characteristics of GOOD EVALUATIONS82Ethical Principles for educatorsAUTONOMY

TRUTHTELLING

CONFIDENTIALITY

FIDELITY

BEMEFICIENCE AND NON-MALEFICENCE

JUSTICE

83Vocation toTeachingConnect both heart and mind

Receiving and welcoming

Questioning ourselves and learners

Telling the story; beware of national amnesia

ProfessingCareful listeningThinking larger than settingDiscourse of friendship

Challenge: WRITE IT DOWN!

84ASSESSMENT OPTIONSPRODUCT EXAMINATIONADVANTAGES high validity; simulates / measures real life situation or performanceDISADVANTAGES scoring is difficult (different faculty must agree on criteria) time consuming to score SITUATIONAL ASSESSMENTS ADVANTAGES simulates real life situations; has high validityDISADVANTAGES scoring is difficult & time consuming; maybe unreliable; requires trained observerWRITTEN ASSESSMENTS ADVANTAGES economical & efficientDISADVANTAGES may not be validFREE RESPONSEEncourages organization of ideas; relatively easy to develop, but scoring is time consuming and may be unreliable STRUCTURED RESPONSEEfficient, effective, relatively easy to develop, but difficult and time consuming85TEACHING LEARNING EVALUATION MODELDefine and Develop Instructional ObjectivesDevelop Instruments to Measure ObjectivesEmploy the Instruments to Obtain Student Evaluation of the Instructor and InstructionEmploy the Instruments to Measure Student AchievementProvide Feedback to the Instructor and to the StudentsProvide Feedback to the Instructor and to the StudentsDetermine if the Instructional Objectives have been AccomplishedSummarize Data During and at the End of the Term for Final Grade DeterminationStudent Learning Assessment ProcessFaculty Evaluation ProcessCourse Evaluation ProcessSource: Adapted from J. Galen Saylor, William J. Alexander and Arthur J. LewisCurriculum Planning for Better Teaching and Learning (New York; Holt, Rinehart and Winston, 1981)

86Goal: Admission of qualified studentsOutcome data bearing on goal:

Demographic data Standardized test results Undergraduate origins Undergraduate courses taken Undergraduate grades Data from admissions interviews Extracurricular activities

Goal: Success in the Educational program

Outcome data bearing on goal :

Course examination results Standardized test results Record of faculty and / or professional observation of skills Examinations of specific practice competencies Student questionnaires on satisfaction Performance of graduates on licensure and specialty examinations Acceptance of graduates into further study

G

G87 utcomes Outcomes are results. The value of results depends on the goals We must begin with the clear identification of goals, Then find outcomes that allow determination of the extent to which the goals are met. These outcomes are gathered, And an analysis made of how well (or to what extent) goals have been met. That summative analysis can be useful externally in documenting the success of a school or of its programs. But of greater importance is the formative use of outcome data and analysis to provide guidance to constructive change.

o ow Can Outcomes Help Schools? Demonstrating to the public that supports them that a good educational job is being done and continued investment in the schools is warranted; Meeting external requirements, such as those of accrediting agencies and funding organizations; Providing a basis for making informed decisions; Most importantly, seeking effective ways to stimulate and document constructive change within the institution.

O

H88

To assess skills and performance the use of:

Direct testing of the performanceAnalysis of naturally occurring eventsRatingsChecklists of behaviorRubrics are suitable methods

89To assess attitudes

Observation of instructionBehavior using rating scalesSurveying, questionnaires Interviewing are most effective.

90KEY POINT from Designing Effective Instruction There should be a direct relationship between instructional objectives and the assessment measures. (Morrison, Ross, & Kemp)

When designing a test you need to determine if the purpose of the test or evaluation is formative or summative and determine the type of information you are trying to gather. Help students to retrieve information from their memory by writing the questions in a manner that it relates to how the information was presented. For example, if the students are taking a test about information that was partially presented with a video and then a demonstration, the wording of a question might look like, remember when we saw the video and the scientist did the experiment how were the results different from when I did the experiment in class? This would be opposed to: What are the differences between (blank & blank)? 91KEY POINTfrom Designing Effective Instruction All three types of evaluation (especially formative) typically require multiple data sources. (Morrison, Ross & Kemp)

During formative evaluation the instructor uses observations, one-minute papers, chapter tests, etc. 92KEY POINTfrom Designing Effective Instruction Multiple measures are valuable in providing a more accurate and complete picture of a particular outcome. (Morrison, Ross & Kemp)

And the last key point from the authors of our test. This is particularly important because of the different learning styles. We all know what kind of test we do better on.93KEY POINTfrom Designing Effective Instruction, 5th edition All three types (formative, summative, and confirmative) of evaluations are driven by instructional objectives and goals. (Morrison, Ross, & Kemp. P.24)

Whether we are talking about evaluations, assessments or tests there must be an alignment with the instructional objectives and goals. 94

What makes you think that we have a grade inflation problem?We cant talk about assessments or evaluations without mentioning grades.95