Assessing Knowledge, Process,

Understanding, and Product/Performance

Dr. Carlo Magno

Further Correspondence: crlmgn@yahoo.com

1

Task #1

• Group work: Write one sample item/task for each skill.

• Group 1: Knowledge

• Group 2: Process

• Group 3: Understanding

• Group 4: Performance/Product

2

Task # 2: Answer the following questions:

• What is assessment for you?

• When do you conduct assessment?

• What do you use to assess academic skills of students?

3

Advance Organizer

• Assessment competencies

• The need for Standards

• KPUP

– Day 1: KPU

– Day 2: Performance/Product

– Day 3: Standards-Based Assessment in English, Math, and Science

4

Assessment Competencies for Teachers

• Constructed by the AFT, NCME, NEA: • Teachers should be skilled in: 1. choosing assessment methods appropriate

for instructional decisions. 2. Administering, scoring, and interpreting the

results of both externally produced and teacher produced assessment methods.

3. Using assessment results when making decisions about individual students, planning teaching, and developing curriculum and school improvement.

American Federation of Teachers, National Council on Measurement and Evaluation, and National Education Association in the United States of America.

Assessment Competencies for Teachers

4. Developing valid pupil grading procedures that use pupil assessment.

5. Communicating assessment results to students, parents, other lay audiences, and other educators.

6. Recognizing unethical, illegal, and otherwise inappropriate assessment methods and uses of assessment information.

Why do we need standards?

• To make sure that everyone delivers quality work

• To produce quality students

• To deliver quality programs

• Basis on what to assess

7

Mathematics Standards for Junior HS

• Math – solve equations involving rational expressions

• English – Share prior knowledge about a text topic

• Science – explain how the respiratory and circulatory

systems work together to transport nutrients, gases, and other molecules to and from the different parts of the body;

8

DepEd Taxonomy

• real-life application of understanding

• enduring big ideas, principles, and generalizations inherent to the discipline

• cognitive operations that the student performs

• content of the curriculum, the facts and information that the student acquires

Knowledge Process

Product/Performance Understanding

Determine whether: Knowledge, process, understanding, product/performance

1. Uses different strategies (looking for a pattern, working backwards, etc.) to solve for the unknown in simple equations involving one or more operations on whole numbers and fractions involving quadratic equation by completing squares.

2. Solves word problems involving quadratic equation

3. Explain the different patterns of non-Mendelian inheritance

4. Describe the different types of volcanoes

5. Compose an informative essay 6. explain how earthquake waves provide

information about the interior of the earth 7. Uses models or illustrations, explain how

movements along faults generate earthquakes 8. Measures time using a 12-hour and a 24-hour

clock 9. Utilize coping reading strategies to process

information in a text 10.Draw a circle and illustrate 5 different chords.

Determine whether: Knowledge, process, understanding, product/performance

Determine whether: Knowledge, process, understanding, product/performance

Determine whether: Knowledge, process, understanding, product/performance

Determine whether: Knowledge, process, understanding, product/performance

Determine whether: Knowledge, process, understanding, product/performance

Determine whether: Knowledge, process, understanding, product/performance

Determine whether: Knowledge, process, understanding, product/performance

Knowledge

• Define

• Describe

• Identify

• Label

• Enumerate

• Match

• Outline

• select

• State

• Name

• reproduce

18

Six Facets of Understanding

Explain - provide thorough and justifiable accounts of phenomena, facts, and data

Interpret — tell meaningful stories, offer apt translations, provide a revealing historical or personal dimension to ideas and events; make subjects personal or accessible through images, anecdotes, analogies, and models

Apply — effectively use and adapt what they know in diverse contexts

Have perspective — see and hear points of view through critical eyes and ears; see the big picture

Empathize — find value in what others might find odd, alien, or implausible; perceive sensitively on the basis of prior indirect experience

Have self-knowledge — perceive the personal style, prejudices, projections, and habits of mind that both shape and impede our own understanding; they are aware of what they do not understand and why understanding is so hard

Explain

Which of the following statements of the relationship between market price and normal price is true?

a. Over a short period of time, market price varies directly with changes in normal price.

b. Over a long period of time, market price tends to equal normal price.

c. Market price is usually lower than normal price.

d. Over a long period of time, market price determines normal price.

Explain

• Explain how the elements specific to a genre contribute to the theme of a particular literary selection

• Explain why the cell is considered the basic structural and functional unit of all organisms

21

Translation from symbolic form to another form, or vice versa

Which of the graphs below best represent the supply situation where a monopolist maintains a uniform price regardless of the amounts which people buy?

A B C D

S

Pri

ce

Quantity

S

Pri

ce

Quantity

S

S

Pri

ce

Quantity

S S

Pri

ce

Quantity

S

Interpret

Interpret

• Interpret simple maps of unfamiliar places, signs and symbols

• Interpret pictographs

• Interpet tables

23

Apply In the following items (4-8) you are to judge the effects of a particular policy on the

distribution of income. In each case assume that there are no other changes in policy that would counteract the effect of the policy described in the item. Mark the item:

A. If the policy described would tend to reduce the existing degree of inequality in the distribution of income,

B. If the policy described would tend to increase the existing degree of inequality in the distribution of income, or

C. If the policy described would have no effect, or an indeterminate effect, on the distribution of income.

__ 4. Increasingly progressive income taxes.

__ 5. Confiscation of rent on unimproved

__ 6. Introduction of a national sales tax

__ 7. Increasing the personal exemptions from income taxes

__ 8. Distributing a subsidy to sharecroppers on southern farms

Have Perspective

• If you are a priest what will be your stand on the RH Bill?

• What would be the explanation in the creation of the universe in a scientific framework?

• In a communist point of view how should schooling be implemented?

• In a liberal perspective, are adults allowed premarital sex?

25

Empathize

After reading the passage answer the following questions…

2. How did Carol feel on this walk?

a. envied

b. sad

c. relaxed

d. happy

3. Carol envied the people around her because they _____________________.

a. were sad and lonely

b. love the city life

c. were laughing and joking

d. don’t like the city

Empathize

• Your new maid from the mountain destroyed your very expensive Narra door and she used it as firewood and cooked rice in your newly landscaped garden. How should you react?

• A…

• B…

• C…

• D…

• Ability to Recognize the Relevance of Information

28

• Ability to Recognize Warranted and Unwarranted Generalizations

29

• Ability to Recognize Inferences

30

• Ability to Interpret Experimental Findings

31

• Ability to Apply Principles

32

• Ability to Recognize Assumptions

33

Reading comprehension • Bem (1975) has argued that androgynous people are

“better off” than their sex-typed counterparts because they are not constrained by rigid sex-role concepts and are freer to respond to a wider variety of situations. Seeking to test this hypothesis, Bem exposed masculine, feminine, and androgynous men and women to situations that called for independence (a masculine attribute) or nurturance (a feminine attribute). The test for masculine independence assessed the subject’s willingness to resist social pressure by refusing to agree with peers who gave bogus judgments when rating cartoons for funniness (for example, several peers might say that a very funny cartoon was hilarious). Nurturance or feminine expressiveness, was measured by observing the behavior of the subject when left alone for ten minutes with a 5-month old baby. The result confirmed Bem’s hypothesis. Both the masculine sex-typed and the androgynous subjects were more independent (less conforming) on the ‘independence” test than feminine sex-typed individuals. Furthermore, both the feminine and the androgynous subjects were more “nurturant” than the masculine sex-typed individuals when interacting with the baby. Thus, the androgynous subjects were quite flexible, they performed as masculine subjects did on the “feminine” task.

34

35. What is the independent variable in the study? a. Situations calling for independence and

nurturance b. Situation to make the sex type react c. Situations to make the androgynous be

flexible d. Situations like sex type, androgynous and

sex role concepts 36. What are the levels of the IV? a. masculine attribute and feminine attribute b. rating cartoons and taking care of a baby c. independence and nurturance d. flexibility and rigidity

Interpreting Diagrams Instruction. Study the following illustrations and answer the following

questions.

Figure 1

35

Pretest Posttest

101. Which group received the treatment? a. group A b. group B b. c. none of the above 102. Why did group B remain stable across the experiment? a. there is an Extraneous Variable b. There was no treatment c. ceiling effect occured 103. What is the problem during the pretest phase of the experiment? a. the two groups are nonequivalent b. the groups are competing with each other c. the treatment took place immediately

Group B

Group A

Examples of Process Competencies

• Employ appropriate listening strategies suited to type of text

• Employ analytical listening to make prediction/projections

• Use illustration strategies to solve the math problem

• Use trial and error to factor the squares • Reread the sentence with difficult words to better

understand the paragraph • Plan the steps in the project before showing the

final poster in class.

36

Process

• Cognitive operations

• Cognitive and Metacognitive skills

• Self-regulation

• Learning strategies

37

Two components of Metacognition

• Knowledge of cognition • reflective aspect of metacognition • Individuals’ awareness of their own knowledge, learning

preferences, styles, strengths, and limitations, • Awareness of how to use this knowledge that can

determine how well they can perform different tasks (de Carvalho, Magno, Lajom, Bunagan, & Regodon, 2005).

• Regulation of cognition • Control aspect of learning • Procedural aspect of knowledge • Allows effective linking of actions needed to complete a

given task (Carvalho & Yuzawa, 2001).

Components of Metacogniton Knowledge of Cognition

• (1) Declarative knowledge – knowledge about one’s skills, intellectual resources, and abilities as a learner.

• (2) Procedural knowledge – knowledge about how to implement learning procedures (strategies)

• (3) Conditional knowledge – knowledge about when and why to use learning procedures.

Examples of knowledge of cognition in Mathematical Investigation

• Declarative Knowledge

– Knowing what is needed to be solved

– Understanding ones intellectual strengths and weaknesses in solving math problems

• Procedural knowledge

– Awareness of what strategies to use when solving math problems

– Have a specific purpose of each strategy to use

• Conditional knowledge

– Solve better if the case is relevant

– Use different learning strategies depending on the type of problem

Components of Metacogniton Regulation of cognition

1) Planning – planning, goal setting, and allocating resources prior to learning.

(2) Information Management Strategies – skills and strategy sequences used on- line to process information more effectively (organizing, elaborating, summarizing, selective focusing).

(3) Monitoring – Assessing one’s learning or strategy use.

(4) Debugging Strategies – strategies used to correct comprehension and performance errors

(5) Evaluation of learning – analysis of performance and strategy effectiveness after learning episodes.

Examples of regulation of cognition

• Planning • Pacing oneself when solving in order to have enough time • Thinking about what really needs to be solved before beginning

a task • Information Management Strategies

• Focusing attention to important information • Slowing down when important information is encountered

• Monitoring • Considering alternatives to a problem before solving • Pause regularly to check for comprehension

• Debugging Strategies • Ask help form others when one doesn’t understand • Stop and go over of it is not clear

• Evaluation of learning • Recheck after solving • Find easier ways to do things

Shifts in assessment

• Testing Alternative assessment • Paper and pencil Performance assessment • Multiple choice Supply • Single correct answer Many correct answer • Summative Formative • Outcome only Process and Outcome • Skill focused Task-based • Isolated facts Application of knowledge • Decontextualized task Contextualized task

Workshop

• Group work

• Each group will accomplish the following task:

• 1 Knowledge item – multiple choice

• 2 Process items – multiple choice

• 1 Explain item – multiple choice

• 1 Apply item – multiple choice

• 1 Interpret item – multiple choice

44

Alternative forms of assessment

• Performance based assessment

• Authentic assessment

• Portfolio assessment

Terms

• Authentic assessment

• Direct assessment

• Alternative assessment

• Performance testing

• Performance assessment

• Changes are taking place in assessment

Method

• Assessment should measure what is really important in the curriculum.

• Assessment should look more like instructional activities than like tests.

• Educational assessment should approximate the learning tasks of interest, so that, when students practice for the assessment, some useful learning takes place.

What is Performance Assessment?

• Testing that requires a student to create an answer or a product that demonstrates his/her knowledge or skills (Rudner & Boston, 1991).

Features of performance assessment

• Intended to assess what it is that students know and can do with the emphasis on doing.

• Have a high degree of realism about them.

• Involve: (a) activities for which there is no correct answer, (b) assessing groups rather than individuals, (c) testing that would continue over an extended period of time, (d) self-evaluation of performances.

• Likely use open-ended tasks aimed at assessing higher level cognitive skills.

Push on performance assessment

• Bring testing methods more in line with instruction.

• Assessment should approximate closely what it is students should know and be able to do.

Emphasis of performance assessment

• Should assess higher level cognitive skills rather than narrow and lower level discreet skills.

• Direct measures of skills of interest.

Characteristics of performance-based assessment

• Students perform, create, construct, produce, or do something. • Deep understanding and/or reasoning skills are needed and

assessed. • Involves sustained work, often days and weeks. • Calls on students to explain, justify, and defend. • Performance is directly observable. • Involves engaging in ideas of importance and substance. • Relies on trained assessor’s judgments for scoring • Multiple criteria and standards are prespecified and public • There is no single correct answer. • If authentic, the performance is grounded in real world contexts

and constraints.

Variation of authenticity

Relatively authentic Somewhat authentic Authentic

Indicate which parts of a garden design are accurate

Design a garden Create a garden

Write a paper on zoning Write a proposal to change fictitious zoning laws

Write a proposal to present to city council to change zoning laws

Explain what would you teach to students learning basketball

Show how to perform basketball skills in practice

Play a basketball game.

Constructing Performance Based tasks

1. Identify the performance task in which students will be engaged

2. Develop descriptions of the task and the context in which the performance is to be conducted.

3. Write the specific question, prompt, or problem that the student will receive.

• Structure: Individual or group?

• Content: Specific or integrated?

• Complexity: Restricted or extended?

Complexity of task

• Restricted-type task – Narrowly defined and require brief responses – Task is structured and specific – Ex:

• Construct a bar graph from data provided • Demonstrate a shorter conversation in French about what is on a

menu • Read an article from the newspaper and answer questions • Flip a coin ten times. Predict what the next ten flips of the coin will

be, and explain why. • Listen to the evening news on television and explain if you believe

the stories are biased. • Construct a circle, square, and triangle from provided materials

that have the same circumference.

• Extended-type task – Complex, elaborate, and time-consuming. – Often include collaborative work with small group of

students. – Requires the use of a variety of information – Examples:

• Design a playhouse and estimate cost of materials and labor • Plan a trip to another country: Include the budget and itinerary,

and justify why you want to visit certain places • Conduct a historical reenactment (e. g. impeachment trial of

ERAP) • Diagnose and repair a car problem • Design an advertising campaign for a new or existing product

Identifying Performance Task Description

• Prepare a task description • Listing of specifications to ensure that essential if

criteria are met • Includes the ff.:

– Content and skill targets to be assessed – Description of student activities

• Group or individual • Help allowed

– Resources needed – Teacher role – Administrative process – Scoring procedures

Performance-based Task Question Prompt

• Task prompts and questions will be based on the task descriptions.

• Clearly identifies the outcomes, outlines what the students are encourage dot do, explains criteria for judgment.

Example of a task Prompt:

Performance Criteria

• What you look for in student responses to evaluate their progress toward meeting the learning target.

• Dimensions of traits in performance that are used to illustrate understanding, reasoning, and proficiency.

• Start with identifying the most important dimensions of the performance

• What distinguishes an adequate to an inadequate demonstration of the target?

Example of Criteria

• Learning target: – Students will be able to write a persuasive paper to

encourage the reader to accept a specific course of action or point of view.

• Criteria: – Appropriateness of language for the audience – Plausibility and relevance of supporting arguments. – Level of detail presented – Evidence of creative, innovative thinking – Clarity of expression – Organization of ideas

Rating Scales

• Indicate the degree to which a particular dimension is present.

• Three kinds: Numerical, qualitative, combined qualitative/quantitative

• Numerical Scale

– Numbers of a continuum to indicate different level of proficiency in terms of frequency or quality

Example: No Understanding 1 2 3 4 5 Complete

understanding

No organization 1 2 3 4 5 Clear organization

Emergent reader 1 2 3 4 5 Fluent reader

• Qualitative scale

– Uses verbal descriptions to indicate student performance.

– Provides a way to check the whether each dimension was evidenced.

• Type A: Indicate different gradations of the dimension

• Type B: Checklist

• Example of Type A: – Minimal, partial, complete – Never, seldom, occasionally, frequently, always – Consistent, sporadically, rarely – None, some, complete – Novice, intermediate, advance, superior – Inadequate, needs improvement, good excellent – Excellent, proficient, needs improvement – Absent, developing, adequate, fully developed – Limited, partial, thorough – Emerging, developing, achieving – Not there yet, shows growth, proficient – Excellent, good, fair, poor

• Example of Type A: Checklist

• Holistic scale – The category of the scale contains several criteria, yielding

a single score that gives an overall impression or rating

Example level 4: Sophisticated understanding of text indicated with constructed meaning

level 3: Solid understanding of text indicated with some constructed meaning

level 2: Partial understanding of text indicated with tenuous constructed meaning

level 1: superficial understanding of text with little or no constructed meaning

Example holistic scale

• Analytic Scale

– One in which each criterion receives a separate score.

Example

Criteria Outstanding

5 4

Competent 3

Marginal 2 1

Creative ideas

Logical organization

Relevance of detail

Variety in words and sentences

Vivid images

Rubrics • When scoring criteria are combined with a

rating scale, a complete scoring guideline is produced or rubric.

• A scoring guide that uses criteria to differentiate between levels of student proficiency.

Example of a rubric

Guidelines in creating a rubric

1. Be sure the criteria focus on important aspects of the performance

2. Match the type of rating with the purpose of the assessment

3. The descriptions of the criteria should be directly observable

4. The criteria should be written so that students, parents, and others understand them.

5. The characteristics and traits used in the scale should be clearly and specifically defined.

6. Take appropriate steps to minimize scoring frame

Workshop

• Create a performance based task.

• Indicate the following:

– Nature of the final product

– What students are suppose to do

– Criteria for the marking