chapter 6 ppt eval & testing 4e formatted 01.10 mo edits

© 2013 Springer Publishing Company, LLC.

Chapter 6Short-Answer (Fill-in-the-Blank)and EssayOermann & GabersonEvaluation and Testing in Nursing Education4th edition


Short-Answer (Fill-in-the-Blank)

♦ Answered by word, phrase, or number♦ 2 types: question and completion

(fill-in-the-blank)♦ Less affected by guessing than true-false♦ Tests mainly for factual information

2


Writing Short-Answer Items

♦ Word the question to require a single unique answer

♦ Think of intended answer first♦ Avoid clues to correct answer♦ Avoid verbatim statements from a text♦ If using sentence-completion variety, place

blank at the end of the sentence

3


Completion Example

What congenital cardiac defect results in communication between the pulmonary artery and aorta? ____________

4


Completion Example

Three physiological changes that occur with aging are: 1._________2._________3._________

5


Fill-in-the-Blank Example

You are caring for a patient who weighs 156 lb. She is ordered 20 mcg/kg of an IV medication. The correct dose in micrograms is ___________.

6


Essay Items

♦ Requires students to develop their own answers to questions

♦ Opportunity to: – Select content to discuss– Present ideas in own words– Develop original and creative response to question

7


Uses of Essay Items

♦ Testing– Complex learning outcomes– Higher-level learning, not recall– Content not appropriate for objectively

scored items

8


Levels of Essay Items

♦ Low level– Similar to short-answer– Precise answer

♦ ExampleDescribe three signs of increased intracranial

pressure in children under 2 years old.

9


Levels of Essay Items

♦ High level– Responses are not limited as with low level

♦ Example:Defend the statement “Access to health care is a

right.”

10


Types of Essay Items

♦ Restricted response♦ Extended response

→ Difference is based on extent of freedom in responding to essay item

11


Restricted-Response Essay Item

♦ Limit answer by – Specifying content to be discussed– Restricting amount of discussion allowed

• “Answer in one paragraph” or “one page”

12


Examples

1. Define patient-centered care. Limit your response to one paragraph.

2. Compare metabolic and respiratoryacidosis. Include the following in your response: definitions, precipitating factors, clinical manifestations, diagnostic tests, and interventions.

13


Example

3. A patient is 76 years old and 1 day postoperative femoral popliteal bypass graft. Name a major complication the patient could experience at this time. List two nursing interventions for this patient during the initial recovery period with related evidence.

14


Extended-Response Essay Items

♦ Less restrictive♦ Opportunity for students to choose content

for answer, integrate learning from multiple sources, organize ideas

♦ Assess higher-level learning

15


Writing Essay Items

♦ Assess knowledge of content, not writing ability

♦ Frame the problem so that students know what to write about

16


Example

1. Select an article describing a nursingresearch study. Critique the study, specifying the criteria used. Based on your evaluation, describe how the research findings could be used in clinical practice.

17


Example

2. In the clinic you note that patients who are uninsured receive fewer diagnostic tests and less comprehensive care than patients with similar health problems who are insured. Analyze these differences in care and related issues.

18


Issues With Essay Tests

♦ Limited ability to sample course content♦ Low reliability of scoring ♦ Item-to-item carryover effect

– Impression of quality of one response carries over to next question

– Read all responses to one item before evaluating next one

19


Issues With Essay Tests (cont’d)

♦ Test-to-test carryover effect– Impression of student carried from one test to next

♦ Halo effect– Teacher’s general impression of a student

influences scoring of test♦ Essay tests should be scored anonymously

20



♦ Effect of writing ability– Rating influenced by sentence structure, grammar,

spelling, punctuation– Longer answers often receive higher scores

regardless of content

21



♦ Rater drift– Essay tests read early may be scored higher than

those read near the end– Rater drift: stray from scoring criteria– Read papers in random order– Read each answer twice before scoring

22



♦ Usability– Allow more time for students to answer and

teacher to score♦ Student choice of essay items

– May affect measurement validity• Students taking different tests

23


Scoring Essay Items

♦ Two methods– Holistic (global-impression)– Analytical

24


Holistic Scoring

♦ Read entire answer and evaluate its overall quality

♦ Ways to score– Relative standard– Model answer– Scoring rubric

25


Analytical Method

♦ Identify elements required in ideal answer♦ Each element must be included in answer♦ Evaluate and score each element separately

rather than overall response– Scoring rubric

26


Scoring Essay Items

♦ Identify method of scoring♦ Construct ideal answer and how to score it♦ Evaluate:

– Content– Organization– Process

27


Ideal Answer

♦ Write ideal answer while drafting essay item– Save for use in scoring

28


Scoring Essay Items

♦ Check your ideal answer– Read random sample of papers– Score answers to one item at a time

• Compare responses across students

♦ Alter ideal answer and scoring criteria if needed

29


Scoring Essay Items (cont’d)

♦ Use same scoring system for all papers♦ Read each answer twice before scoring♦ Read students’ responses in random order

30


Scoring Essay Items (cont’d)

♦ Read anonymously♦ Cover scores to previous answers♦ If possible, have colleague read and score

– Improves reliability♦ Adopt policy on scoring writing ability

31


Prepare Students for Essay Tests

♦ Ask thought-provoking questions♦ Use sample essay items to review content

before test– Discuss ideal answers with students

32


Prepare Students for Essay Tests (cont’d)♦ Students practice responding to sample items

in small groups– Pass answers to next group to critique– Have students score responses

33