Claire Palermo (Monash University)Susan Ash (Queensland University of Technology) Sandra Capra (The University of Queensland)Helen Truby (Monash University)Eleanor Beck (Dietitians Association of Australia, University of Wollongong) Brian Jolly (University of Newcastle, formally at Monash University)

Download this resource as a PowerPoint: http://www.med.monash.edu.au/scs/nutrition-dietetics/research.html

ISBN print: 978-1-74361-386-3; ISBN PDF: 978-1-74361-387-0; ISBN DOCX: 978-1-74361-388-7.

Competency Assessment in Nutrition and Dietetics

The CAND Resource

Who produced this resource? The project team for the project, Professional Competence Standards, learning outcomes and assessment: Designing a valid strategy for nutrition and dietetics, which comprised: Dr Claire Palermo, Professor Susan Ash, Associate Professor Eleanor Beck, Professor Sandra Capra, Professor Helen Truby and Professor Brian Jolly. The project was lead by Monash University in partnership with Queensland University of Technology, The University of Queensland and the Dietitians Association of Australia (DAA).

Who funded this resource? Funding for the project and this resource was provided by the Australian Government Office for Learning and Teaching. The views expressed in this report do not necessarily reflect the views of the Australian Government Office for Learning and Teaching.

AcknowledgementsSimone Gibson; Janeane Dart; Judi Porter; Bethany Whitcher; Jorja Collins

Who should use this resource? This resource is intended for use by those involved in the assessment of entry-level nutrition and dietetics practitioners. It is provided under Creative Commons Attribution-ShareAlike 4.0 International Licence.

How this resource should be used This resource is intended to be used to guide professional development for individuals or groups. It includes information as well as activities to assist users to engage in the content. Look out for these icons:

Theory Overview Video clip Activity More information

Aims of this resourceThe aim of this resource is to develop the skills of academics and educators/supervisors involved in assessment.

Learning objectivesAfter working through this resource you will be able to: 1. Discuss the principles of assessment for quality teaching and

learning in nutrition and dietetics.2. Distinguish between competency-based and performance-based

assessment.3. Explain the components of best practice assessment methods for

the health professions.4. Critically reflect on current approaches to assessment design and

delivery and provide a framework for the evaluation of existing practices.

Activity 1 – Multiplication A student was given a multiplication task and worked it out as follows:

Multiplication Task: 5220 x 1.4 =

Student’s Answer: 5220 X 1.4208805220072080 = 7208

Correct Answer: 5220 x 1.4 = 7,308

Think about how you would assess this task

Activity 1 – MultiplicationDiscuss What mark would you give the student out of 10 for completing

this question? If you are doing this activity as a group, plot marks from 0 to 10

noting how many people gave 10, 9 etc. How variable are the marks?

Why did you give the mark you gave? What factors are important? or What factors will assist in

getting more consistent score of assessor? If this question was contextualised in estimating energy

requirements for a 35 year old female requiring enteral nutrition where BMR was 5220 and Stress factor was 1.4, how would this change how you mark this?

Activity 1 – Multiplication

Discussion points Context and the implications of making the mistake Skill or competence of assessor Standard criteria in relation to what you would expect Purpose of assessment Expected level of student Judgment on small sample of activity – it is only one question Focus of assessment e.g. Testing multiplication versus estimating

energy requirements

Key elements of assessmentThis activity illustrated some key elements of assessment which include:

Reliability in the context of educational assessment refers to the precision or reproducibility of the examination.

Validity refers to what it is that is being assessed and can also be categorised into many sub-groups.

These are important elements to consider when designing assessment systems for curricula.

Eraut’s Model of Curriculum Michael Eraut, a curriculum theorist, has done considerable work studying professional curricula in architecture, medicine, nursing, teaching and other areas.

The aims or drivers of curriculum effect all the other elements in some way.

His model shows the interrelatedness of the elements – aims, objectives/outcomes, teaching and learning/communication, assessment and the subject matter/content.

Eraut’s Model of Curriculum

Discuss the interrelatedness of the elements of the model

for taking a dietary history

Activity 2 – Eraut’s Model

Activity 3 – Curriculum drivers in dietetics

Discuss What influences your nutrition and dietetics curriculum? What are your nutrition and dietetics curriculum aims? How would you describe the key features of your curriculum? How does this influence your assessment?

Competency standards for dietitiansCompetencies have been used to in dietetics since as early as 1976.

The Dietitians Association of Australia National Competency Standards for Entry-level Dietitians (2009) provide the framework and benchmark for the preparation of dietitians for entry into the workforce in Australia. These standards describe what a graduate should be able to do upon completion of an accredited degree in nutrition and dietetics in order to be able to enter the workforce.

The Competency Standards are made up of: Units: work roles expressed in outcome terms Elements: tasks to fulfil these work roles Performance criteria: how to measure if the

roles have been fulfilled

Click here to see the standards

Professional competenceProfessional competence has been defined as “the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values and reflection”

Competency standards provide a focus on observable outcomes of professional competence.

Competency based education is based on the progressive development of the knowledge, skills, attitudes, attributes, values (competency standards) that are required for professional competence.

Competency based assessment is robust work-based assessment that emphasizes what trainees will ultimately do in their profession

Advantages and disadvantages of competency standards Competency standards for health professions have a number of benefits: They provide a focus on observable outcomes Are relevant to the workplace Enable delineation of curricula Define readiness for practice Provide a framework for assessment

The majority of health professions have developed entry-level competency standards, including dietetics. Despite this, the competencies-based approach to health professional education has been criticised for simplifying education and learning and reducing the integrity of professional practice, especially of making complex decisions.

Competency based education The novice-to-expert continuum describes the gradual process of competency development from student (novice) to competent (entry-level) through to expert (advanced).

Competency based education needs strong assessment and feedback systems

Novice

Beginner

Competent

ProficientExpert

Entry level education

Practice and continuing professional development

Competency based assessmentTraditionally competency development has been described as a process from knowing through to doing.

Figure 1. Competence and Performance. Millers Pyramid

Performance(does)

Competence (shows how)

Knows how

Knows

E.g. Multiple choice questions, case presentations

E.g. Objective Structure Clinical Exam (OSCE), Simulated Patients

E.g. Function on professional placement

Competency based education in dietetics Competency based education in nutrition and dietetics occurs through the gradual development of competencies ideally leading to quality practice in improving the nutritional health of individuals and populations.

While the competency standards provide an effective scaffold for assessment of the practice of nutrition and dietetics, they should not be viewed as a ‘tick box’, rather as a framework to: Guide the process of development of a professional Describe curricula Define readiness for practice Provide a framework for assessment

When designing assessment … Include the following:

Competency-based assessment measures what is completed by the student in single point situations e.g. OSCE.

Performance-based assessment or work-based assessment represents what a student does in real life practice where they are “dealing with complexity, uncertainty and continual changes in service provision ethos and practice”. Learners are active participants making several judgments across a range of contexts, dispersed geographically, where the expertise of those making the judgment matters. Authentic assessment in the workplace can be instrumental for learning. Assessment of professional attributes such as empathy, reflection and scope of practice are suited to work-based assessment.

Assessment of learningAssessment must be designed to influence constructive learning. Assessment should facilitate the development of competence and promote a commitment to learning and reflective practice.

For students assessment “provides targets that focus and drive the depth and direction of learning”.

For educators “assessment provides opportunities for feedback to students on current performance, and enables the development of specific strategies to improve performance and achieve learning outcomes”.

Effective assessmentEffective and meaningful assessment must be aligned with learning objectives and teaching methods or content, build on previous learning and assessment and provide students with the opportunity to demonstrate what they have learnt.

The PURPOSE of the assessment will define the content, strategy, and technique used. Most inefficient, inappropriate or unmanageable assessment strategies occur because people have not though enough about the PURPOSE of the assessment.

The most effective way to assess students is through: A number of measurements over time (longitudinal observation) A range of different contexts This approach also improves the validity of judgements based on assessments by reducing error inherent in single-point assessments.

Activity 4 - Effective assessmentsThink about an assessment task you give your students.

Discuss What is the assessment trying to measure? What level? What is being achieved through the assessment process?

(formative, reflective, experiential learning, engagement with learning and thinking)

What is the minimal threshold pass level and how do grades above the pass level show student achievement?

Have students had adequate opportunity to learn the skills and knowledge being assessed?

Acknowledgements Margaret Hay

Assessment in CBEIdeal assessmentsObjectives/Outcomes Teaching & Learning Assessment

“The content and form of assessment need to be aligned with their purpose and desired outcomes”

“Quality is inextricably interwoven with goals, which are closely related to assessment”

Acknowledgements Margaret Hay

Assessment in CBEAssessments in realityObjec ves/Outcomes ≠ Teaching & Learning ≠ Assessment

Assessments in realityWhat we may see in practice is scenarios that are very stressful for learners:

Scenario A Scenario B Scenario C Scenario DNot all stated ‘goals’ or learning content are being assessed

Under assessment

Stated ‘goals’ and learning content are not being assessed

No assessment

Competencies not important

Assessment extends beyond the stated ‘goals’ and the delivered content

Too much assessment

Assessment not all aligned with objectivesor outcomes

Assessment of content that was not taught

Assessment not attached to objectives or teaching

Activity 5 – Assessments in realityIdentify one example of each scenario (A,B,C,D) from the previous slide that you have personally experienced, either as a learner or teacher.

Consider Why do they occur? What is the impact on learners? What is the impact on assessors? What is the impact on the assessment system/program? How can this be avoided?

The Australian Learning and Teaching Council principles of assessment 1. Assessment is used to engage students in learning that is productive2. Feedback is used to actively improve student learning3. Students and teachers become responsible partners in learning and

assessment4. Students are inducted into the assessment practices and cultures of

higher education5. Assessment for learning is placed at the centre of subject and program

design6. Assessment for learning is a focus for

staff and institutional development7. Assessment provides inclusive and trustworthy

representation of student achievement

Click here to see the principles

Five elements of an effective assessment system for nutrition and dietetics

1. Clearly defined performance criteria2. Multiple assessors with training and appropriate skills and

attributes3. Multiple methods over time using tools that facilitate learning4. Include authentic encounters5. Effective Feedback

Click here to see the elements

Activity 6 – Assessment methodsConsider the methods of assessment you currently use.

Discuss Do they align with the principles of assessment and effective

assessment? Can you identify 2-3 key areas for improvement?

Activity 7 – Putting it all togetherWatch the video below featuring a student discussing her food service placement experience with her supervisor. (Click video to play)

Discuss What are the competencies for entry level dietitians in relation to

food service? Do the tasks that the student describes she has undertaken

provide evidence that she has achieved the competencies? Discuss this dialogue as a method of assessment to provide

evidence that the student has achieved the competence.

Activity 8 – Putting it all together

Watch the clips below featuring a dietetic student interviewing a patient and then having two contrasting feedback sessions with her placement educator. (Click videos to play)

Feedback A Feedback B

Activity 8 – Putting it all together

Reflect on the clips and the feedback sessions: What does Feedback A reveal in relation to the students

developing professional competence? What does Feedback B reveal in relation to the students

developing professional competence? Discuss the educators approach in Feedback A and B? How does the educator/assessor influence assessment of the

students performance? What would you recommend following on from Feedback B in

terms of guiding the students progress and further assessment? How do the differing feedback sessions influence your perception

of the students competence?

Activity 8 – Must readsThese articles are recommended for further information on assessment for health professionals:

1. Crossley J & Jolly B. Making sense of work-based assessment: ask the right questions, in the right way, about the right things, of the right people. Medical Education. 2012;46:28-37.

2. Epstein R & Hundert E. Defining and assessing professional competence. Journal of American Medical Association. 2002;287(2):226-35.

3. Norcini J et al. Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 conference. Medical Teacher. 2011;33:206-214.

References Boud, D. and Associates (2010).Assessment 2020: Seven propositions for assessment reform in higher education. Sydney: Australian

Learning and Teaching Council Brownie, S., M. Bahnisch, et al. (2011). Exploring the Literature: Competency-based Education and Competency-based Career

Frameworks: Deliverable fulfilling part of the requirements for NHPRC Projects 4 and 5 regarding frameworks for competency-basededucation, training and health career frameworks. Adelaide, Australia.)

Chambers, D., C. Gilmore, et al. (1996). "Another look at competency-based education in dietetics." Journal of the American Dietetic Association 96(6): 614-617.

Dietitians Association of Australia. (2009). "National competency standards for entry-level dietitians." from www.daa.asn.au. Dijkstra, J., C. Van der Vleuten, et al. (2010). "A new framework for designing programmes of assessment." Adv Health Sci Educ

Theory Pract 15(3): 379–393. Dreyfus, H. and S. Dreyfus (1996). The relationship of theory and practice in the acquisition of skill. Expertise in nursing practice. P.

Benner, C. Tanner and C. Chesla. New York, Springer: 29-48. Epstein, R. and E. Hundert (2002). "Defining and assessing professional competence." JAMA 287(2): 226-235. (p. 226) Eraut M. Developing professional knowledge and competence. London: Falmer Press, 1994. Fish, D. and L. de Cossart (2006). "Thinking outside the (tick) box: rescuing professionalism and professional judgement." Med Educ

40: 403-404Ash S, Phillips S. What is dietetic competence? Competency standards, competence and competency explained. Australian Journal of Nutrition and Dietetics. 2000 57: 147–51.

Holmboe, E., J. Sherbino, et al. (2010). "The role of assessment in competency based medical education." Med Teach 32: 676-682. Jolly, B. (2012). "Shining light on competence." Med Educ 46: 346-348. Keating, J., M. Dalton, et al. (2009). Assessment in clinical education. Clinical Education in the Health Professions. E. Molloy and C.

Delany. Sydney, Elsevier. Miller, G. (1990). "The Assessment of Clinical Skills/Competence/Performance." Acad Med 65 (Supp): S63-67. Morris, C. and D. Blaney (2010). Work-based learning. Understanding Medical Education: Evidence, Theory and Practice. T. Swanwick.

London, UK, Wiley-Blackwell: 69-82. Norcini, J. (2010). Workplace assessment. Understanding Medical Education. The Association for the Study of Medical Education.

London, Wiley-Blackwell: 232-245. Rethans, J., J. Norcini, et al. (2002). "The relationship between competence and performance: implications for assessing practice

performance." Med Educ 36: 901-909. van der Vleuten, C. (1996). The Assessment of Professional Competence: Developments, Research and Practical Implications.

Advances in Health Sciences Education, 1, 41-67.