bench marking workbook

8/14/2019 Bench Marking Workbook

1/42

OPHA 2004Toronto, Ontario

BenchmarkingMini-Workbook

Charlene BeynonMonique Stewart


2/42


3/42

OPHA 2004

If We Knew Then What We Know Now:Benchmarking in the Real World

Dear Colleague:

The enclosed resources represent work in progress and are based on our lived experiencesfrom a number of public health benchmarking projects If only we knew then what we know now . We wish to express our sincere thanks and appreciation to those colleagues who haveparticipated in the benchmarking projects and have assisted us in articulating many lessonslearned and developing these resources.

Our intent is to offer resources and practical tools which will make the implementation of benchmarking in public health more meaningful, effective, efficient and successful in identifyingbest practices.

We do hope that this Mini-Workbook will: enhance your understanding of benchmarking encourage you to explore the application of benchmarking in your work setting assist you in seeking out benchmarking partners highlight common pitfalls and identify strategies to lessen their potential impact encourage you to share your results on the Ontario Public Health Benchmarking

Partnership Website ( www.benchmarking-publichealth.on.ca )

We welcome your ongoing comments and feedback as we move closer to identifying what arethose best practices in using benchmarking to enhance the practice of public health. Weinvite you to share your experiences. May we move closer to realizing the full potential of thisquality improvement tool.

Happy benchmarking!

Until next time,

Charlene BeynonPHRED Program Director Middlesex-London Health Unit50 King StreetLondon, OntarioN6A 5L7Phone: 519-663-5317 ext. 2484Email: [email protected]

Monique StewartPHRED Program Director Ottawa Public Health495 RichmondOttawa, OntarioK2A 4A4Phone: 613-724-4122 ext. 23467Email: [email protected]
http://www.benchmarking-publichealth.on.ca/mailto:[email protected]:[email protected]://www.benchmarking-publichealth.on.ca/


4/42


5/42

OPHA 2004Benchmarking Mini-Workbook

Table of Contents

Presentation: If We Knew Then What We Know Now: Benchmarking in the Real World...1

Benchmarking Code of Conduct.............................................................................................5

Ontario Public Health Benchmarking Partnership: Benchmarking at a Glance .................7

Deciding What to Benchmark..................................................................................................9

A Benchmarking Checklist ....................................................................................................11

Prioritizing Program Components for Benchmarking.........................................................13

Recommended Resources.....................................................................................................15

Tools........................................................................................................................................19

Benchmarking Work Sheets ..............................................................................................21

Selecting Our Options/Increasing the Odds.......................................................................25

Assessing Feasibility of Success.......................................................................................27Draft Project Discussion Points..........................................................................................29

Southwest Benchmarking Feasibility Assessment.............................................................31

Personal Notes .......................................................................................................................33


6/42


7/42

BenchmarkingBenchmarkingThe process of identifying, sharing,understanding, learning from and adaptingoutstanding or best practices fromorganizations anywhere in the world in thequest for continuous improvement andbreakthroughs.

(APQC Benchmarking Code of Conduct, 2002)

Another DefinitionAnother DefinitionThe process of consistently researching for newideas for methods, practices, processes

Either adopting the practices or adapting the goodfeatures and implementing them to become thebest of the best.

Balm, 1992

Relevance to Public HealthRelevance to Public Healthimproves quality of service and program deliveryeliminates the tendency to re-invent the wheel byrecognition and sharing of informationcost savings from improved practices (financial savings)support creative initiativesfacilitates communication, team building & networkingpromotes accountability

Goal of BenchmarkingGoal of BenchmarkingCapture comparable data in order todeduce meaningful comparison onperformance between organizations for thepurpose of inspiring improvement andevaluating performance.

If Only We Knew Then,If Only We Knew Then,What We Know Now:What We Know Now:BenchmarkingBenchmarking IInn TheThe

Real WorldReal WorldCharlene Beynon, Middlesex-London PHRED ProgramMonique Stewart, Ottawa PHRED ProgramMichelle Sangster Bouck, Middlesex-London PHRED Program

55 th Annual Ontario Public Health Association Conference

OPHA 2004OPHA 2004Learning ObjectivesLearning Objectives

Thinking SmartCritically reflect on current practice, andincorporate research and evaluation findings toimprove clients services

Ends in ViewPromote benchmarkingExplore common pitfalls and Critical SuccessFactors

A collaborative initiative involving:Public Health Research, Education &Development (PHRED)The Association of Local Public Health Agencies(alPHa)The Ontario Council on Community HealthAccreditation (OCCHA)

Ontario Public HealthOntario Public HealthBenchmarking PartnershipBenchmarking Partnership

Pilot Projects (1998Pilot Projects (1998 --1999)1999)Pilot project done to address the following: What is benchmarking? Relevance of benchmarking in public health

3 Pilot projects: Immunization Record Processes Food Premises Inspection Partner Notification for Chlamydia

1


8/42

Lessons Learned fromLessons Learned fromPilot ProjectsPilot Projects

Requires many steps, patience andcommitmentData collection not standardizedUse data that are available and easilyretrievable

Keep the indicators simpleBeware of seasonal realitiesRemember contextResource intensiveDetermining best practices is challenging!

www.benchmarking-publichealth.on.caWeb-BasedHealth Units enter own data

Select comparator Health UnitsSelect basis for comparisonProgram picks three bestCan browse through practices related todifferent indicators

Public Health BenchmarkingPublic Health BenchmarkingWeb SiteWeb Site

Survey of PilotSurvey of PilotProject ParticipantsProject Participants

Overall, participants were very positiveAcknowledged role of PHRED and OPHBP inproviding support, expertise & coordinationBenchmarking process created networkingopportunitiesNeed to keep benchmarking process simpleSome specific program changes were made or a conscious decision was made not to change2 areas of concern: anonymity & comparabilityParticipating in projects had a positive influenceon participants practice

A case study:A case study:Dental screeningDental screening

benchmarkingbenchmarkinginvestigationinvestigation

Benchmarking ProjectsBenchmarking Projects9 benchmarking projects completed or

in progress 3 Pilot Projects Breastfeeding Supports Heart Health Coalitions School Health Universal Influenza Immunization Dental Screening West Nile Virus

Our ReportOur Report

Dental Benchmarking Project: Report 1:Descriptive Characteristics of DentalScreening Programs in 10 Ontario PublicHealth Units

www.benchmarking-publichealth.on.cawww.phred-redsp.on.ca

Participation of Health UnitsParticipation of Health UnitsBenchmarking Project Development

32 Health Units have participated in at leastone, and 24 have participated in more thanone

Benchmarking Website34 Health Units have completed at leastone on-line survey, and 28 have completedmore than one on-line survey

10 Dental sites10 Dental sites

Haliburton-Kawartha-Pine RidgeHamiltonHastings & Prince EdwardLeeds-Grenville-LanarkMiddlesex-LondonNiagaraPeelSimcoeWaterlooWellington-Dufferin-Guelph

2


9/42

Our MiddlesexOur Middlesex --LondonLondonproject teamproject team

Charlene BeynonJoan CarrothersMeizi HeBernie LueskeRuth Sanderson

Consultant:Monique Stewart, Ottawa PHRED Program

Strategic QuestionsStrategic Questions

Necessaryprerequisites

Advancingbenchmarking

in public health

If we knew then,If we knew then,what we know nowwhat we know now

-- GO for it!GO for it!

The Lived ExperienceThe Lived Experience

From the eyes of the project team:

focus and limit scope

take time to develop a project planknow the programidentify a few key indicatorscritically examine the data informationsystems

Key QuestionsKey Questions

What is the performance issue? Is benchmarking the right tool? What level of benchmarking is required?

Does the issue justify the investment?

Is there sufficient variability?

Is there a commitment to change practice?

And more lived experiencesAnd more lived experiences

ensure availability of quality datadevelop a data collection tool fit for thetaskdevelop your analysis planprovide specific directions re dataextractionpilot / revise toolsview benchmarking as an iterative process

A Closer LookA Closer Look

Identify the benchmarking question.

Estimate time required.

Build the project team

Recruit partners to ensure comparability

Assess data availability/quality

Dental ScreeningDental ScreeningBenchmarking OutputsBenchmarking Outputs

rich description

reflection & dialogue

learning about the process and theprogram

local program changes

3


10/42

Best PracticesBest Practices

Definitionprocesses that yield better outcomes

effectiveness

less resources not moreefficiency

Question: Why are they successful?

Best PracticesBest PracticesFor next time:

identify at the outset how intend to definebest practiceswill the indicators and data support thedefinition?

For Further Information:For Further Information:

Charlene BeynonMiddlesex-LondonPHRED Program

519-663-5317ext. 2484

[email protected]

Monique StewartOttawa

PHRED Program

613-724-4122ext. 23467

[email protected]


Caveatsmisnomer unlikely one best practiceprerequisite practicesimportance of comparatorscustomizationtime limited


What is the benchmark?is there an industry standard?data sources, e.g. RRFSS, CCHSliteratureexpert opinion

Key MessagesKey Messages

Benchmarking is a quality improvement toolthat can promote exemplary performanceand demonstrate accountability

Success is dependant on many factorsComparable quality data is a prerequisite!

Potential is tremendous

The Question:The Question:Is there a better way?Is there a better way?

collect the indicator data first?

then collect the practice data based onindicator results?

need to know need to know vsvs nice to know nice to know

4


11/42

ONTARIO PUBLIC HEALTH BENCHMARKING PARTNERSHIP(OPHBP)

The Ontario Public Health Benchmarking Partnership is a collaborative initiative of the PublicHealth Research, Education & Development (PHRED) Programs, the Association of LocalPublic Health Agencies (alPHa), and the Ontario Council of Community Health Accreditation(OCCHA).

MISSION : To promote and facilitate the development and sustainabilityof benchmarking in Ontario public health units.

BENCHMARKING : The process of consistently researching for new ideas for methods, practices, and processes; and of either adopting

the practices or adapting the good features, andimplementing them to become the best of the best. (Balm,1992, p. 16)

CODE OF CONDUCT

PURPOSE: To address the appropriate behaviour for all participants involved inbenchmarking in Ontario public health units through the Ontario Public HealthBenchmarking Partnership (OPHBP).

To contribute to efficient, effective and ethical benchmarking, public health units and all other individuals /partners agree to abide by the following principles for benchmarking:

1. Principle of Information Exchange:Be willing to provide the same level of information that you request, in any benchmarkingexchange.

2. Principle of Use:Use information obtained through the benchmarking partnership only for the purpose of improvement of operations within the partnering health units themselves. External use or communication of a benchmarking partners name with their data of results and/or observed practices requires the permission of that health unit.

3. Principle of Preparation:

Demonstrate commitment to the efficiency and effectiveness of the benchmarking processwith adequate preparation at each process step.

4. Principle of Contact:Initiate contacts, whenever possible, through the benchmarking contact person designatedby the participating health unit /organization. Obtain mutual agreement with the contact onany hand-off of communication or responsibility to others. Obtain a partners permissionbefore providing their name in response to a contact request.

5


12/42

5. Principle of Confidentiality:Treat any benchmarking interchange as something confidential to the individuals andorganizations involved. Information obtained must not be communicated outside theOntario Public Health Benchmarking Partnership without prior consent of participatingbenchmarking partners. An organizations participation in a benchmarking project shouldnot be communicated to any third party without their permission.

ETIQUETTE AND ETHICS:The Ontario Public Health Benchmarking Partnership believes participation in thebenchmarking process is based on openness and trust. The following guidelines apply to allparticipants in the benchmarking process:

Treat information obtained through participation in the benchmarking process as internalprivileged information.

Do not disparage a participating health units business or operations to a third party. Enter into each benchmarking project with a mutual goal of improvement.

BENCHMARKING PROTOCOL Know and abide by the Benchmarking Code of Conduct. Have basic knowledge of benchmarking and follow the benchmarking process. Have determined what to benchmark, identified key performance variables, recognized

superior performing partners and completed a rigorous self-assessment. Have developed a questionnaire and will share this in advance if requested. Have the authority to share information. Work through a specified host and mutually agree on scheduling and meeting

arrangements.

Face-to-face meeting guidelines: Provide meeting agenda in advance. Be professional, honest, courteous and prompt. Introduce all attendees and explain why they are present. Adhere to the agenda; maintain focus on benchmarking issues. Use language that is universal. Do not share proprietary information without prior approval from the proper authority of

all participants. Share information about your process(es) if asked, and consider sharing study results.

SUMMARY: BENCHMARKING CODE OF CONDUCT

Be willing to give what you get.Respect confidentiality.Keep information internal.

Dont refer without permission.Be prepared at initial contacts.

Source/Adapted from : The Electronic College of Process Innovation. The Benchmarking Code of Conduct.(http://www.dtic.mil/c3i/bpred/0057.htm )

Reference : Balm, G.J. (1992). Benchmarking: A practitioners guide for becoming and staying the best of thebest. Schaumberg, Illinois: OPMA Press

6
http://www.dtic.mil/c3i/bpred/0057.htmhttp://www.dtic.mil/c3i/bpred/0057.htm


13/42

OO NNTTAARR IIOO PP UUBBLLIICC HHEE AALLTTHH BBEE NNCC HHMMAARR KKIINNGG PP AARR TTNNEE RR SS HHIIPP BBEE NNCC HHMMAARR KKIINNGG AATT AA GG LLAANNCC EE

Benchmarking:the process of consistently researching for new ideas for methods, practices, processes; and of either adopting the practices or adapting the good features and implementing them to become the best of the best.(Balm 1992, p. 16)

A Seven Step Benchmarking Process:Identify what needs to be benchmarked

Determine performance measures, collect and analyze internal data

Select benchmarking partners, e.g. comparator health units

Access data from benchmarking partners

Identify best practices and determine performance gaps

Implement an action plan

Monitor results and recalibrate benchmarks

. . . and the cycle continues (Sales & Stewart, Benchmarking Tool Kit,

What factors influence your decision to initiate a benchmarking investigation?

Some factors to consider: Is this program

benchmarkable? Can youmap out program components?

Are other organizationsexperiencing better results?

Do the differences in resultsmerit the investment of resources?

Is this issue an organizationalpriority? Are there more urgentpriorities?

Is there political/communitypressure to do thingsdifferently?

What questions(s) do you wantanswered? Need to bespecific

What processes are used byother sites that achieve better results (important to know process sometimes articulating your process will highlightwhere changes are indicated)

Other issues to explore:

What level of data collection is needed? Is the information you need to answer your question(s) already available, e.g. literature

searches, systematic reviews, and if so are the findings transferable to your setting or aremodifications required? Do you have enough information to make and support arecommendation? If so, develop, implement and evaluate an action plan

7


14/42

OR do you need to initiate a more comprehensive benchmarking investigation?

Additional factors to consider:

Determine organizational readiness to proceed.Is there:

management and organizational support, including commitment, dedicated time and

budget? involvement of front line staff? a benchmarking team with the necessary commitment and expertise? an environment conducive to change, i.e. dependent on findings are you willing to introduce

a new program or to change a program component? a reasonable timeline?

If so , initiate a benchmarking investigation refer to the above seven-step process.

Remember to keep it manageable.

Factors to consider when selecting benchmarking partners:

Look for organizations that are: recognized for their expertise similar 1 to yours, e.g. serve a similar population, e.g. rural/urban, age distribution,

socioeconomic status, staffing mix, geography, etc. are accessible, e.g. site visit, conference calls willing to share information and to participate

1 Can be from a different sector but offer a similar service, e.g. community health centre, call centre.

The number of partners will depend on your benchmarking question, timeline and resources available.

For Additional Information - see the Benchmarking Tool Kit

References:Balm, G. J. (1992). Benchmarking: A practitioners guide for becoming and staying best of the best.Schaumburg, Illinois: QPMA Press

Sales, P. D., & Stewart, P. J. (1998). Benchmarking Tool Kit: a blueprint for public health practice.PHRED Program: Middlesex-London Health Unit, Ottawa-Carleton Health Department.

Source:Beynon, C. (1999). Benchmarking. From Regional Workshops, Public Health Needs, EffectiveInterventions, Benchmarking: Implications for Public Health Units

Recommended Readings:Ammons, D. N. (2001). Municipal benchmarks: Assessing local performance and establishingcommunity standards. California: Sage Publication Inc.

Keehley, P., Medlin, S., MacBride, S., and Longmire, L. (1997). Benchmarking for best practices in thepublic health sector: Achieving performance breakthroughs in federal, state and local agencies. SanFrancisco: Jossey-Bass.

November 7, 2001/Reviewed: November 2004 Charlene Beynon, Middlesex-London Health Unit

8


15/42

Deciding What to Benchmark?

The process should:

be meaningfulbe highly visible

be resource intensivehave a history of problemshave the opportunity to improvehave an environment conducive to changebe understoodsupport the agency mission, vision and strategic directions

need ideas from other sources to be improved

Is this process worth benchmarking at this time?

- Keehley, 1997, pp 87-88

9


16/42

10


17/42

A Benchmarking Checklist

Before embarking on a benchmarking investigation assess the following and determine thelikelihood of a successful outcome.

Key Decision Points Yes No1. A S.M.A.R.T. 1. benchmarking question can be identified?

2. There is a measure to determine what is best?

3. Performance improvement is possible?

4. There is a willingness to make changes?

5. There is a sustained commitment to participate for the projectsduration?

6. A detailed program description addressing how the program isdelivered can be articulated for each comparators program?

7. A few key indicators (effectiveness and efficiency) can be identified?

8. Required data is available?

9. If data not available, it is feasible with current resources to retrieve or collect?

10. Data collection can be standardized?

11. Benchmarking expertise is available?

12. There are sufficient resources, e.g. staffing, budget, etc to sustain theproject?

1.S = Specific M = Measurable A = Achievable R = Realistic T = Timeline

This Checklist has not been validated. It has been developed from experiences gained frombenchmarking public health programs and is intended to stimulate strategic dialogue anddecision-making with an overall goal of increasing the likelihood of a successful outcome.

For further information contact:Charlene BeynonMiddlesex-London Health [email protected] 519-663-5317 ext. 2484

11
mailto:[email protected]:[email protected]


18/42

12


19/42

Prioritizing Program Componentsfor Benchmarking

Criteria Rationale ScoringEase of datacollection

New data collection for the solepurpose of benchmarking is abarrier to implementation

+ no new data collection required0 minor effort will likely be

required- substantial effort required for

new data collection

Data collection tool Drafting and piloting of

measurement tools has potentialto be resource and timeconsuming

+ data collection tool(s) available

0 minor tool creation required- substantial effort likely requiredto develop tools

Sufficient programimplementation

Can not benchmark an activitythat is not being done

+ component activelyimplemented

0 some implementation- little or no implementation

Sufficiently similar programimplementation

Mandatory Program requirementsleave discretion regardingapproaches. Processes need tobe sufficiently similar to allowcomparison

+ programs highly similar 0 similar - large divergence in approaches

Resourcecommitment

The more resources arecommitted to a component, more

likely willing to engage inbenchmarking effort

+ significant resources0 moderate resources-

little or not resourcescommitted

Source: Ontario Public Health Benchmarking Partnership

13


20/42

14


21/42

RECOMMENDED RESOURCES

www.benchmarking-publichealth.on.ca

BENCHMARKING :

Ammons, D.N. (2001). Municipal benchmarks: Assessing local performance and establishingcommunity standards. Thousand Oaks: Sage Publication Inc.

Ammons, D.N. (1996). Municipal benchmarks: Assessing local performance and establishingcommunity standards. Thousand Oaks: Sage Publication Inc.

Balm, G.J. (1992). Benchmarking: A practitioners guide for becoming and staying best of thebest. Schaumburg, Illinois: QPMA Press.

Beynon, C. & Wilson, V. (1998). Benchmarking in Public Health: An idea whose time hascome. Public Health & Epidemiology Report Ontario, 9(7), 162-163.

Bolan, S. (2001). Competitive calibration. Computing Canada, 27(10), p. 25.

Camp, R. C. (1989). Benchmarking: The search for best practices that lead to superior performance. Part I: Benchmarking defined. Quality Progress, January, 61-68.

Camp, R. C. (1989). Benchmarking: The search for best practices that lead to superior performance. Part II: Key process steps. Quality Progress, February, 70-75.

Camp, R. C. (1989). Benchmarking: The search for best practices that lead to superior performance. Part III: Why benchmark?. Quality Progress, March, 61-68.

Codling, S. (1996). Best Practice Benchmarking: An International Perspective. Texas: Gulf Publishing Company.

Dattakumar, R. & Jagadeesh, R. (2003). A review of literature on benchmarking.Benchmarking, 10(3), 176-209.

Davies, A. J. & Kochhar, A. K. (2000). A framework for the selection of best practices.International Journal of Operations and Production Management, 20(10), 1203.

Doebbeling, B.N., Vaughn, T.E., Woolson, R.F., Peloso, P.M., Ward, M.M., Letuchy, E., Boots-Miller, B.J., Tripp-Reimer, T., & Branch, L.G. (2002). Benchmarking veterans affairs medicalcenters in the delivery of preventive health services. Medical Care, 40(6), 540-554.

Dunkley, G., Stewart, M., Basrur, S., Beynon, C., Finlay, M., Reynolds, D., Sanderson, R., &Wilson, V. (2001). Benchmarking in public health. Public Health & Epidemiology ReportOntario, 12(6), 211-215.

Dunkley, G., Wilson, V., & Stewart, M. (2000). Benchmarking Pilot Project: Testing theconcept in public health. Public Health & Epidemiology Report Ontario, 11(1), 14-21.

15
http://www.benchmarking-publichealth.on.ca/http://www.benchmarking-publichealth.on.ca/


22/42

Ellis, J., Cooper, A., Davies, D., Hadfield, J., Oliver, P., Onions, J. & Walmsley, E. (2000)Making a difference to practice: Clinical benchmarking part 2. Nursing Standard, 14(33), 32-35.

Fitz-Enz, J. (1992). Benchmarking best practices. Canadian Business Review, 19(4), 28-31.

Gunasekaran, A. (2001). Benchmarking tools and practices for twenty-first centurycompetitiveness. Benchmarking, 8(2), 86-87.

Herman, R. C. & Provost, S. (2003). Interpreting measurement data for quality improvement:Standards, means, norms, and benchmarks. Psychiatric Services, 54(5), 655-657.

Johnson, B., & Chambers, J. (2000). Food service benchmarking: Practices, attitudes andbeliefs of foodservice directors. The American Dietetic Association, 100(2), 175-180.

Keehley, P., Medlin, S., MacBride, S., & Longmire, L. (1997). Benchmarking for best practicesin the public health sector: Achieving performance breakthroughs in federal, state and localagencies. San Francisco: Jossey-Bass.

Loveridge, N. (2002). Benchmarking as a quality assessment tool. Emergency Nurse, 9(9),24-29.

Mancuso, S. (2001). Adult-centered practices: Benchmarking study in higher education.Innovative Higher Education. 25(3), 165-181.

Ossip-Klein, D.J., Karuza, J., Tweet, A., Howard, J., Obermiller-Powers, M., Howard, L., Katz,P., Griffin-Roth, S., & Swift, M. (2002). Benchmarking implementation of a computerizedsystem for long-term care. American Journal of Medical Quality, 17(3), 94-102.

Sales, P.D., & Stewart, P.J. (1998). Benchmarking Tool Kit: A blueprint for public healthpractice. Middlesex-London and Ottawa-Carleton Public Health Research, Education andDevelopment (PHRED) Programs.

Tepper, D. (2002). Benchmarking: Measuring productivity and outcomes. PT-Magazine of Physical Therapy, 10(1), 48-52.

Vassallo, M.L. (2000). Benchmarking and evidence-based practice: Complementaryapproaches to achieving quality process improvement. Seminars in Perioperative Nursing,9(3), 121-124.

Wilson, B. & Beynon, C. (1998). Introducing benchmarking to Ontario Health Units. Public

Health & Epidemiology Report Ontario, 9(8), 183-186.Witt, M. J. (2002). Practice re-engineering through the use of benchmarks: Part II. MedicalPractice Management, March/April, 237-242.

Zairi, M. (1998). Benchmarking for Best Practice. London: Butterworth-Heinemann.

Zairi, M. & Leondard, P. (1994). Practical Benchmarking: The Complete Guide. London:Chapman & Hall.

16


23/42

P ROGRAM LOGIC MODELS :

Dwyer, J. (1996). Applying program logic model in planning and evaluation. Public Health &Epidemiology Report Ontario, 17(2), 38-46.

Rush, B. & Ogborne, A. (1991). Program logic models: Expanding their role and structure for program planning and evaluation. The Canadian Journal of Program Evaluation. 6(2), 95-106.

McLaughlin, J.A., & Jordan, G.B. (1999). Logic models: A tool for telling your programsperformance story. Evaluation and Program Planning, 22(1), 1-14.

WEBSITES FOR P ROGRAM LOGIC MODELS : www.benchmarking.co.uk www.bja.evaluationwebsite.org/html/roadmap/basic/program_logic_models http://garberconsulting.com/Program_Logic_Model.htm

www.uottawa.ca/academic/med/epid/toolkit.htm

ONTARIO P UBLIC HEALTH BENCHMARKING P ARTNERSHIP D OCUMENTS & R EPORTS :

see - www.benchmarking-publichealth.on.ca (documents & reports) Benchmarking: Breastfeeding Support in Public Health December 2000 Towards Benchmarking Heart Health Coalitions: Developing A Systematic Process for

Documenting and Enriching Community/Health Unit Partnerships April 2001 Benchmarking in Public Health in Ontario: Web-Site Users Manual (2 nd Ed.) July 2002

Charlene BeynonMiddlesex-London Health UnitPHRED [email protected] 519-663-5317 ext. 2484

Monique StewartCity of Ottawa Public Health & Long-TermCare BranchPHRED [email protected] 613-724-4122 ext. 23467

November 7, 2001/Updated: November 2004

17
http://www.benchmarking.co.uk/http://www.bja.evaluationwebsite.org/html/roadmap/basic/program_logic_modelshttp://garberconsulting.com/Program_Logic_Model.htmhttp://www.uottawa.ca/academic/med/epid/toolkit.htmhttp://www.benchmarking-publichealth.on.ca/mailto:[email protected]:[email protected]:[email protected]:[email protected]://www.benchmarking-publichealth.on.ca/http://www.uottawa.ca/academic/med/epid/toolkit.htmhttp://garberconsulting.com/Program_Logic_Model.htmhttp://www.bja.evaluationwebsite.org/html/roadmap/basic/program_logic_modelshttp://www.benchmarking.co.uk/


24/42

18


25/42

Tools

The following resources were developed from lessons learned from the Dental ScreeningBenchmarking Investigation to assist future project teams in :

determining if benchmarking is the tool of choice, and mapping out pre-requisites for a successful benchmarking investigation.

These resources were used to facilitate discussion with colleagues in the Southwest todetermine the feasibility of conducting a benchmarking investigation focusing on postpartumdepression.

o Benchmarking Work Sheetso Selecting Our Options/Increasing the Odds

o Assessing Feasibility of Success

o Draft Project Discussion Points

o Southwest Benchmarking Feasibility Assessment

We look forward to improving these resources and welcome your questions and comments.

19


26/42

20


27/42

Benchmarking Worksheets

1. Identify major components for a Program Logic Model.

2. What component do you want to benchmark? i.e. Where do you need to improve performance? Aimprove performance?

3. What is your benchmarking question?

We want to benchmark:

21


28/42

4. What is the benchmark? (i.e. What is the gold standard? What will you compare your results to in order to identPractices?)

5. Develop a Program Logic Model for the component to be benchmarked.

Component

Activities

Target Audience

Short term Outcomes

Long Term Outcomes

22


29/42

6. Identify a maximum of 3 4 indicators

Principle: include both effectiveness and efficiency indicators.

Does it work? Is it worth doing?

Indicators Data Sources R

1.

2.

3.

4.

An indicator:a quantifiable measure of an outcome or objective

For further information contact:Charlene BeynonMiddlesex-London Health Unit50 King Street, London, Ontario, N6A [email protected] /519-663-5317 ext. 2484

23
mailto:[email protected]:[email protected]


30/42

24


31/42

Benchmarking Worksheet

Selecting Our Options/Increasing the Odds

ProgramComponent 1.

What is thebenchmarkingquestion?

What is thebenchmark?

What are theindicators?

Can we get thedata?

Other

Post Note:The goal is to focus on only one program component. Other components can be addressed in subsequent benchmarkinvestigations. If there is debate about which component should be addressed, this Work Sheet can assist in articulatfocus is more likely to yield a successful benchmarking outcome.

November 2003 Revised: November 2004

1. Program Logic Model

25


32/42

26


33/42

BenchmarkingAssessing Feasibility of Success

Worksheet

Component to bebenchmarked

Ease of Data Collection

Data Collection Tools

Sufficient Program Implementation

Sufficiently Similar Implementation

Resource Commitment

27


34/42

28


35/42

Postpartum DepressionBenchmarking Investigation

Draft Project Discussion Points

1. Identify major components for a Program Logic Model.

2. What component do you want to benchmark?Principle: Select only 1

3. What benchmarking question do you want to answer?Principle: the question should be SMART a Question: is there a benchmark?

4. Initiate discussion about how intend to identify Best Practices.

5. What data is required to answer the question?

Question: is the data easily retrievable?6. Identify a maximum of 3-4 indicators.

Principle: include both effectiveness and efficiency indicators7.. Review Benchmarking Checklist.

Decision Point: i) continue, ii) different action, iii) stop8. Develop questionnaire or other data collection tool to collect indicator data

only*.

9. Pilot the questionnaire and revise as needed. Several iterations may berequired.

10. Collect indicator data.11. Review findings with practitioners and other recognized experts.12. Collect practice data to link with best results.13. Identify Best Practices.14. Implement.15. Evaluate.16. Disseminate experiences.17. Monitor results.

a = S-Specific, M-Measurable, A-Achievable, R-Realistic, T-Timeline

* Usually indicator and practice data are collected simultaneously. This option requires further study. It is being presented as one way to keep the project manageable, time limited and isbased on experiences from other projects.

Although represented as a series of linear steps, benchmarking is an interactive process

November 2003 Revised: November 2004

29


36/42

30


37/42

Postpartum DepressionSouthwest Benchmarking Feasibility Assessment

The Question: is benchmarking the right tool to answer the question(s) that needs to beanswered

When to Benchmark*:

1. Is there a performance issue?

2. Is someone doing better? Is it really better? Compared to what?

3. Can we identify and compare practices

4. Are the practices transferable and able to be customized to other settings?

* see the Benchmarking Checklist and Priority Program Components for Benchmarking

Key Phrases: Who is doing better at. . . (identifying women at risk for postpartum depression)?

What practices are most effective and efficient. . .(identifying women at risk for postpartumdepression)?

type of contact with what tool? when?

November 2003

31


38/42

32


39/42

Personal Notes

33


40/42

34


41/42

35


42/42

bench marking workbook

Documents