leukemia & lymphoma society (lls) information resource center (irc) planning for a service...
Post on 31-Dec-2015
213 Views
Preview:
TRANSCRIPT
Leukemia & Lymphoma Society (LLS) Information
Resource Center (IRC)
Planning for a Service Program Evaluation - Case Study
Public Administration 522
Presented by Ervina Castillo-Newton, John Toadlena and Scott
Atole
October 16, 2012
Leukemia & Lymphoma Society (LLS) Information Resource Center (IRC)
“World’s largest voluntary health organization dedicated to funding blood cancer research, education, and patient services”
Private non-profit service organization Funded by individual and corporate contributions
68 chapters: U.S and CanadaMasters level Information specialists responding to 80,000 inquiries on leukemia, lymphoma, myeloma and other blood cancer patients, etc. Provide information, guidance & support (phone based)
Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation
Background on Service Program Case Study
Evaluation of Information Resource Center(IRC) Email – initial contact Program and academic interest & expertise match Successful response to RFP Initial discussions including questions identified in RFP response.
Additional details expected in the planning process RFP: Pilot a patient navigation model for IRC follow up services
Navigate health care systemsOvercome barriers
Evaluation of service enhancement-scheduled phone call follow up to select callers. Need identified by program staff
LLS-service oriented and committed to the evaluation processEvaluate new programs and ongoing evaluations Internal funding for this evaluation
Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation
Background on Service Program Case Study
Initial Planning: 3 months (email to launch) Report provided 6 months later LLS moved to fund a second/larger evaluation on Patient navigation
EPIC model (steps before the pilot study)
Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation
Background on Service Program Case Study
Understanding the Political & Organizational EnvironmentPre-project-consulted various sources
Internal/external people Internet/LLS website LLS Promotional items/collaterals RFP LLS Mission Statement LLS Organizational structure (background materials/site visit)
Commitment to organizational cause. Political environment associated with advocacy and
organizational commitmentFunding-individual/corporate LLS operations (private sector-accountability)How will the evaluation findings be used? Future funding
from LLS Board of trustees. Internally competitiveLocation of IRD within LLS structure. department
Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation
Assess the Context
Understanding the Political & Organizational EnvironmentLLS values program evaluation
Objective data to support the IRC program enhancementIRC – immediate program context
7 full time information specialists 80,000 phone calls annually Navigation model – innovative among other national programs Navigation to identified subgroup (2 specialists)
National organization 68 chapters – IRC link to local chapters Need for cancer information (consumer friendly) Potential for collaboration “community of cancer information service
professional”
Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation
Assess the Context
Define the Relationship Between Evaluator & Sponsor Internal/External Evaluator and Funding
LLS External Evaluator/Internal fundingFindings were reported to the LLS Board of TrusteesPrimary Point of Contact – LLS Vice President or Patient Services & Disease Programs (Includes IRC)
Advantages/Disadvantages-resistance to int/ext pressures, produce positive findings, objectivity, credibility, timelines(learning/building relationship), potential biases
Clarify roles and the relationship earlyEvaluator and Sponsor rolesFunding arrangements
LLS –Participatory Relationship in planning and implementationLLS – 2 ManagersIRC – 1 Supervisor (Ongoing Liaison) and 1 staff member
Ongoing Education – ResponsibilitiesLLS Staff – Provide feedbackEvaluator-Drafts, Technical assistance (research process), Assess stakeholder knowledge
Evaluator skillRelevant experience, experience and interestQualitative vs. Quantitative
Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation
Assess the Context
Determine the Level of the Evaluation Leukemia & Lymphoma Society (LLS) is local Information Resource Center (IRC) is national. Seek to create the connection with local chapters. LLS Target population
Newly DiagnosedThose seeking information on clinical trialsPatients residing in Wyoming
Reasoning for Target Population selection (Where will this enhancement be most helpful?):
Limited LLS resources to offer the program to all callersNavigation approach –reduction in health disparities to vulnerable populationsWyoming – no local chapters
Outcome vs. Process EvaluationPrimary focus on outcomesIRC program enhancement – include and document the process
Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation
Assess the context
Specify Evaluation UseLLS-findings will be used for program sustainability and continued and amount of future funding. Evaluations used to make decisions. Important to understand the implication of negative findingsWill this program be duplicated? Will the literature/results be made available nationally?
Validate PerspectivesCommunications with nearly every level of LLS/IRCWelcome evaluationSupportive of the projectImportant to understand that it is a program evaluation
Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation
Gather Reconnaissance
Potential stakeholders for a service program include: Two main reasons why the evaluator must engage stakeholders in
evaluation planning. Identify and Invite Stakeholders.
Establish rapport. Participants involved in the evaluation planning. “Sponsor” role.
Define Stakeholder Roles and Structure for Their Input. Importance of engaging the few stakeholder groups. Survey design.
John Toadlena UNM-Gallup PADM 522 Planning for a Service Program Evaluation
Engage Stakeholders
Establish Group Processes for Ongoing Stakeholder Involvement. The communication process.
Methods used.Evaluation plan timeline.Preliminary reports.
John Toadlena UNM-Gallup PADM 522 Planning for a Service Program Evaluation
Engage Stakeholders
Establishment of the stakeholder. Become immersed in the nature of the program to be evaluated. “Framework for Program Evaluation in Public Health”. Understanding the IRC. Conceptualize Program Theory or Rationale.
Usefulness of logic models and conceptual models. Evaluation Planning Matrix (EPM). (Holden and Zimmerman, pg.78-79)
Lists the evaluation questions considered. Short-term, Intermediate, or Long-term outcomes and identifies relevant data
sources corresponding to each question. Outcome surveys and questions.
John Toadlena UNM-Gallup PADM 522 Planning for a Service Program Evaluation
Describe the Program
Focusing the evaluation. The planning process. Study design team. Preliminary evaluation questions. Theoretical frameworks. Ensure Feasibility.
The focus on feasibility. Control or comparison group.
Assess Potential Data Collection Burden. Data collection burden an issue on two levels.
How much surveying could the participants tolerate. What kinds of data collection operation could IRC take on. Sample size.Draw conclusions from the results.
John Toadlena UNM-Gallup PADM 522 Planning for a Service Program Evaluation
Focus the Evaluation
Service Programs are unique Start with the Mission Statement
Size and scope of the activities
Evaluation Attitudes Stakeholders’ knowledge
Stakeholders Expertise & knowledge Minimizes resistance
Planning Process Dissemination Goals This case – positive and knowledgeable
Ervina Castillo-Newton UNM-WS PADM 522 Planning for a Service Program Evaluation
Lessons Learned
top related