101 Townsend Street, Suite 282

San Francisco, CA 94107

www.excelmeetings.com

Results Based Meetings Three Steps of Evaluation That Will Clearly Demonstrate the Value of

Your Conference, Sales Meeting or Annual Scientific Meeting

Presentation by Jon Wollenhaupt, VP, Excel Meetings & Events

www.excelmeetings.com

Processes outlined in this presentation are based on the ROI Methodology

developed by Jack Philips, PhD, Chairman, The ROI Institute

Definition of Results Based Meetings• Meetings are initiated, developed, and delivered

with the end in mind.

• A measurement and evaluation system is in place for each meeting.

• Perceived value, learning, application of learning evaluations are regularly developed.

• Participants understand their responsibility to obtain results with meetings.

• Support groups (management, sponsors, colleagues, etc.) help to achieve results from meeting.

Paradigm Shift in Meetings and Events

Activity Based Results Based

No clearly defined business need for the meeting

Program linked to specific business needs

No assessment of performance issues

Assessment of performance effectiveness

No specific measurable objectives

Specific objectives for behavior and business impact

No effort to prepare meeting participants to achieve results

Results and expectations communicated to participants

Paradigm Shift in Meetings and Events

Activity Based Results Based

No effort to prepare the work environment to support meeting

Environment prepared to support meeting

No efforts to build partnerships with key managers

Partnerships established with key managers and clients

No measurement of results or benefit-cost analysis

Measurement of results and benefit-cost analysis (ROI)

Reporting on meetings is input focused

Reporting on meetings is output focused

Evaluation Levels Level Measurement Focus

1. Reaction & Measures participant reaction to thePerceived meeting and captures perceived value.Value

2. Learning Measures changes in knowledge,

information, skills, and attitudes gained in the meeting.

3. Application & Measures changes in on-the-jobImplementation behavior and progress with planned

actions.

Chain of Impact From Meetings & EventsParticipant has a positive reaction to the meeting

Participant acquires knowledge, develops skills or changes attitudes/perceptions

Participant uses new knowledge, skills and attitudes on the job

Use of new knowledge, skills and attitudes drives business performance

Monetary value of business performance exceeds cost of meeting

Levels of Assessment

Level 1 Reaction and Perceived Value• Relevance

• Importance

• Usefulness

• New Information

• Motivational

•Appropriate

• Rewarding

• Challenging

Level 2 Learning

• Information

• Knowledge

• Understanding

• Skills

• Contacts

• Capability

•Expertise

• Capacity

Levels of Assessment

Level 3 Application and Implementation

• Use of Skills

• Use of Knowledge

• Completion of Actions

• Success with Application

• Enablers

• Tasks Completed

• Steps

• Procedures Followed

• Barriers

Levels of Assessment

The ROI The ROI MethodologyMethodology

Develop EvaluationPlans and

Baseline Data

Develop EvaluationPlans and

Baseline Data

EvaluationPlanning

Data Collection Via Questionnaire

Collect DataDuring Meeting

Collect DataDuring Meeting

Level 1: REACTION AND

PERCIEVED VALUE

LEVEL 2:LEARNING

Collect DataAfter MeetingCollect Data

After Meeting

LEVEL 3: APPLICATION ANDIMPLEMENTATION

OPTIONAL LEVEL 4:

IMPACT AND CONSEQUENCES

Develop Objectives

ofMeeting

Develop Objectives

ofMeeting

Plan Meetings with Evaluation in Mind

Developing Clearly Defined and Measurable Reaction Objectives for the Meeting After attending the Executive Strategy meeting, participants will perceive the meetings to be:

• Conducted by relevant and effective speakers

• Valuable in terms of inspiration, and in enable participants to be more effective leaders

• Full of new information

• Valuable in terms of helping attendees attain short and long term goals

• An excellent use of their time

Plan Meetings with Evaluation in Mind

Developing Clearly Defined And Measurable Learning Objectives For The Meeting Measuring Changes in Information, Knowledge, Skills, Perceptions, and Contacts

At the end of the meeting, participants will be able to:

• Identify three new e-commerce strategies relevant to

their department

• Identify new ways to foster ideas sharing and collaboration with other departments

• Explain two new ways to motivate creative thinking within their department

• Explain 5 steps to innovation that were presented during the meeting

Project Example:

Development of Measurable

Learning Objectives and

Post Event Questionnaire for

an Annual Scientific Meeting

Overview:

Excel Meetings & Events was under contract with a San Francisco Bay Area Pharmaceutical Company to assist in the development of measurable learning objectives for an annual scientific meeting and in the design of a post meeting questionnaire. This meeting brought together research scientist from three separate company facilities located in Silicon Valley, Alabama and India.

Overview:

General desired outcomes for the meeting were stated as follows:

Energized participants - Key scientists and engineers from all research sites will have an opportunity to interact, discuss science, think about the future, get to know each other and establish alliances that will enable success.

Stated Measurable Learning Objectives

After the annual scientific meeting, research scientist will be able to: • Explain the process for implementing new research• Name three new colleagues you met with whom you can collaborate on your research• List three reasons you agree with the research

funding for 2008• List the three most significant things you learned in the poster session

• Identify at least one science or technology the company should acquire or develop

Stated Measurable Learning Objectives

After the annual scientific meeting, research scientist will be able to:

• Explain the process for implementing new research• Name three new colleagues you met with whom you can collaborate on your research• List three reasons you agree with the research funding for 2008• List the three most significant things you learned in the poster session

• Identify at least one science or technology the company should acquire or develop

Questionnaire Development Based on the stated learning objectives, Excel Meetings worked with client staff to develop a post meeting questionnaire that assessed participants learning and intentions to apply that learning upon return to work.

Sample questions included:

1) I feel confident that I can influence research within my role in the R&D organization. Please circle a number that best reflects your response.

5 = Strongly Agree, 4 = Agree, 3 = Neutral, 2 = Disagree, 1 = Strongly Disagree

Questionnaire Development Sample questions included:

2) Identify two or three things you learned about future research ideas in the poster session. (open-ended)

3) Identify three major accomplishments attained in 2008that you learned about in the poster session. (open-ended)

4) List the three reasons why you agree with the research funding for this year as discussed. (open-ended)

Questionnaire Development Sample questions included:

5) Name three new colleagues you met at this year’s Research Summit , and list at least one way you will collaborate with each on your research in the coming year. (open-ended)

6) What I learned from the presentations I will be able to use in my work and with all colleagues when I return to work. Please circle a number that best reflects your response.

5 = Strongly Agree, 4 = Agree, 3 = Neutral, 2 = Disagree, 1 = Strongly Disagree

“Excel Meetings & Events worked with

our internal meeting planning resources

and provided guidance, tools, and ideas.

Their contributions enhanced the quality

of our annual scientific meeting and set

up the foundation for continuous

improvement.”

– CARLOS SCHULERSENIOR DIRECTOR OF SCIENCE & TECHNOLOGY

Alliance Partner Profile

The ROI Institute, Inc. is the leading resource on research, training, and networking for practitioners of the Phillips ROI Methodology™. Founded in 2003, the ROI Institute represents over 25 years of experience in measurement and evaluation of training, human resources, technology and quality programs and initiatives.

“The (Phillips) ROI process has become the cornerstone of organizational assessment, measurement, and evaluation strategies.”

- Debi Wallace,Vice President, Advanced Analytics Wachovia Corporation

Alliance Partner Profile