mqa-kqa team leader handbook - midwestexcellence.org · the mqa-kqa award office has three...
TRANSCRIPT
MQA-KQA Team Leader Handbook
Table of Contents
The MQA-KQA Team Leader............................................... 5
Your Voice .................................................................................. 5
ACTIONS ..................................................................................... 5
First Steps ............................................................................ 7
1 ................................................................................................... 7
Contact team members & request info .............................................................................................7
Communicating with New Examiners ...............................................................................................8
2 ................................................................................................... 9
Set the Team Calendar ....................................................................................................................9
3 ................................................................................................. 10
Send calendar to the team and the MQA-KQA Office ................................................................... 10
4 ................................................................................................. 10
Prepare Team Assignments .......................................................................................................... 10
5 ................................................................................................. 11
Assign Category/Item Leads ......................................................................................................... 11
Independent Review .......................................................... 12
Key Factors .............................................................................. 12
Consensus ......................................................................... 14
Site Visit ............................................................................. 18
Pre-Site Visit Checklist ............................................................ 18
Site Visit: Best Practice Suggestions .................................... 19
Ending the Site Visit ................................................................ 21
Final Wrap-up Meetings (at hotel after Site Visit) .................. 21
Putting the “L” in A-D-L-I ........................................................ 22
Keeping the Team Focused & Productive ....................... 23
Appendix ............................................................................ 25
Communication Templates ..................................................... 25
MQA-KQA Team Leader Communication Overview ..................................................................... 26
Welcome ........................................................................................................................................ 28
Key Factors & Independent Review .............................................................................................. 30
Getting Ready for Consensus ....................................................................................................... 32
Focusing on Consensus ................................................................................................................ 34
‘See You’ at the Consensus Meeting ............................................................................................ 36
Final Preparations for Site Visit ..................................................................................................... 38
We’re DONE! ................................................................................................................................. 39 Sample Consensus Schedule ................................................. 39
Site Visit Issue Tracking ................................................................................................................ 40
The MQA-KQA Team Leader............................................... 5
Your Voice .................................................................................. 5
Team Leader Checklist ............................................................. 5
ACTIONS ..................................................................................... 5
First Steps ............................................................................ 7
1 ................................................................................................... 7
Contact team members & request info .............................................................................................7
Communicating with New Examiners ...............................................................................................8
2 ................................................................................................... 9
Set the Team Calendar ....................................................................................................................9
3 ................................................................................................. 10
Send calendar to the team and the MQA-KQA Office ................................................................... 10
4 ................................................................................................. 10
Prepare Team Assignments .......................................................................................................... 10
5 ................................................................................................. 11
Assign Category/Item Leads ......................................................................................................... 11
Independent Review .......................................................... 12
Key Factors .............................................................................. 12
Consensus ......................................................................... 14
Site Visit ............................................................................. 18
Pre-Site Visit Checklist ............................................................ 18
Site Visit: Best Practices ......................................................... 19
Ending the Site Visit ................................................................ 21
Final Wrap-up Meetings (at hotel after Site Visit) .................. 21
Putting the “L” in A-D-L-I ........................................................ 22
Keeping the Team Focused & Productive ....................... 23
Appendix ............................................................................ 25
Communication Templates ..................................................... 25
MQA-KQA Team Leader Communication Overview ..................................................................... 26
Welcome ........................................................................................................................................ 28
Key Factors & Independent Review .............................................................................................. 30
Getting Ready for Consensus ....................................................................................................... 32
Focusing on Consensus ................................................................................................................ 34
‘See You’ at the Consensus Meeting ............................................................................................ 36
Final Preparations for Site Visit ..................................................................................................... 38
We’re DONE! ................................................................................................................................. 39 Sample Consensus Schedule ................................................. 39
Site Visit Issue Tracking ................................................................................................................ 40
5
The MQA-KQA Team Leader
Thank you for accepting your MQA-KQA assignment! As a Team Leader, you play a critical role in meeting the expectations of our customers (the applicants) and our workforce (your team members).
f you have been a Team Leader in the past, you know what a demanding and fulfilling job this can be. If this is your first experience as a Team Leader, you are in for an exceptional opportunity to improve your leadership skills and deepen your understanding of the Baldrige Excellence FrameworkTM.
This guide can help you lead efficiently and effectively. In addition, we hope it will help you manage your anxiety and stress. Based on a collection of lessons learned and best practices that have been gathered from some of our most effective Team Leaders, it is designed to give you guidance along the way, using the insight of your peers.
The MQA-KQA award office has three conference lines available for use by Examiner teams (if needed). Please contact the office to request a line once your team schedule is set.
If you encounter a situation that you’re not sure how to handle, don’t hesitate to call the MQA-KQA office (573.817.8310). We will do whatever we can to help you be successful.
Your Voice
As you complete this year’s assignment, please take note of experiences that can help future Team Leaders, and send them to MQA-KQA to help build our organizational knowledge.
Team Leader Checklist
STEPS ACTIONS
Getting Started Send team welcome email and request Biography Forms and Calendar Forms from
team members; ensure no conflicts of interest with assigned application
Set and distribute team calendar
The MQA-KQA office will set up a scorebook for everyone in Navigator, but they
will not be able to begin recording comments until the common list of key factors
is entered into your (Team Leader’s) scorebook
Prepare key factors scorebook for your team in Navigator. Then, contact your
team members and instruct them to log into Navigator and check their scorebook
I
“Done”. As Team Leader, your list of key factors will be used for the team
scorebook moving forward
Once completed; share your scorebook with the rest of the team so they can begin
independent review. All scorebooks will contain the key factors from your
scorebook. (Team Leader’s Navigator Guide, Steps 2 and 3—be sure Team Leader’s name
is in drop-down box.)
Stage 1 – Managing Independent Review
Consider reviewing item 1.1 for each team member during the first week to make
sure everyone is on track
Check in with new Examiners and those who may be struggling throughout
independent review; remind everyone to check “done” when IR is complete
Monitor work in Scorebook Navigator and follow up with any stragglers
Stage 2 – Consensus Review
Move team to consensus review in Navigator; assign item leads and back-ups
Host a team call to review instructions and expectations for consensus
Monitor work in Scorebook Navigator and follow up with any stragglers
Distribute consensus meeting agenda and consensus script
Manage consensus meeting, including identification of key themes and site visit
issues
Notify MQA-KQA when consensus scorebook is final
Stage 3 – Site Visit (Preparation)
Attend site visit training
Collect site visit questions and document requests from team members’ SVI forms
Provide applicant site visit agenda, questions, and document requests 1-2 weeks
prior to site visit
Collect and review completed site visit issues from members
(On Site and at hotel)
Conduct opening meeting, daily team meeting(s), and closing meeting
Manage site visit, including checking in with applicant daily and monitoring team
member progress (make sure there are solid strengths and OFIs for each item)
Maintain interview and document logs
Stage 4 – Finalize Feedback Report
Manage team wrap up meeting, including finalizing key themes, item comments,
and item scoring ranges in Navigator
Complete Summary of Sites Visited Form with team
Read and edit the final scorebook before turning in to MQA-KQA
Forward all final documents to MQA-KQA
7
First Steps
Please follow these guidelines to get your team started on the assessment process when you receive your application. While numbered, all these steps should be done right away, and some overlap – so read through them all and start planning now (you’ll be glad you did). As you follow the steps, employ the “Tips on Keeping Your Team Focused and Productive,” outlined in the last chapter.
Contact team members & request info
As soon as your team receives its assignment, send an enthusiastic email to welcome members and remind them to read the application from start to finish and begin identifying key factors. This is your opportunity to introduce yourself, review the assessment process, set clear expectations, and
role model the attitude you want to see from the team. It is also appropriate in this email to establish an email protocol. For example, when determining site visit and consensus meeting dates, encourage team members to “reply to all” to expedite the process. A sample Welcome Email is available in the Team Leader Communication Templates in this manual.
Some Team Leaders have found it helpful to draft this introductory email before receiving their team’s assignment, so they are ready to contact the team as soon as the assignment is received. This practice can make setting the team’s calendar more efficient. Also, ensure at this time that team members are checking for personal conflicts of interest.
Include the Biography Request and Calendar Request forms with email, and state the deadline when you want them completed – no more than a few days. You will need the information from these forms to determine a team calendar.
Chapter
1
1
Find resources on
the Examiner
Resources page
of the MQA-KQA
website:
www.MQA-KQA.org/Examiners/resources.php
8
Tip
Ask team members to include why they are unavailable on their Calendar Request form. This encourages team members to consider rescheduling or missing an already planned event.
It doesn’t hurt to mention in this email the site visit date your applicant has chosen (if applicable); just to remind/confirm it for Examiners:
Site Visit Week #1 – September 8 – 13, 2019 Site Visit Week #2 – September 15 – 20, 2019
Finally, point out that it is critical to meet all team deadlines. If something catastrophic happens in a team member’s life and he or she can’t meet a deadline, impress upon the team to notify you as soon as possible.
Follow-up call
After an initial email, we recommend you call each Examiner individually. A phone call is more personal and usually provides better two-way communication. It’s also particularly reassuring to new Examiners. This call will give you an opportunity to get to know the individuals on your team, allowing you to determine their experience and special areas of interest. This will prove helpful when you assign category/item leads and other team assignments.
During this conversation, confirm contact information to ensure that nothing has changed (email, telephone). Encourage team members to respond to your communications quickly throughout the process.
Communicating with New Examiners
We suggest you communicate more frequently with new Examiners to check in and offer encouragement. One Team Leader suggests that either the Team Leader or Overseer review and provide feedback on item 1.1 of each new Examiner’s independent review. If an Examiner’s work is off-track, an early correction will prove helpful for the team in the long run.
C HECK L IS T
During your initial contacts with Examiners:
Set a positive tone for the team. Role model the behavior you want to see from your team members.
Confirm contact information and determine the best method to contact individual Examiners.
Ensure that team members are reading the application documents and Organizational Profile to determine if there are personal conflicts of interest with the applicant.
Tell Examiners to start independent review right away by first reading the application from cover to cover and then identifying key factors and inputting them in Scorebook Navigator.
9
Set expectations: give the team a due date for completing key factors and one item for your review.
Give the team a deadline for filling out the Biography Request and Calendar Request forms and returning them to you.
Remind Examiners to respond as quickly as possible to all team communications.
Ask team members to prioritize the process categories they would like to lead in consensus and site visit (they can number their choices 1 through 6, with 1 being their first choice).
Set the Team Calendar
As soon as you receive the Calendar Request forms from all team members, grab a calendar and set up an assessment schedule for your team.
First, check the site visit date and work backwards from the site visit to establish other deadlines. Some tips:
Allow time between the independent review deadline and the consensus meeting for team members to synthesize their assigned items for consensus.
Allow time between your team’s consensus meeting and the site visit to complete the Consensus Scorebook and send issues to the applicant. Remember: applicants must receive site visit issues at least one week before the site visit, but two weeks is preferable.
The final wrap-up meeting should be held the day after the site visit while the team is still together.
Some Team Leaders recommend using a shared calendar in Google Docs to ensure consistency
within the team and reduce misunderstandings about dates and deadlines.
C HECK L IS T
Your team calendar should include the following deadlines and events:
Complete one assigned item (one to two weeks after application received) – applies to team.
Complete independent review and check “done” – applies to team.
Move team to consensus and complete category/item lead assignments – applies to Team Leader.
Complete synthesis for all assigned categories (3-4 days before consensus meeting) – applies to team. This includes:
Synthesize team members’ independent reviews for your assigned items.
Draft comments for each assigned item.
Work with item backup to receive and provide feedback.
Rewrite comments, incorporating feedback from your backup.
2
10
Prepare to present assigned items at team’s consensus meeting.
Review comments of all team members.
Consensus meeting – applies to team.
Site visit – applies to team.
Site visit wrap-up meeting, (remainder of Site Visit week following visit to applicant’s site) – applies to team.
Best Practice
As the assessment progresses, it’s a good idea to send team members a weekly reminder email on Thursday (allowing time to work on assignments over the weekend), especially during weeks when there is no other correspondence among team members. Keep it short and sweet: “Dear [Examiner] Hope all is well. Your next deliverable is ______ and it is due on _______.” Please let me know if you have any questions, I’m here to help.” Don’t over-manage – but do stay in touch. Regular contact and follow-up are required if everyone is to stay on schedule. If deadlines are missed, make compensating adjustments, but once you’re in the throes of the assessment, it is nearly impossible to change the consensus meeting date.
Send calendar to the team and the MQA-KQA Office
When your calendar is set, email it, including a list of all deliverables and due dates (use the Examiner Deliverables Checklist as a guide), to team members for review. Remind Examiners to
respond quickly to any correspondence from the team.
As soon as your team calendar is set, send a copy to the MQA-KQA office as well. Email it to [email protected]. Make sure this calendar includes your Consensus meeting date. Upon receipt, the award office will provide you with a conference line, if needed.
Best Practice
Add [email protected] to the email distribution list you create for your team. This way, MQA-KQA receives all group correspondence, which will include the calendar, materials and other updates.
Prepare Team Assignments
When you have received team members’ completed Biography Request forms and have had a chance to get to know everyone, you can assign team roles:
3
4
11
Back-up Team Leader. Fills in if a Team Leader cannot fulfill responsibilities. Keeps close tabs on the assessment process in case he/she needs to step in. Helps Team Leader:
o communicate with team members throughout the process,
o review team members’ comments and notify Team Leader of anyone not making expected progress,
o draft key themes, and
o edit the final scorebook.
Results Guru. A team member who easily makes sense of category 7 and can help the team evaluate results during the site visit – both from a category perspective and from a holistic viewpoint. This is typically someone who likes numbers, graphs and auditing tasks.
Time Keeper. During team meetings and site visit, keeps the team on track to complete the task within the allotted schedule. (Overseer can serve this role, if needed.)
Assign Category/Item Leads
You will assign category leads and backups after your team completes its independent review. You have access to resources that will help you make these assignments: Examiners’ category preferences (captured in biography form) and their independent review work. When assigning
items, try to give new Examiners their category preferences because these are the categories they feel most comfortable and confident working with. Have them serve as backups on their weakest categories, so they can learn the criteria.
Once you determine process category leads, assign corresponding results items as well (note that not every results item corresponds directly/completely with a process category).
Results Items that correspond with Process Categories
Process Category Corresponding Results Item (Category 7)
Category 1 Leadership Item 7.4 Category 2 Strategy Item 7.5 Category 3 Customers Item 7.2 Category 4 Measurement, Analysis and Knowledge Management Item 7.1—7.5 Performance Results Category 5 Workforce Item 7.3 Category 6 Operations Item 7.1
5
12
Independent Review
You and your team should start the first steps of independent review (reading the full application and identifying key factors) as soon as the MQA-KQA application packet arrives. Much of your independent review work will overlap with the scheduling and Team Leader tasks outlined in the previous chapter. It is key that you set a team schedule in the earliest stages of the assessment process.
Scorebook Navigator
As soon as you’ve been assigned a team, please familiarize yourself with the Team Leader tools in Scorebook Navigator. Scorebook Navigator Help
If you or a team member runs into trouble using Scorebook Navigator, help is available via phone or email.
MQA-KQA Office
Monday—Friday, 8-5 [email protected]; 573.817.8310
After Hours/Weekends Christi’s Personal cell: 573.291-1949 (Please allow extra response time after hours/on weekends; I may not be immediately available, but will respond as soon as possible after receiving your voicemail.)
Key Factors
The first steps of independent review are to read the full application and then identify a master list of key factors. As Team Leader, your list of key factors will be used for the entire team. Please enter the key factors as soon as possible, as other team members cannot begin entering comments until your list of key factors has been shared.
Item Evaluations
Use the “Team Steps” from the “Team” screen in Scorebook Navigator to assign a due date for Independent Review. Encourage team members to complete their work on time, as moving to the consensus stage in Scorebook Navigator requires that all team members have finished their independent
Chapter
2
Scorebook
Navigator
Resources:
Scorebook
Navigator User
Manual
13
reviews and checked “done.” Remind Examiners to use you or the team Overseer as a resource if they get stuck.
The Team Leader tools in Scorebook Navigator allow you to monitor your team’s work. However, you will not be able to see each team member’s scorebook entries. If team members are slow getting started, ask how you can help. Be sure to communicate with each team member in the method preferred by that member. Past Team Leaders suggest having team members complete item 1.1, and then download and email the item worksheet for you or the scorebook editor to review and provide feedback. This practice ensures Examiners are getting started on the work, and it allows you to identify early in the process Examiners who may be struggling so you can offer assistance.
Make sure Examiners know to click “done” from the Scorebook Navigator “Scorebooks” screen when their independent review work is complete.
14
Consensus
The Consensus meeting helps build the team. Examiners get to know each other better, and it creates a sense of peer pressure, so assignments are done on time.
Consensus by Phone
A successful telephone Consensus meeting must have structure. Provide team members a format or Consensus Script to use for their reports and use a round robin approach to ensure consensus after each discussion point. You may also consider hosting two separate four-hour calls rather than a single eight-hour day. Of course, should the team choose, the Consensus meeting can also be held in-person. .
Some Team Leaders use Google Docs during consensus calls to keep everyone on track. If you do this, be sure you are familiar with Google Docs before the call.
Before the Consensus Meeting
Starting Consensus in Scorebook Navigator
In order to begin the consensus process, you will need to move the team to consensus in Scorebook Navigator and make item assignments. From the “Team” screen, select “Team Steps” to begin the consensus process. Select your name from the drop down to indicate your (the Team Leader) scorebook will be used as the basis for the consensus scorebook. Make consensus assignments using the “item assignments” function. You can change item and back-up assignments during the consensus process.
Completing Assigned Items
Each Examiner is responsible for synthesizing and presenting findings from his or her assigned items. Ask team members to develop a short consensus report for their items that includes:
A short description of the proposed strengths and OFIs (noting which Examiners made similar comments),
Rationale for proposing each strength and OFI comment,
A summary of comments not used and why (again, noting the team members who support the comments),
Chapter
3
15
The score and the rationale behind the score.
These reports will be distributed for review among team members before the consensus meeting. MQA-KQA has developed a Consensus Script template that incorporates all of these points. As team members begin to synthesize items for consensus, we suggest you send your team this template – including a sample you’ve completed – so everyone uses the same format. A unified approach will facilitate and speed up the meeting.
Additional Pre-Meeting Tasks
Before meeting day, remind Examiners of their roles: timekeeper, results guru, etc. Emphasize using the consensus script, and advise team members to focus on the content of the comment rather than reading each comment. If possible, send out an agenda and ground rules ahead of time (through email, Google Docs, etc.).
Best Practice
If a Team Leader or Backup Team Leader doesn’t have assigned category/items, consider capturing feedback in real time in Scorebook Navigator during the consensus meeting.
Meeting Day
Although team members will be leading different portions of consensus, you are responsible for the overall meeting. Start the meeting by defining/reviewing objectives, agenda, schedule, and ground rules. See the Sample Consensus Schedule for ideas. We recommend providing the agenda and ground rules to your team ahead of time.
It helps to build camaraderie when you begin with a simple ice breaker. An easy one is to ask Examiners to introduce themselves and offer what they think will help the group stay on task during the meeting.
At some point during the day, schedule 45 minutes to discuss the preparation of Site Visit Issues in Scorebook Navigator. During this time, include a review of what to expect at site visit and the things that Examiners should NOT say and do. You can review the MQA-KQA Rules of Conduct and Code of Ethical Standards in the Examiner Handbook as a refresher.
Consensus Reports
Once the pleasantries are over, it’s time to get down to business. Using their pre-written reports, each Examiner will lead the discussion of his or her assigned categories/items.
The Team Leader or an experienced Examiner should handle the first report-out to demonstrate the process.
During these reports, the Team Leader and Overseer should capture ideas for key themes.
16
Keep moving
Consensus is not the time for wordsmithing. In fact, make this a ground rule. The consensus meeting should only address comments that are not agreed upon or not used if the team member who made the comment wants to discuss it. Agree on the “meat” of what should be included; the category leader can finalize the comment after the meeting. That said, be sure to establish a deadline for finalizing comments.
Timekeeping
As you develop a consensus meeting agenda, we suggest you schedule 15-20 minutes per item to discuss site visit issue strategies and questions.
Site Visit Issues (SVIs)
During the consensus meeting your team will agree on site visit issues. Category leaders should be prepared to recommend SVIs for their items.
When SVIs are identified, discuss site visit issue strategies and develop a list of questions to present to the applicant. You will use Scorebook Navigator to document your SVI statement (verify/clarify), the documents you are requesting, and the people you would like to interview. Determine what material should be reviewed and, if applicable, decide what on-site meetings individual team members will attend.
Remember, site visit issues should be relevant to the applicant. They are important points that need to be verified or clarified on site, they are related to key
factors, and they must be linked to comments.
You may consider using a list of interview questions that all team members ask during walk-around interviews (a sample set of Walk-Around Questions are available on MQA-KQA Resource website). Select pre-set questions based on site visit issues that cut across categories.
Prepare Examiners for Site Visit
An important aspect of the consensus meeting is preparing team members – especially new Examiners – for the upcoming site visit. Train team members to treat the site visit like a sales call – NOT like a police interrogation. Share some tips and best practices. Tell Examiners to practice active listening and let the applicant tell its story. Use the site visit to clarify, verify, and learn all you can about the applicant.
Coordinate
Decide when the team will meet for a final coordination meeting before going on site. Teams usually meet at the hotel on Sunday afternoon/evening before the visit begins on Monday.
Arrive together for the site visit, and try to fit the group into as few cars as possible. This cuts down on last-minute logistical issues and helps if there are limitations on parking. Be sure to discuss parking with the applicant as you correspond with them prior to the site visit.
17
Wrapping up Consensus
After the consensus meeting, category leaders will finalize their Site Visit Issues (SVIs) in Scorebook Navigator. As the Team Leader, you (or the Back-up Team Leader) prepare a list of SVIs and requested documents for the applicant using the information in Scorebook Navigator. Be sure to identify a deadline for completing SVIs in Scorebook Navigator before the consensus meeting wraps up.
Keep in mind, you may send site visit issue topics to the applicant before SVIs are finalized. The Site Visit Issues will help your team conduct interviews and take notes on site. Past Team Leaders have found the following to be an efficient sequence for consensus wrap-up:
1. Within 72 hours of the consensus meeting, team members complete Site Visit Issues and document/interview requests in Scorebook Navigator. Team members complete any edits to comments identified during the consensus meeting.
2. Team leader contacts MQA-KQA to finalize the consensus scorebook.
3. Two weeks before site visit, the Team Leader sends site visit questions and document request to the applicant.
4. Team members finalize SVIs at least three days before site visit. Most importantly, get moving as soon as the consensus meeting is over, and make sure your team members know the deadlines they are expected to meet.
18
Site Visit
Site Visit is your team’s opportunity to see the applicant at work. It provides you an opportunity to clarify and verify, and an opportunity for the applicant to tell its story. Your team is there to learn and help the applicant improve – not put it through the wringer.
Pre-Site Visit Checklist
The applicant schedules the hotel meeting and sleeping rooms. The MQA-KQA award office will be involved in the coordination of this, and will make sure the team know where they are staying.
Develop a tentative site visit schedule, incorporating time for follow-ups and unplanned needs.
Send SVIs and tentative schedule to applicant. At least one week before your site visit, send the applicant your tentative site visit schedule, along with your team’s site visit issues. Offer to talk with the applicant by phone to clarify any questions that may arise from these materials.
Sending the site visit issue topics at least two weeks before site visit is a firm commitment and every effort should be made to meet it. The applicant plans pre-site visit activities around this material. A back-up plan (or two) is needed in case information from team members is late.
Best Practice
Send your applicant site visit issue topics two weeks prior to site visit. The earlier you prepare the applicant, the more productive your site visit will be. It is important to have regular contact with the applicant, so they do not become overwhelmed when they receive their site visit questions.
Send a copy of SVIs and tentative site visit schedule to MQA-KQA. When you send the applicant your team’s list of site visit issues and the tentative schedule, also send a copy to MQA-KQA. You can email these materials to [email protected].
Chapter
4
19
Finalize Consensus Scorebook and alert MQA-KQA. When the Consensus Scorebook is complete, alert MQA-KQA via email: [email protected]. MQA-KQA will download a copy of the team’s consensus scorebook.
Contact the applicant. The week before your team’s site visit, finalize the site visit schedule and other site visit details with the applicant. Keep your team informed of any changes.
Pre-Site Visit Coordination Meeting
The day before going on site (Sunday) schedule some time to review final details with the team. The meeting gives you an opportunity to convene as a group away from the site, and review the site visit plan and team objectives. Check in, making sure everyone has what they need.
Best Practice
Encourage team members to review the applicant’s Organizational Profile before going on site to refresh their memories about the organization.
Site Visit Introductory Meeting
As Team Leader, you will lead the introductory meeting with the applicant on the first day of your site visit. You can find presentation material on the MQA-KQA Resources page (https://midwestexcellence.org/volunteer-resource-center/). During the meeting, review the purpose of your visit, your team’s schedule, ground rules and expectations.
Answer questions regarding MQA-KQA and the assessment process. Provide time for an opening presentation from the applicant and a very short tour of the site. Prior to the site visit, reach agreement with the applicant on the length of this presentation and tour (typically around one hour for a three-day site visit). Walking around the site allows the applicant to become more comfortable with the team and the team to become acquainted with the facility, the people, the company culture, and lingo.
Site Visit: Best Practice Suggestions
As you launch into the nuts and bolts of your site visit, consider implementing these best practices collected from MQA-KQA’s Team Leaders over the years:
Set the tone. In the pre-site visit meeting, ask the team “When it’s all over, what do we want them to think of us?” Each night revisit this question and ask “where did we fall short today?” This will help keep the team focused on helping identify improvement opportunities for the applicant vs. conducting an audit.
Have all team members sit in/take notes for category 1. This gets all the team members on the same page. Plus, some of your SVIs for later categories may be answered, which will save time in the long run. One Team Leader likes to include all team members in this first meeting by assigning each team member to ask one of the pre-determined category 1 questions.
Monitor the category sessions. During the site visit, category leaders should lead the meetings that deal with the evaluation of their categories. As Team Leader, you should keep abreast of how these sessions are
20
playing out. Emphasize the importance of team members starting meetings with applicant personnel when they said they would – it shows respect for our customer.
Schedule and lead team caucus sessions. Throughout the site visit, your team will need to schedule time to compare notes and discuss findings. These meetings should be incorporated into your site visit agenda. Guide these sessions so they are as efficient as possible, and make adjustments to the schedule and individual assignments based on what has been learned. Are team members clearing OFIs? If so, have “new” OFIs been identified? Remember that the applicant expects the Examiner team to help identify its gaps.
Make use of your team’s results/category 7 guru (if you have one).
Ask this team member to help review updated data and information provided by the applicant at site visit.
Assign a team member to “follow” the key themes. The purpose of this is to have someone (often the back-up team leader) focused on validating, adding to, and revising the key themes using the most current information.
Monitor conduct and ethical issues. Make sure the Rules of Conduct and Code of Ethical Standards are being followed.
Lead evening review sessions at the hotel. Make necessary adjustments to the schedule and individual assignments based on what has been learned. During this meeting, have each category lead take everyone’s notes and quickly compile the SVI findings. (They can be finalized later, as the site visit progresses.) Also, take this time to develop follow-up questions for the next day. It doesn’t hurt to begin work on the Final Scorebook during these sessions.
Be aware of the team’s biorhythms. One experienced Team Leader has noticed that on site visit, the team’s energy takes a dip the evening of the second day (Tuesday). So he holds a brief (30 minute) team meeting the evening of day 2 and asks just one question: “What else do you need to know?” Then he dismisses the team to relax and rest up for the third and final day on site with the applicant.
Meet daily with the applicant. After the first day of your site visit, arrange to meet briefly one-on-one with the applicant’s designated contact person. Use this time to touch base, review open issues, clear up any misunderstandings, and ask “Have you had the opportunity to tell your story?” Based on feedback from the applicant, make adjustments to the next day’s schedule to ensure that the team’s work adds value and meets the applicant’s expectations. Sherry Marshall, CEO of the Midwest Excellence Institute will contact the senior leader of the organization daily to ensure their satisfaction with the progression of the site visit.
Use the Site Visit Interview Summary. Keeping up with who is interviewed can help you determine areas that have not been addressed.
Return Documentation. It is imperative that documents reviewed by the team are tracked and returned to the applicant. If a document is requested, you must make sure that it is reviewed and returned. Every time you receive a document, attach the business card of the individual who gave you the document to ensure it is returned to the right person.
21
Ending the Site Visit
Schedule a quick caucus with your team before you leave the applicant site. Has all documentation been returned? Did you get all your questions answered? If OFIs have been cleared, have new ones been identified?
After this caucus, gather the team and the applicant and hold the closing meeting. Keep it short and sweet. Quickly review confidentiality rules: If a team member and someone from the applicant organization happen to run into one another (at the MQA-KQA Awards Banquet, for example), the Examiner will not acknowledge the applicant unless the applicant approaches first. Examiners aren’t being rude; they’re just following protocol to protect confidentiality!
Once confidentiality is covered, describe what the applicant can expect next (feedback report in approximately 60 days), answer final questions and say good-bye.
Final Wrap-up Meetings (at hotel after Site Visit)
As you begin these wrap-up meetings, define objectives, agenda, and ground rules. You can use the same report format and ground rules as you did in the consensus meeting, if you choose.
Although team members will be leading different parts, you are responsible for the overall meeting. This is an important time when your team will finalize its Scorebook. The following should take place at these wrap-up meetings:
Item leaders guide discussion of their items, including changes to comments and recommendations to change the scoring range.
Item leaders document site visit findings in Scorebook Navigator.
Item leaders update comments in Scorebook Navigator.
Agree on key themes.
Complete Summary of Sites Visited, if your applicant has more than one site.
In your final scorebook, make sure the comment wording is factual and accurate. Agree on what should be included. The category leader can refine as he or she prepares the final edits after the meeting.
Remember that all team members are released at the same time; and only after all tasks are complete and the MQA-KQA office has been notified that your site visit
is complete. About an hour before the team is finished, notify the applicant to pick up any equipment or materials from the hotel meeting room. The Team
Leader and Overseer must stay until this is taken care of.
22
Best Practice
Don’t allow more than a few days for your team to complete work after site visit. Memory fades quickly, and margin notes lose their meaning. Get buy-in on a deadline before you leave the wrap-up meeting.
Putting the “L” in A-D-L-I
It is helpful to MQA-KQA and to you as you grow in your role to take time during the wrap-up meeting to discuss the process.
What went well and what did not go so well?
What lessons have been learned? Capture these thoughts in writing and submit them to MQA-KQA—it doesn’t have to be formal.
Producing the Final Scorebook
Review the Final Scorebook as soon as you receive the final category reports from all your team members, and alert the Overseer so he or she can also review the Final Scorebook.
After compilation, the Final Scorebook needs to read like ONE person wrote it (instead of six different category leads). This is where the Overseer comes in handy, particularly if he or she has strong writing skills. With an editor’s polish, the report will appear more professional and add value to the applicant and credibility to our organization.
Notify the MQA-KQA office that the Final Scorebook is complete and send MQA-KQA your other site visit documentation. This should occur within ten days of your site visit.
Use the Final Scorebook Checklist to be sure you are sending MQA-KQA all the required documents. It is available on the Examiner Resources website.
Final Steps
Celebrate! After everything has been turned in to MQA-KQA, provide your team closure by thanking members for a job well done. This announcement should be celebratory in tone. You worked hard and have every reason to congratulate one another!
Destruction of Documents
The team will be notified to destroy all examination materials, including the application, the Final Feedback Report you just received, and all work documents. We take these measures to protect the confidentiality of our applicants.
23
Keeping the Team Focused &
Productive
What does it take to run a strong, efficient MQA-KQA team? This chapter contains some general advice from MQA-KQA’s legacy of Team Leaders. Remember, if you have any nuggets to add, please send them to MQA-KQA!
Clearly set expectations. Tell team members what you expect from them, as well as what they should expect from each other.
If necessary, encourage team members. Tell team members to ask for help as soon as they realize they’re in trouble. Remind them that the sooner you’re aware of a situation that can hamper the team’s progress, the easier it will be to deal with it.
Mentor new Examiners. Do so from start to finish of your team’s assessment process. Ask other experienced Examiners on the team to help new Examiners. Touch base often with new Examiners to see how they are doing. Expect questions and coach new Examiners with respect to the process, procedures, expectations, and performance.
Keep the team informed. Keep the team posted about new information and feedback received from outside sources throughout the process. Even if you have no news to share, stay in touch. For example, if you are waiting for information from the applicant or another team member, let everyone know.
Be accessible. You expect team members to respond quickly. Treat them with the same respect. Keep up with email and phone messages.
Handle the “outlier.” After a review of independent review work, you may discover a team member who is not on task, doesn’t “get it,” or doesn’t seem to take his or her position on the team seriously. While uncomfortable, it may be necessary to do a quick “sanity check” phone call with this person to see if he or she is going to help or hinder the team as time goes on. In extreme cases, a team member may need to be asked to step down from the team. If you think this is the case, please notify MQA-KQA of the situation, and, if appropriate, MQA-KQA will ask the Examiner to make a graceful exit.
Chapter
5
24
Remind the team that OFIs are NOT negatives. Do this during consensus and subsequent team meetings. OFIs are defined as “opportunities for improvement.” And opportunities are always a good thing.
Think of it this way: The applicant is participating in the MQA-KQA program, presumably, to improve processes and results. If the applicant is committed to the process, it may set its sights on someday earning a Missouri or Kansas Quality Award – or even national recognition through the Baldrige Performance Excellence Program. The applicant needs to understand its OFIs to help it get there.
Help us Improve!
We would love to hear how you think we can improve this Team Leader’s Guide. The effectiveness of Team Leaders has a tremendous impact on whether Examiners – and applicants – stick with the MQA-KQA program to drive organizational improvement. Please help us build our knowledge base, improve the effectiveness of Team Leaders, and retain these valuable resources!
25
Appendix
Communication Templates
Use these communication templates as a guide to communicating with your team; but please use caution when using email templates to check for the following:
Ensure you change all references to your name and contact information, Examiner Team Number,
dates, meeting locations, and other team-specific information prior to sending. These areas have been
highlighted in yellow.
Verify all web links used to ensure they are valid.
Verify all instructions for Scorebook Navigator by accessing the system yourself before
communicating those instructions to team members. Remind team members that user guides are
available on the MQA-KQA Resource Center (https://midwestexcellence.org/volunteer-resource-
center/)
Please note: sample emails are meant to be used as references to guide you as you develop
team communications, so please minimize use of the “copy/paste” function whenever
possible—add/delete as necessary to make it ‘your own’.
26
MQA-KQA Team Leader Communication Overview
When To whom Format Subject Content & main points References
Immediately after receiving assignment
packet
Examiner Team
Email Welcome!
Reminder to verify no Conflict of Interest
Request to complete/send Biography Request and Calendar Request forms to Team Leader
Read application, start Key Factors
Enter Key Factors in Navigator
Other administrative details
Email template & Team Leader Guide, Chapter 1, Step 1
Within 1st week of receiving assignment
packet
Examiner Team
Individual phone calls
N/A
Get to know teammates personally – their experience and special areas of interest
Confirm contact information
Follow up if you haven’t received initial deliverables
Reassure new examiners to “trust the process”
Answer questions
Team Leader Guide, Chapter 1, Step 1
After receiving & reviewing application
Applicant
Phone
(Follow up with email)
Introduction
Overview of MQA-KQA process
Communication preference
Nuances of Site Visit – location(s), attire, etc.
Team Leader Guide, Chapter 1, Step 2
At beginning of independent
review
Examiner Team
Key Factors &
Independent Review
Due date for Independent Review
Tips and lessons learned
Email template & Team Leader Guide,
Chapter 2
Throughout independent
review
Examiner Team
Phone Status Follow up with teammates who are not on
track to ensure deadlines are met N/A
Near end of independent
review
Examiner Team
Email Getting
Ready for Consensus
Reminder of steps to finalize Independent Review
Category assignments
Consensus Meeting logistics
Schedule reminder
Email template & Team Leader Guide,
Chapter 3
After moving to consensus
Examiner Team
Email Focusing on Consensus
Cheer for and encourage your team
Tips for using Scorebook Navigator for Consensus
Due date for Key Factor review
Suggested interim dates for Consensus
Email template & Team Leader Guide,
Chapter 3
A few days before
consensus meeting
Examiner Team
See you at the
Consensus Meeting
Agenda for Consensus
What to bring
How to prepare
Proposed ground rules
Email template
27
After consensus;
2 weeks before site visit
Applicant
& MQA-KQA
Letter N/A Site Visit Issues
One week before site visit
Examiner Team
Email Final
Preparations for Site Visit
Site Visit Agenda
Other logistics
What to bring
Email template & Team Leader Guide,
Chapter 4
After submitting
Final Scorebook
Examiner Team
Email We’re
DONE!
Wait to hear from MQA-KQA before disposing of applicant materials
Submit hours worked
Complete team and peer evaluations
Email template
28
Welcome
Subject: [MQA-KQA XXXX] Welcome Examiner Team!
Greetings!
By now you should have received the package from MQA-KQA assigning you to applicant #XXXX. I’m looking forward to working with each of you and helping our applicant take the next step on its performance excellence journey!
Now that we’re off and running, please recall the following action items from Examiner Training:
1. If you’ve not yet done so, read the applicant’s Intent to Apply Form, Application Form, and Org Profile and return the Conflict of Interest form to MQA-KQA.
COI form completed Due Wednesday, 8/1
2. Use the Calendar Request Form to let me know the dates you ABSOLUTELY, POSITIVELY are not available over the next several months. We will need one day for the Consensus Meeting, and a four-day block for the Site Visit (three days)/Wrap-up meeting (one day). Coordinating schedules tends to be one of the most challenging aspects of leading a team, so I’d like to confirm dates on our calendars and our applicant’s calendar as soon as possible.
Calendar to Team Leader Due Wednesday, 8/1
3. To help us get to know each other, please complete the Biography Request Form and send it to our entire team. I have attached my biography. I will use your responses on this form to identify item leads for consensus.
Biography to Team Due Friday, 8/3
4. Identifying Key Factors is a critical first step as we each complete our Independent Review. In order to allow as much time as possible for Independent Review, PLEASE read the application from cover to cover and prepare your master list of Key Factors in Scorebook Navigator as soon as possible.
Key Factors in Scorebook Navigator Due Monday, 8/6
The sooner, the better, so if Fridays are quiet at work, or you can spend a little time this weekend, please make every effort to complete your Key Factors by the deadline.
a. You are all set up in Scorebook Navigator! Use the same user id and password that you used during training.
b. If you have trouble logging in to Scorebook Navigator, try clearing your cache/cookies and ensure you’re using the correct URL, https://www.scorebooknav.org/.
29
c. When you log in, Applicant XXXX should appear under Stage 1 of your Scorebooks screen. If not, please notify me immediately.
d. You can access MQA-KQA’s Examiner Handbook, on the Examiner Resources page of the MQA-KQA website. It contains step-by-step instructions for completing Independent Review. Also on the site is the Scorebook Navigator Handbook, with tips for navigating and troubleshooting Scorebook Navigator.
5. We will work backwards from our Site Visit date to see when the Independent Review phase must be completed, which we should know by next week. But, don’t delay in getting started! The Examiner Deliverables Checklist will help you keep track of our team’s calendar and due dates.
6. Finally, from an administrative standpoint:
Please use [MQA-KQA XXXX] in the subject line of all team communications. If you’re like me, you get a ton of email, and hopefully this will call our attention to important items from the team.
Please use “reply all” when an email is sent to the entire team to make sure everyone is kept in the loop.
And last but not least, please make every effort to meet all team deadlines! However, if something catastrophic happens in your life and you find you will miss a deadline, please notify me ASAP so we have time to figure out a Plan B.
I generally do NOT have access to this email address during the day (tomorrow will be an exception, as I am off). If you need to reach me urgently, feel free to send an email to both [home email address] and my work email [work email address] or call me on my cell: [cell phone number]
Again, I look forward to getting to know you and working with you throughout this process!
Feel free to call or email if you have any questions,
Your Name
Team Leader
MQA-KQA Applicant XXXX
30
Key Factors & Independent Review
Subject: [MQA-KQA XXXX] Key Factors & Independent Review
Hello, MQA-KQA Team XXXX!
Everyone completed their Key Factors AT LEAST one day ahead of schedule—way to go! And now we are off and running with Independent Review.
I have established a due date for the Independent Review for XXXX. That gives us four weeks from when you received your assignment, but the deadline could change once we finally schedule the Site Visit and work backwards to schedule our Consensus Meeting.
Reminders about Independent Review and some Lessons Learned:
CAUTION: Avoid using “our” and similar language when pulling info from the application. At this stage, it’s best to refer to our applicant as “the applicant” rather than its name or acronym. This helps to protect confidentiality and makes it easier to perform a “search and replace” for the final Feedback Report.
We ALL complete our Independent Review for ALL 17 Items. Please note this will take longer to complete than our training pre-work, so be sure to set aside sufficient time.
o Remember – we are one another’s customers for Independent Review, so it’s important that we have everyone’s input. For this reason, I’ll wait a little longer before assigning Category Leads and Backups (besides, with the logistical issues, I haven’t had a chance to do it yet!).
o We are outlining 6-8 suggested comments for each Item. These should be a combination of strengths and OFIs. Although we’re using bullets and short observation statements, we need to include enough information to clarify why it’s there. Please specify whether something is anecdotal, systematic, etc. Here is a sample of the level of detail to include:
Strength Evidence A D L I
Applicant uses the BCSLS to systematically review, update, and communicate its Mission, Vision, and Values
• Leadership system (Figure 1.1-1) establishes and communicates MVV
• Deployed to staff, volunteers, partners, and others; MVV and PhilPs posted in facilities and on internet site
• Refined through SPP; evidence of changes to vision 2 years ago
• Integrated with SPP and Shared Vision Development Process
X X X X
31
Strength Evidence A D L I
ESOP helps ensure high performance and an engaged workforce. Open communication is ensured through CNet and various 2-way communication methods.
Fig. 1.1-2 Methods of Communication
X X X
Remember to score each item!
I will follow up with each of you individually over the next week to check in and see how you’re doing, but feel free to call or email me if you have any questions.
Don’t forget to mark “done” when you have completed your independent review!
Also, I’ve contacted our applicant and confirmed the dates for our site visit. Stay tuned!
Keep up the good work!
Team Leader
Cell
32
Getting Ready for Consensus
Subject: [MQA-KQA XXXX] Independent Review Wrap-up - Getting Ready for Consensus
Hello MQA-KQA Team XXXX!
Just a reminder to check “done” for your independent review and to be sure to record your hours for this stage.
In case some of you don’t get this message until tomorrow, I’ll check back Monday morning to advance us to the next stage in the system. The Category leads/backups and Item assignments based on your bio information are as follows:
Items Primary Backup
1.1, 1.2, and 7.4 Person 1 Person 2
2.1 and 2.2 (Look at 7.1c) Person 2 Person 1
3.1, 3.2, and 7.2 Person 3 Person 4
4.1, 4.2, and 7.5 Person 5 Person 6
5.1, 5.2, and 7.3 Person 4 Person 3
6.1, 6.2, and 7.1 Person 6 Person 5
It looks like our Consensus Meeting will be on XXXX. We can discuss plans for our post-Site Visit meeting during the Consensus Meeting.
Date of Consensus Meeting: Day, Date
Teleconference Information: Call-in number
Time: 8:30 a.m. Central until we’re done (Agenda to be finalized)
Here is a run-down of the dates still in front of us:
XXX – Category/Item Lead Assignments for Consensus made in Scorebook Navigator
XXX – Consensus Feedback-Ready Comments complete (and ready for review by your backup)
XXX – Backups provide Feedback to Category/Item Leads
XXX – Category/Item Leads incorporate Feedback from Backup (and ready for entire Team to review prior to Consensus Meeting)
XXX – Consensus Meeting
33
XXX – All Site Visit Issues and interview and document requests completed in Scorebook Navigator. Team Leader sends key Site Visit Issues to Applicant
XXX – Category/Item Leads complete Comment Revisions (as necessary); Team Leader contacts MQA-KQA to print consensus scorebook. Consensus Scorebook Complete.
XXX –) Site Visit Issue Worksheets completed.
XXX – Site Visit
XXX – Category/Item Leads revise Comments and complete Site Visit Issue documentation in Scorebook Navigator so Team Leader and Scorebook Editor can finalize Scorebook.
XXX – Submit Final Scorebook to MQA-KQA
Thanks, all!
Team Leader
Cell
34
Focusing on Consensus
Subject: [MQA-KQA XXXX] Focusing on Consensus! Three Cheers for MQA-KQA Team XXXX!
So, here’s what you need to know (maybe MORE than you need to know) about the Consensus phase prior to our Consensus Meeting:
1. Item assignments have been made per XXX email. When you log in to Scorebook Navigator, you will see your assigned items under “Stage 2” on the Scorebooks screen.
2. You will have editing rights for your assigned items (as primary) and read-only access to the other items (via the Item drop-down box).
3. When you click one of your links, you will see a blank screen to enter KF, Strengths, OFIs and score. To view team members’ Independent comments for your assigned item, click on the gold “1.”
o TIP: You can sort your teammates’ strength and OFI comments by examiner or Item Reference by selecting the related header. MQA-KQA recommends sorting by item reference.
o BIG TIP: You have the ability to print out your teammates’ Independent Review input for your Item synthesis. Use the “print preview” icon on the gold menu bar – and remember to sort the strengths and OFIs tables separately. Disregard the feedback-ready comments prepared by your teammates while you are selecting your 6-10 comment topics.
4. Select up to 6 of the most important (relevant) Key Factors for each of your assigned items.
5. The end product for each item at this stage is 6-8 comments (key messages) and rationales. 6 to 8 is the TOTAL for strengths and OFIs, based on integrating the independent evaluations and focusing on the most important feedback for the applicant.
o TIP: To save on typing, you can highlight-copy-and-paste information from independent review input.
o TIP: Keep the initials of examiners whose independent review work was incorporated into the comments you develop.
o TIP: Place notes about your synthesis (what was used/not used and why) in the “Notes” field below your completed comments.
6. When you recommend a consensus score for your items, it should be consistent with the suite of comments (Strengths & OFIs) you are recommending. As you might recall from training: “It’s not an average!”
35
7. Email your back-up when are ready for your work to be reviewed. The back-up’s feedback should be entered in the “observations” box after each comment.
8. Do NOT click “Done” – – I’ll do that once for the full Consensus report.
Please complete your work by the following deadlines, so we can have an efficient Consensus Meeting:
XXX – Consensus Feedback-Ready Comments complete (and ready for review by your backup)
XXX – Backups provide Feedback to Category/Item Leads
XXX – Category/Item Leads incorporate Feedback from Backup (and ready for entire Team to review prior to Consensus)
Feel free to work with your Item Backup to improve your work, as long as we have “final” comments available to the whole team to review no later than XXX. That gives us two days to review the scorebook prior to our Consensus Meeting, which should ensure that we have an efficient meeting.
Remember:
CONSENSUS: XXX, READY TO GO at XXX a.m. Central
Thank you all for your efforts.
Keep up the exceptional work on behalf of our applicant and MQA-KQA!
Team Leader
Cell
36
‘See You’ at the Consensus Meeting
Subject: [MQA-KQA XXXX] ‘See You’ at the Consensus Meeting – Day, Date
MQA-KQA Team XXXX is doing great!
Well, we’ve been on our own little island for Independent Review. Then we found refuge in synthesizing the evaluations and working with our backups to prepare for Consensus. Now, we get the full benefits of being part of a TEAM of excellent examiners!
The Consensus Meeting is a few short days away, so what follows is some logistical information as well as tips and suggestions to help make it a productive
day. You might want to print this message.
Date: XXX
Conference Telephone Number: Conference Number
If you have an emergency, call Team Leader at [cell phone number] (mobile)
Agenda:
8:00 – 8:30 a.m. ‘Meet-and-Greet’ Team
8:30 – 8:45 a.m. Review/Revise Ground Rules Team Leader
8:45 – 9:45 a.m. Items 5.1 – 5.2 – 7.3 Team Leader
9:45 – 10:45 a.m. Items 1.1 – 1.2 – 7.4 Category/Item Lead
10:45 – 11:00 a.m. Break/Catch Up
11:00 – 11:40 a.m. Items 2.1 – 2.2 Category/Item Lead
11:40 – 11:55 a.m. Key Themes Team Leader
12:00 – 12:30 p.m. Lunch Break
12:30 – 1:30 p.m. Items 3.1 – 3.2 – 7.2 Category/Item Lead
1:30 – 2:30 p.m. Items 4.1 – 4.2 – 7.5 Category/Item Lead
2:30 – 2:45 p.m. Break/Catch Up
2:45 – 3:45 p.m. Items 6.1 – 6.2 – 7.1 Category/Item Lead
3:45 – 5:00 p.m. Next Steps Team Leader
Key Themes
Finalize Consensus Scorebook
Site Visit Overview
Site Visit Issue Worksheets
What to Have Available and Ready:
Your copy of the application
37
Your copy of the Framework book our applicant used to write its application
If you do not use a printed copy of the full Pre-Consensus Scorebook, it will be helpful to at least have a hard copy of the Key Factors list.
How to Prepare for the Consensus Meeting:
1. Review Applicant Responses to all items (important for the items you have not been focusing on as Lead or Backup)
2. Review the Draft Consensus Scorebook (from Scorebook Navigator)
3. Note areas of uncertainty, disagreement, or conflicts by Item so we can address them as we go along.
Review Consensus Meeting Topics for areas that should be part of the discussion.
4. Be on the look-out for potential Key Themes
5. As we go through each item, we will have Scorebook Navigator up on the screen. However, for the sake of time we will not be making real time changes during the meeting.
6. Refer to the Consensus Script for a role model approach to lead your Items in the Consensus Meeting. Note that I will go first with Category X to demonstrate this approach.
Proposed Ground Rules:
We will be present and attentive to team discussion.
We will NOT read all information aloud, but will summarize key points.
We will not be defensive.
We will keep the applicant’s best interest our focus.
We will verbally agree on comments and scores.
We will keep the impact of score variations in perspective. Please let me know if you have any questions.
I look forward to XXX!
Team Leader
Cell
38
Final Preparations for Site Visit
Subject: [MQA-KQA XXXX] Final Preparations for Site Visit
MQA-KQA Team XXXX,
As we get ready for Site Visit, I want to provide you all with an update on where we stand:
1. The Consensus Report is complete. Thanks for all your work in your Category and Backup areas!
2. Below is the email I sent to our applicant’s Point of Contact. It includes a draft agenda and questions from our OFIs and significant strengths. You’ll notice I used “describe” and “discuss” interchangeably, rather than “clarify” or “verify” in an attempt to make our meetings with the applicant more conversational and to not tip-off whether we had Strengths or OFIs.
3. Logistics:
The applicant has provided us a conference room to house our things and use as a base of operations.
We start at XXX on XXX. Please plan to meet me in the lobby of the hotel at XXX. We will then head to our team meeting room.
We can stay a little late in the conference room, if needed.
The conference room will have wireless access.
Meal logistics (if needed)
4. Don’t forget to bring:
Your copy of the application
Your MQA-KQA Name Badge
Copies of your completed Site Visit Issues. You will use these worksheets to guide your interviews and to take notes. We will document the resolution of each site visit issue in Scorebook Navigator AFTER the site visit is complete.
Also, I encourage you to re-read the applicant’s Organizational Profile while you prepare for our time on-site with our applicant.
Please let me know if you have any questions.
Team Leader
Cell
39
We’re DONE!
Subject: [MQA-KQA XXXX] Done! Done! Done!
Thanks, everyone!
I pressed “Done!” If you are interested in the Final Scorebook, you can go into Scorebook Navigator and view it. If you print it, just don’t forget to delete when MQA-KQA gives us the okay.
I’m proud of our final product, and I hope you are too. MQA-KQA will do a final edit and revise the formatting before sending to our applicant.
Thanks again for all your efforts on behalf of our applicant. I look forward to our paths crossing again in the future!
Team Leader
P.S. For the first-timers: You were outstanding! The process can be a little overwhelming, but it works. . . AND it gets easier with practice. I sincerely hope to see you back on the Board of Examiners next year.
Sample Consensus Schedule
Team XXXX
Owner
8:00 – 8:30 a.m. Arrival, Ground Rules, Ice Breaker Team Leader
8:30 – 9:10 a.m. Category 1 and item 7.5 review Examiner 1
9:10 – 9:15 a.m. Stretch break ALL
9:15 – 10:15 a.m. Category 2 and item 7.4 review Examiner 2
10:15 – 11:15 a.m. Category 3 and item 7.2 review Examiner 3
11:15 – 11:25 a.m. Stretch break (catch up if necessary) ALL
11:25 – 12:05 p.m. Category 4 Examiner 4
12:05 – 12:45 p.m. Lunch break (catch up if necessary) ALL
12:45 – 1:25 p.m. Category 5 and item 7.3 Examiner 6
1:25 – 2:25 p.m. Stretch break ALL
2:25 – 2:35 p.m. Category 6 and item 7.1 Examiner 7
2:35 – 3:15 p.m. Key themes Team Leader
40
3:15 – 5:00 p.m. Discuss next steps:
Site Visit
Site Visit Issues
Updating Consensus Scorebook
Team Leader
Site Visit Issue Tracking
Compile a spreadsheet listing SVIs and send it to the applicant at least one full week prior to the site visit to allow the applicant time to prepare. It could look something like this:
Category Site Visit Issue Topics Contact in Applicant’s Organization
Keeping up with Site Visit Issues
Create a table to track progress as the team resolves site visit issues. For example:
Item ref.
Site Visit Issue Key Contacts
Documents required
Key word
1.1-1 Verify process improvement process with Six Sigma and ISO
Senior leaders
Six Sigma
1.1-2 Verify deployment of senior leaders’ communications to the workforce
Senior leaders
Staff employees
Booklets containing vision, mission, values, code of ethics
Communication
1.1-3 Clarify how employee evaluations are conducted & how info from performance measures is used to accomplish objectives, improve performance, attain the vision
Senior leaders
Staff employees
HR manager
Examples of evaluation tools
Employee performance