MEASURING NONCOGNITIVE BEHAVIORS AND CHARACTERISTICS IN ALLIED

HEALTH PROGRAM ADMISSIONS PROCESSES

by

DEANNE DOTSON COLLINS

(Under the Direction of Karen E. Watkins)

ABSTRACT

Screening processes for competitive adult education health professional programs are a

necessity. Appropriately screening candidates for these professional programs makes the best

use of faculty and institutional resources and better ensures future success of the student.

Gardner’s (1983) theory of emotional intelligence provides a conceptual framework for the

assessment of noncognitive behaviors and characteristics. An action research case study using

mixed methods examined alternative measures of noncognitive behaviors and characteristics for

allied health programmatic admissions processes. Working over a three year time period, an

action research team of healthcare professionals collaboratively identified and implemented

alternative measures to the interview. The Emotional Quotient Inventory ® (EQi), (Bar-On,

1988) was chosen to measure noncognitive behaviors because it is a standardized assessment

proven to be both valid and reliable. A total of 40 allied health students participated as well as

five action research team members who are all healthcare faculty. Using the Statistical Package

for the Social Sciences, results of the EQi were compared to student interviews and clinical

evaluations. Within this study, the results showed that, when compared to programmatic

interviews, the EQi was not strongly correlated with the interview tool currently used in one of

the allied healthcare programs, though it correlated with the clinical evaluations in the other

program. More research needs to be conducted given the small size of the sample and the

differing results for each program. Based on these findings, action research team members

improved current assessment tools used in each program, and are implementing an assessment

that measures these behaviors in other programs. Recommendations for further study include

adoption of the assessment model used here that monitors the relationship between structured

admissions interviews and mid-program clinical performance. Finally, more research is needed

on the use of the EQi as one component of a multi-dimensional admissions process in healthcare

fields that depend on both cognitive and emotional intelligence.

INDEX WORDS: Medical Program Admissions, Interviews, Noncognitive Behaviors,

Emotional Intelligence, Action Research

MEASURING NONCOGNITIVE BEHAVIORS AND CHARACTERISTICS IN ALLIED

HEALTH PROGRAM ADMISSIONS PROCESSES

by

DEANNE DOTSON COLLINS

BS, Columbia Southern University, 2005

MSRS, Midwestern State University, 2008

A Dissertation Submitted to the Graduate Faculty of The University of Georgia in Partial

Fulfillment of the Requirements for the Degree

DOCTOR OF EDUCATION

ATHENS, GEORGIA

2013

© 2013

Deanne Dotson Collins

All Rights Reserved

MEASURING NONCOGNITIVE BEHAVIORS AND CHARACTERISTICS IN ALLIED

HEALTH PROGRAM ADMISSIONS PROCESSES

by

DEANNE DOTSON COLLINS

Major Professor: Karen E. Watkins

Committee: Khalil M. Dirani

Wendy E. A. Ruona

Lorilee R. Sandmann

Electronic Version Approved:

Maureen Grasso

Dean of the Graduate School

The University of Georgia

May 2013

iv

DEDICATION

This dissertation is dedicated to my amazing family and husband, who have provided

love, support, and encouragement which have translated into my confidence and success

throughout this journey. These loved ones have seen and nurtured my gifts and have exhorted

me to be better than I could possibly imagine and have encouraged me the most.

To my precious daddy, Dywane B. Dotson, you are my heart and I am my father’s

daughter. You have continually pushed me towards a higher level of performance, and have

shown me through your many years of hard work and dedication, what one can accomplish and

be proud of. You wear your Southern Tech college ring to this day, which is a constant reminder

to me that you pushed yourself and never gave up, even when times were tough. I don’t think

you realize that although I know how proud of me you are, I am even more proud of, and

humbled by you. I love you daddy and will always be a “daddy’s girl,” no matter my age.

To my cherished mama, Glenda Dotson, thank you for your many prayers and words of

faith. You are a godly woman of grace; your thoughtful and caring ways have made such a

difference to others. Your many small kindnesses have touched numerous lives, especially mine.

I love you and appreciate the hours you have spent in prayer asking God to give me the support I

needed to make it through this degree. Without them, this would not have been possible.

To my big “little brother” Shane Dotson, I am grateful to you for your quiet and gentle

support of my achievements. In the span of a lifetime, there are few people who know you like a

brother. I pray that I have always been someone to whom you could look up to and hope that my

accomplishment will be an inspiration to you in your future endeavors. I am so proud of you and

v

who you have become today, and who I know you will be in the future. Thank you for allowing

me, today, to be a small part of what you strive to be tomorrow. I love you dearly.

For my wonderful husband, Brian Dean, who has loved me and motivated me throughout

the years of this endeavor, I thank you for your endless support and encouragement. With every

step of this process, you have been right beside me, embracing each of my successes as your

own. The smiles and laughter we create together have attributed to my accomplishment in this

program. You continue to remind me to take one day at a time and I love you for it.

Most importantly, I dedicate this work to my Lord and Savior, who has sustained me and

strengthened me through this life-changing journey. “For I know the plans I have for you, says

the Lord, plans to prosper you and not to harm you, plans to give you hope and a future.”

(Jeremiah 29:11)

vi

ACKNOWLEDGEMENTS

First, and foremost, I would like to thank my major professor, Dr. Karen Watkins, who

has spent countless hours guiding me through this process, supporting and encouraging me along

the way, and who has been honest in telling me to “relax” when necessary. You have set a fine

example of a woman of great accomplishment and I aspire to your caliber of leadership.

I would like to thank my committee members, Dr. Khalil Dirani, Dr. Wendy Ruona, and

Dr. Lorilee Sandmann for their time and continued support given to me during this journey; a

special thanks to Dr. Khalil Dirani “the numbers guy” for the time he devoted to my quantitative

results. I would also like to recognize the members of my action research team; each of you has

provided valuable insight, knowledge, and wisdom and has helped to make this research

possible.

I must acknowledge my fellow classmates at UGA for their support and encouragement

throughout this process. Being part of the first cohort of this particular program will always be

special, but what has made it even more special are the friendships which I have formed with my

doctoral classmates which will last a lifetime.

To my past, present, and future radiography students and graduates, remember to always

value the patient as if they were your own family member and continue to strive for excellence in

the healthcare profession. Thank you for your support of my educational pursuit.

Finally, I would like to acknowledge the interview panel of the first cohort of the

University of Georgia’s Ed.D program in Adult Education for possibly recognizing certain

noncognitive behaviors or traits in me that would make me a good “fit” for their program.

vii

TABLE OF CONTENTS

Page

ACKNOWLEDGEMENTS ........................................................................................................... vi

LIST OF TABLES ...........................................................................................................................x

LIST OF FIGURES ....................................................................................................................... xi

CHAPTER

1 INTRODUCTION .........................................................................................................1

Issue Identification ...................................................................................................2

Purpose and Research Questions .............................................................................6

Theoretical Base for Study.......................................................................................7

Significance............................................................................................................14

2 LITERATURE REVIEW ............................................................................................16

Competitive Admissions Processes .......................................................................17

Noncognitive or Affective Behaviors and Characteristics Defined .......................20

Assessment of Noncognitive Behaviors and Characteristics .................................28

Summary of Literature Review ..............................................................................40

3 METHODOLOGY ......................................................................................................44

Action Research Methodology ..............................................................................45

Qualitative Methods ...............................................................................................52

Quantitative Methods .............................................................................................62

Limitations of the Study.........................................................................................71

viii

4 CONTEXT AND CASE ..............................................................................................74

Description of the Context .....................................................................................75

Development of the Action Research Process .......................................................79

The Action Research Team Members....................................................................84

Action Research Team Intervention Option ..........................................................93

The Beginning of the Story; Am I in the Right Frame of Mind? ........................102

Reflection in Action .............................................................................................103

Conclusion ...........................................................................................................105

5 FINDINGS .................................................................................................................106

What are the Noncognitive Behaviors and Characteristics for Healthcare? ........107

What Noncognitive Behaviors are assessed in an Interview? .............................109

Relationship between EQi Scores and Various Assessments ..............................111

Action Research Team Process ............................................................................116

Conclusion ...........................................................................................................151

6 CONCLUSIONS AND IMPLICATIONS .................................................................152

Summary of Findings ...........................................................................................153

Study Conclusions ...............................................................................................159

Implications and Recommended Next Steps for the Organization ......................165

Implications for Practice ......................................................................................168

Leader of Change on the “Inside” ........................................................................169

Conclusion ...........................................................................................................171

REFERENCES ............................................................................................................................172

ix

APPENDICES

A Consent Form for Action Research Team Members .................................................180

B Critical Incident Interview Questions ........................................................................182

C Verbal Script ..............................................................................................................183

D Consent Form for Participants of Study.....................................................................184

x

LIST OF TABLES

Page

Table 1: Health Occupations Basic Entrance Test Skills.................................................................3

Table 2: Bar-On Emotional Quotient Inventory ® Categories ......................................................26

Table 3: Literature Review Table of Key Empirical Studies .........................................................32

Table 4: Action Research Process Stages ......................................................................................47

Table 5: Action Research Team Member Profile ..........................................................................49

Table 6: Sample of Master File of Coded Data .............................................................................60

Table 7: Summary of Assessment Tools and Participants .............................................................69

Table 8: Team Intervention Options ..............................................................................................95

Table 9: Action Research Process Timeline ................................................................................100

Table 10: Findings of Noncognitive Behaviors and Characteristics in the Literature .................108

Table 11: Findings of Noncognitive Behaviors Assessed During Interview ...............................110

Table 12: Correlations between EQi, Clinical, and Interview for Group A ................................113

Table 13: Correlations between EQi, Clinical, and Interview for Group B ................................115

Table 14: Table of Findings .........................................................................................................118

xi

LIST OF FIGURES

Page

Figure 1: Conceptual Framework ....................................................................................................9

Figure 2: Operational Framework ..................................................................................................48

Figure 3: Conclusive Findings .....................................................................................................162

1

CHAPTER 1

INTRODUCTION

Imagine you or a loved one becoming very ill or suffering a traumatic life threatening

event in which a part of your body is hurt or injured. As you arrive at the hospital, you are met

by an allied health professional, whether it is a nurse or a radiographer or a doctor, who upon the

initial assessment, barely looks you in the eye or speaks to you in a condescending manner.

Perhaps this person is rude and seems to show no care for your pain or the feelings you are going

through. Is this the way a patient can expect to be treated in the healthcare setting? How can

healthcare related educational programs ensure that the best applicant to the program has the

necessary noncognitive skills related to caring for others? These questions gave rise to this action

research study and began the initial engagement with the client system.

Allied health programs in adult educational settings are under pressure to select, accept,

retain, and graduate the most qualified students for the profession. Healthcare needs are rising

and as the large number of baby boomers begin to seek medical care, it becomes even more

important to equip society with suitable caregivers. Suitable caregivers not only need to have the

necessary cognitive abilities to perform the job, but it is also crucial that they have noncognitive

traits which are relevant to the profession. A detailed search of the literature indicates that

admissions processes for allied health programs in secondary educational institutions should

include both assessments of cognitive and noncognitive characteristics of the applicant (Joyner,

Cox, White-Harris, & Blalock, 2007; Salvatori, 2001; Storey, 2008; Weege, 2009). Cognitive

characteristics encompass brain skills that make it possible for us to think and learn and can

2

easily be measured with pen and paper tests, whereas noncognitive characteristics include

emotional behaviors such as empathy, compassion, self-awareness, self-control and altruism.

There are several terms in current use that describe behaviors or characteristics in the

affective domain and which are not viewed as cognitive in nature. I located over ten different

terms with “emotional intelligence” being used most often followed by “noncognitive variables.”

The ones identified in the literature are noncognitive traits, noncognitive behaviors, personal

qualities, personality attributes, non-intellective factors, attitude, interpersonal dimensions,

noncognitive variables, noncognitive abilities, noncognitive factors, noncognitive skills, personal

characteristics, and noncognitive qualities. These terms are used within the literature

interchangeably although some can be defined as influence-able and others are not. I have

chosen to use the term “noncognitive behaviors or characteristics” for this dissertation for

pragmatic reasons because it is an umbrella term for the others.

Issue Identification

Most authors agree it is difficult to assess noncognitive behaviors or characteristics,

particularly through a non-subjective method (Edwards, Johnson, & Molidor, 1990; Weege,

2009; Wright & Miederhoff, 1999). This leads to the question of how to define noncognitive

behaviors or characteristics as well as how to measure them. This action research case study

focused on the evaluation of admissions processes of healthcare programs within a technical

college in the southeastern United States, and how noncognitive behaviors are measured for

admission to these programs.

Within this technical college, some healthcare programs utilize an interview component

as part of their competitive admissions process. However, not all programs currently use an

interview or any means of measuring noncognitive behaviors. The interview component was

3

recently eliminated from the competitive admissions process for one of the healthcare programs

at the researcher’s institution. This decision was implemented by administration at the college

due to concerns about liability. Program faculty, as well as the program director, was not in

agreement with the decision. This program now only uses the grade point average of specific

pre-requisite coursework as well as scores on a standardized test. The students with the highest

scores and the highest grade point averages are the students who are offered admission into the

program. For now, there also is no alternative measurement of a student’s noncognitive or

interpersonal skills prior to the admissions of the program. These behaviors are measured once a

student is a part of the program and working with actual patients in a live clinical setting, but not

prior.

Students applying to the various healthcare programs are required to take a standardized

test such as the HOBET (Health Occupations Basic Entrance Test.) This assessment tests their

cognitive skills. Table 1 outlines the list of skills measured with the HOBET exam.

Table 1

Health Occupations Basic Entrance Test (HOBET) Skills Measured

HOBET Skills Measured

Basic algebra

Advanced algebra

Fractions

Basic math

Intermediate math

Reading speed

Reading comprehension

Reading accuracy

Percents

Ratio

Ability to make inferences

Grammar

Vocabulary

Syntax

4

Part of the significance of this study was the problem facing those of us who were using

some sort of assessment of noncognitive behaviors or characteristics. Because administration

had removed the interview from other programs, discussion began surrounding the possibility

that this decision would be made for all healthcare programs. Fellow educators expressed

concern that the programs were only accepting students with high GPA’s and throughout the

course of the program, were losing half the class due to disruptive clinical behaviors stemming

from noncognitive variables which had not been measured prior to acceptance. This decrease in

students negatively affects programmatic accreditation due to a low retention and high attrition

rate. This study began with the idea that if the decision to eliminate the interview from all

programs came to pass, I would be able to provide the college with an alternate means of

assessment.

Based on the aforementioned issues as well as information gathered in an in-depth review

of the literature, the interview surfaced as the most utilized means for measuring noncognitive

behaviors although, such practices have been met with controversy. Gaps in the research related

to the use of an interview were identified in specific healthcare professions. While information

was plentiful in other healthcare related occupations, a specific gap exists in the literature which

explores liabilities as they relate to the use of this controversial tool. In addition, research

comparing self-assessments and interviews or other evaluations of an applicant with future

performance in the field are almost non-existent.

Identification of Major Stakeholders

There were several stakeholders, or audiences, in the research project. These included

the program directors of the healthcare programs and the program faculty and instructors of the

programs. They are directly responsible for the classroom instruction component of the

5

programmatic curriculum. They have the duty of teaching current students any and all didactic

information in order to competently take care of a potential patient. Program directors were also

interested to know if the deletion of the interview component has an effect on the program, its

faculty, its clinical instructors, and its graduates. This information could help to determine their

future admissions plans and processes.

Another important stakeholder in this evaluation were the clinical instructors located at

the clinical facilities where these healthcare students complete their required clinical hours and

training. These instructors are responsible for the students while they are practicing their

didactic skills on patients in the clinical setting. These instructors have a heightened interest in

the knowledge of this research because they spend the most time with the healthcare students as

they care for actual patients. This would be the area where the student must not only perform

cognitive tasks as they relate to the care of a patient, but also exude appropriate interpersonal

skills and behaviors when working with patients and fellow caregivers.

Another potential stakeholder in this evaluation was the students themselves. Because

they are directly affected by program admissions processes, they benefit from knowing if the

interview is a successful predictor of classroom and clinical success, per the program faculty and

clinical instructors.

Administration of the college, which includes the Vice President of Academic Affairs as

well as the Dean of Allied Health Sciences, could also be viewed as potential stakeholders. They

have direct involvement in permitting the healthcare program directors to utilize a noncognitive

measurement tool, such as the interview, in order to determine the accepted students to a

healthcare program.

6

This institution offers several different healthcare programs including but not limited to:

associate degree nursing, emergency medical technician, health information technology,

medical assisting, occupational therapy assistant, paramedic technology, physical therapist

assistant, practical nursing, radiography, and surgical technology. Although research may not

be conducted on all healthcare programs within the college, the client system of “all medical

programs” would benefit from this research because they all have a process of admitting

healthcare students to their programs. The client system of healthcare programs is managed by

the Dean of Health Sciences who is also interested in the outcome of this research.

Purpose and Research Questions

The purpose of this mixed methods study was to collaborate with an action research team

to examine the relationship between a self-assessment of inter/intra personal skills and various

other assessments which included clinical evaluations, interviews, and mid-term assessments. By

implementing an action research team, the researcher gained the opportunity to work with others

who had a real interest in the original research problem. Coghlan and Brannick (2010) describe

the foundation of action research as a collaborative action relationship between the researcher

and members of an organization who work together to address an issue, solve a problem, and

generate new knowledge in the process.

As this action research study sought to determine the best method for measuring

noncognitive behaviors as well as to determine what specific noncognitive behaviors are suited

for healthcare workers, the following research questions were developed:

(1a) What are the noncognitive behaviors and characteristics identified in the literature

for healthcare professions?

7

(1b) What are the noncognitive behaviors assessed during the interview component of a

competitive healthcare program?

(2) Is there a relationship between EQi scores and ratings on interviews, mid-term

assessments, or clinical evaluations?

(3) In what ways does an action research team process lead to individual, group, and

organizational learning and change?

Throughout the course of the investigation, the research questions were modified to better

reflect the problem and guide the action research process. Based on action research team

meetings as well as feedback from the dissertation advisory committee, the questions became

more focused and later included the specific assessments to be used in the study. With respect to

the purpose and research questions of this study, theoretical and empirical literature was used to

conduct a thorough review of noncognitive behaviors measured for competitive medical

programmatic admissions processes.

Theoretical Base for Study

The theoretical foundation for this study was based on Howard Gardner’s (2011) theory

of Multiple Intelligences. Gardner suggested that human beings possess various intelligences

and are not limited to only intellectual intelligence. Gardner proposes that there is not just one

specific kind of intelligence that is crucial for life success, but actually a whole spectrum of

intelligences, with seven key varieties (Goleman, 1995). In his writings, he describes his theory

of seven intelligences with demonstrations and discusses each intelligence by providing

examples and discussing their importance and use in diverse cultural settings. Gardner suggests

multiple intelligences which include: linguistic intelligence, musical intelligence, logical-

8

mathematical intelligence, spatial intelligence, bodily-kinesthetic intelligence, and the personal

intelligences.

Gardner’s (2011) personal intelligences are broken down into two areas, intrapersonal

intelligence which is access to one’s own feelings or the internal aspects of a person and

interpersonal intelligence which is the ability to notice and make distinctions among other

individuals. Gardner writes in his popular “Frames of Mind,” “we see highly developed forms of

interpersonal intelligence in political and religious leaders (Mahatma Gandhi or Lyndon

Johnson), in skilled parents and teachers, and in individuals enrolled in the helping professions,

be they therapists, counselors, or shamans” (p.253). This statement supports the action research

team discussions which have been that a healthcare provider, who can be classified in a helping

profession, must have good interpersonal skills.

This theory forms the basis for my conceptual framework which is illustrated in Figure 1.

The figure illustrates that if a student has high intra/interpersonal intelligences (emotional

intelligence) then they are more likely to succeed in healthcare professions, which then raises the

question of what is the best way or how can we measure/screen for this?

9

Figure 1. Conceptual framework depicting theory of how to best assess personal intelligences,

which are part of Howard Gardner’s Theory of Multiple Intelligences.

Howard Gardner’s (2011) research suggests that human beings are born with multiple

intelligences, and emotions are a type of intelligence that are controlled by the brain and are

separate from cognitive intelligence. Daniel Goleman (1995) builds from this theory in his book,

“Emotional Intelligence, Why it Can Matter More Than IQ.” Goleman suggests that we have

Gardner’s

Theory of

Multiple

Intelligences

Linguistic

Musical

Logical

Mathematical

*Personal

(Inter/Intra)

Spatial

Bodily-

Kinesthetic

IF S

tron

g …

Th

en

Successful

Performance

in Healthcare

Professions

How Can We

BEST ASSESS

these

Intelligences?

10

two minds, the emotional mind and the rational mind, one mind for thinking and one mind for

feeling. For instance, surgery, injury, or pathology to the brain and areas such as the frontal lobe

can and do affect emotions. Gardner (2011) adds to this theory by suggesting that people’s brain

sizes, shapes, and processing routes, along with their genetic makeup affects these different

intelligences. He maintains that personal intelligences, such as emotional intelligence, are rooted

in biology and, vary as do other types of intellectual abilities.

This concept is further evidenced by research of the anatomy and physiologic functioning

of the brain. Defects in the frontal lobe of the brain can interfere with the development of

personal forms of knowledge and can also cause problems with intrapersonal and interpersonal

behaviors (Goleman, 1995). Gardner (2011) writes “it has been known for well over a century

that destruction of the frontal lobes in an adult exerts only relatively minor effects on that

individual’s ability to solve problems (such as those featured on a standard intelligence test) but

can wreak severe damage on his personality” (p. 275). A member of the action research team

supported the frontal lobe damage findings and stated “not every 4.0 student who does well on a

standardized test is cut out for the medical field.”

Goleman (1995), who has conducted extensive research on emotional intelligence, writes

“Howard Gardner told me, the time has come to broaden our notion of the spectrum of talents.

The single most important contribution education can make to a child’s development is to help

him toward a field where his talents best suit him, where he will be satisfied and competent” (p.

37).

Theory of Emotional Intelligence

The search for a deeper understanding of the term “emotional intelligence” as it relates to

noncognitive behaviors of a medical professional provides a theoretical rationale for the research

11

study. According to Akhouri and Singh (2010) emotional intelligence can be described as a set

of competencies which can demonstrate one’s ability to recognize his or her moods and

behaviors as well as how to manage them based on the situation at hand. Psychologists and

medical doctors have proven that there are specific traits that can be called emotional intelligence

that are responsible for the way people behave, feel, relate to others, perform at jobs, and that

define their degree of health. Bar-On (2006) writes that these non-intellective factors influence

intelligent behavior as well and that all models of human behavior are influenced in some way by

predictors and facilitators such as predisposition, cognitive intelligence, personality, motivation,

and environment. Birks, McKendree and Watt (2009) argued that life success depends more on

emotional intelligence than cognitive intelligence.

The term emotional intelligence was initially coined by Wayne Payne in his doctoral

thesis on a study of emotion, from the 1980’s and was later popularized by Daniel Goleman’s

bestselling book on emotional intelligence (Akhouri & Singh, 2010). Reuven Bar-On (1988), a

clinical psychologist, also described emotional intelligence in his doctoral dissertation and

created the term “EQ” or emotional quotient in 1985 to describe his approach to assessing

emotional and social functions.

Cadman and Brewer (2001) discussed emotional intelligence as it relates to recruitment

for nursing programs. They, along with Wilson and Carryer (2008) believed that emotional

intelligence or emotional competence is an absolute necessity of nursing educators as well as

pre-requisite nursing students and current nursing students. Cadman and Brewer feel that

selection processes for these medical programs need a component that determines levels of

emotional intelligence in prospective candidates. This could be a reliable predictor of success in

clinical and didactic settings. Because the field of nursing is part of the larger overall field of

12

healthcare, other healthcare programs, such as radiography could be viewed as needing the same

component.

Emotionally intelligent human beings excel in human relationships and perform well at

work; (Cadman & Brewer, 2001) therefore emotionally intelligent clinical nursing students are

more likely to be able to cope with clinical practice. If attrition rates for student nurses are to be

reduced, there must be some assurance that the ones admitted to study in these programs can

demonstrate a clear potential to succeed in practice and theory (Cadman & Brewer, 2001).

Wilson and Carryer (2008) add to this by noting that if nursing students want to gain confidence,

effective communication skills, and the ability to convey hope and regard for others, they will

need to be able to understand their own emotions. When nurses or other healthcare workers

learn to process their own emotions, they are better able to deal with and work with a patient’s

emotions. The emotional state of a patient is often heightened during illness or medical

procedures, therefore medical professionals should be able to effectively deal with these

emotions and respond with emotional intelligence traits such as empathy and compassion.

Wilson and Carryer (2008) also suggest that because emotional intelligence takes time to acquire

and training of communication skills within nursing programs is insufficient, testing for

emotional intelligence should occur prior to acceptance into a program.

Bar-On Emotional Quotient Inventory

Reuven Bar-On (1988) developed the Bar-On theory of emotional and social intelligence

which describes emotional intelligence as a cross-section of interrelated emotional and social

competencies and skills that impact someone’s intelligent behavior. His development of the

model proceeded over a period of 17 years and is based on various items such as his experience

as a clinical psychologist, reviews of the literature, input from experienced healthcare

13

practitioners, and the development of norms and validation of the instrument across cultures,

genders, and other demographics. The model was developed from his EQi® or emotional

quotient inventory developed in 1988, a self-report measure of these behaviors which provides

an estimate of emotional-social intelligence (Bar-On, 2006). The EQi contains 133 short

sentence items with a 5-point response scale. The responses range from “seldom or not true of

me” to “often true of me.” The EQi is appropriate for adults age 17 and older and takes

approximately 40 minutes to complete (Bar-On, 2006).

Bar-On (2006) describes more than 20 predictive validity studies performed on over

22,000 people in seven countries around the world who completed the EQi. Based on these

findings, he concluded that this model is indeed able to predict various and multiple aspects of

human performance. These findings led Bar-On to suggest that his EQi be used in schools and

workplaces. Human resources personnel in organizations could utilize it in hiring, training and

succession planning; and healthcare practitioners could use it in diagnostic, remedial, and

preventative work. As Rothstein and Goffin (2006) discuss, personality measures are being

increasingly used by mangers to evaluate the suitability, or “fit” of job applicants. A survey of

recruiters in 2003 indicated that “30% of American companies used personality tests to screen

job applicants” (Rothstein & Goffin, 2006, p. 156); another survey in 2005 indicated that “more

than 40% of Fortune 100 companies reported using personality tests for assessing some level of

job applicant from front line workers to the CEO” (Rothstein & Goffin, 2006, p. 156). Rothstein

and Goffin (2006) also describe in their work that numerous meta-analytic studies on

personality-job performance repeatedly demonstrate that personality does contribute to

predicting job performance and, if used appropriately, may add value to hiring practices.

14

Research has shown a direct relationship between levels of empathy demonstrated by

nurses and positive patient outcomes (Cadman & Brewer, 2001). The EQi measures

interpersonal competencies such as empathy and social responsibility which may make it a good

admission tool for pre-requisite radiography students applying to the healthcare program. Further

research is needed to explore this.

Significance

The initial problem framing this action research study began with the decision not to use

the interview component within some competitive admission’s processes of healthcare programs

at the researcher’s organization. Further inquiry at the organization revealed that administration

had only eliminated the interview component from some allied health programs, but not all, and

when administration was questioned the response indicated this was a decision based on the

potential for legal challenges to an interview given the competitive nature of admissions in these

programs.

This led to an initial search of the literature regarding what constitutes noncognitive

assessments and how noncognitive behaviors are defined as well as a search for evidence of legal

challenges to admissions interviews in healthcare fields. Although the literature overwhelmingly

indicates the interview is a widely used noncognitive assessment (Hall, Regan-Smith, & Tivnan,

1992) few empirical studies related to the effectiveness of an interview in measuring these

behaviors were found. In addition, this review did not provide evidence of legal issues as they

relate to interviews in competitive admissions of healthcare programs.

Therefore the purpose of this study was to demonstrate how action research methodology

can be used in an organization to explore what noncognitive behaviors are necessary for

healthcare professions; why they are essential to assess at admissions; and alternate means of

15

assessment of these behaviors outside of the interview. This investigation contributes to the

body of knowledge pertaining to the methodology of action research by documenting the process

as well as adding recommendations and implications for future research related to competitive

admissions for healthcare professions.

16

CHAPTER 2

LITERATURE REVIEW

There is a responsibility in admissions processes to be fair to the public by choosing the

person with the greatest potential to be a good healthcare professional. There is also a

responsibility to the applicant to be fair in the admissions process. Albanese (1999) expounds by

saying there is an uncomfortably high rate of graduation from clinical education by people who

are not prepared to practice medicine. Cate and DeHaes (2000) elaborate by mentioning most

clinicians can remember students who graduated but should be considered unsuitable for patient

care. By researching both theoretical and empirical works, the author hoped to find an answer to

the problem of how to measure these behaviors in a valid and reliable way. Gaps were identified

in the literature regarding various forms of interviews. Gaps were also identified in the area of

emotional intelligence measurement methods. Many articles concurred that more research needs

to be performed on the measurement of noncognitive characteristics for educational admissions

processes.

This literature review addresses the issues related to the topic of predicting student

success in healthcare programs of adult education institutions, particularly the noncognitive

abilities of the students and their bearing on the clinical component of the program. The review

also discusses the factors associated with the emotional intelligence construct and comparisons

are made to employees and job performance. Research studies in the areas of: interviews in

competitive admissions as predictors or non-predictors of success, which types of medical

programs utilize interviews, how often do the interviewers receive training prior to interviewing,

17

and various types of interviews are discussed, along with the importance of noncognitive traits

and descriptions of these traits. Emotional intelligence measurement tools are discussed with a

focus on the Bar-On Emotional Quotient Inventory assessment. The final section of the

literature review concludes with results of the need for more research related to measuring

noncognitive behaviors for admission to healthcare programs in educational settings.

A focused search of the literature addressing the topics of this study was conducted.

Databases used for the study involved use of Galileo, which is Georgia’s virtual library.

Databases included EbscoHost, ProQuest, EBSCO, Health Source, Legal, Medline, and

Academic Search Complete. Key words used in various stages of the research included

competitive admissions, interviews in competitive admissions, radiography admissions,

admissions processes, interview components in competitive admissions, traditional interviews,

selection factors, admissions policies, predictors of student success, multiple mini interviews,

emotional intelligence, student success, student selection, emotional intelligence of medical

school applicants, emotional quotient inventory and academic success. Numerous abstracts were

located through PubMed and then requested through inter- library loan. The author also used

Google Scholar and contacted several people and requested copies of their dissertations and

theses. The author attempted to locate case law material regarding lawsuits related to admission

processes, but was unsuccessful. This could be an area of further research in the future.

Competitive Admissions Processes

The United States has made a shift from manufacturing to a technological society.

Because of this shift, applicants to educational institutions are finding themselves retraining and

looking for careers that are technologically driven and theoretically do not have the opportunity

to be shipped overseas. Higher education programs are reporting increased applicants from

18

nontraditional students and an increase in health care professions (Ochs & Adams, 2008).

Selection processes generally have an admissions rubric that is based on legally defensible data.

Ochs and Adams (2008) discuss the importance of the role of competitive admissions processes

due to this recognizable shift in health care careers from manufacturing. They argue that

admissions policies and procedures will need to be handled not only according to internal

policies but also higher education policies. Legal ramifications need to be considered in the

admissions processes. Areas that need to be taken into consideration are affirmative action

programs, students with disabilities, and race discrimination (Ochs & Adams, 2008).

Medical programs in educational institutions strive to produce highly skilled and

competent healthcare providers. How do we ensure that students who are admitted to healthcare

programs are able to succeed and meet the demands of a very challenging and demanding

didactic and clinical education program? With a competitive process, screening factors that can

predict student success can be identified (Weege, 2009). Bugg and Pounder (1993) also

discussed the norm of a screening phase in competitive admission processes for health science

programs. Salvatori (2001) notes the need for competitive processes is obvious because

admissions for these types of programs are typically very competitive and includes assessment of

cognitive and noncognitive abilities.

Young (1997) discussed the strain being placed on medical students when applying for

competitive resident-selection processes at medical schools in Canada. She said that because

residency positions are being reduced, it is more difficult for students to obtain placement in the

program of their choice. This can cause students to apply to their less popular program choices

in order to receive some sort of medical training. She elaborates, by saying that this choice of a

student being placed in a less desired program can have numerous negative consequences. This

19

consequence can manifest itself as lack of interest in the program or poor performance in the

program. Both can negatively impact professional relationships as well as patient care.

Scott et al. (1995) conducted a national survey of health program admissions processes.

Their research showed that 81% of respondents indicated that program faculty conduct the

admissions process while 49% indicated the admissions office of the institution conducts the

admissions process. They also demonstrated that “faculty responsibilities during the admissions

process most commonly include pre-admission counseling, review of required documents,

applicant interviews, and discussion concerning admissions decisions; the final admissions

decisions rest predominantly with the program director and the faculty” (Scott et al., p 99).

These programs reported the strengths of their admissions processes were primarily the

objectivity in the use of GPA and point scoring systems, as well as the use of interviews.

Need for Competitive Processes in Healthcare Program Admissions

Several studies (Bugg & Pounder, 1993; Espen, Wright, & Killion, 2006; Ochs & Adams,

2008; Storey, 2008; Weege, 2009) discussed the need for competitive processes, due to the fact

that there are more qualified applicants applying to medical programs than there are slots. Many

applicants are turned away for various reasons including, but not limited to: small class capacity,

inadequate clinical education sites, limited faculty, small classroom size and accreditation

policies (Weege, 2009). Espen et al. (2006) also mentioned the mandatory tracking of attrition

and completion rates as required by accreditation agencies of medical programs.

Both Storey (2008) and Pritchard (2010) agreed that screening processes are especially

important in the field of nursing due to the nursing shortage, as well as the upcoming shortage of

nursing faculty that are nearing retirement age. Pritchard elaborated that the nursing shortage is

compounded by the number of applicants to nursing programs who are not retained within the

20

program. Bugg and Pounder (1993) along with Kudlas (2006) pointed out that educational

programs in the radiologic and imaging sciences have a larger number of qualified applicants

than can be accepted into these programs. This dictates that the screening process is very

important and a crucial step in the competitive process. Weege (2009) also discussed the

increase in interest in radiation therapy programs and that the number of students, who are

accepted, in relation to the number of students who apply, is one half.

Kudlas (2006) conducted a study of radiography programmatic admissions. His results

showed that programs which utilized a competitive process have a higher retention rate than

programs that do not use some sort of basic admissions process. His results supported the need

for competitive screening processes when admitting students to radiography programs. Kudlas

(2006) argued that “a student who leaves a radiography program creates a void that cannot be

filled until the program completes another admission cycle, up to a year later” (p. 162). He also

discussed the shortage of allied health staff in clinics and hospitals. Kudlas noted a need for

understanding the connection between retention in radiography programs and providing guidance

to competitive admission committees. He believes that if a connection can be found between the

admissions processes and attrition rates of radiography programs, this feedback loop could

benefit the retention and completion of students in radiography programs.

Noncognitive or Affective Behaviors and Characteristics Defined

Affective or noncognitive characteristics are considered personal qualities or

interpersonal abilities and can often be hard to explain or assess. Definitions of noncognitive

behaviors and traits lack specificity. Cognitive behaviors, on the other hand can be measured

with definitive tests and measures. Saxon, Levine-Brown, and Boylan (2008) demonstrated that

cognitive assessment of college students measure how much knowledge a student has in a

21

particular subject area when they are tested. This information is generally seen as valid and

reliable. They mention that although practically all assessment done in U.S. colleges is cognitive,

their research shows that 25% of student performance is determined by noncognitive skills.

These authors encourage college advisors to measure noncognitive skills in order to produce a

more successful student.

Noncognitive behaviors are often related to the term “attitude” explains Cate and DeHaes

(2000). Another means of defining attitude is “system of beliefs, feelings and action intentions”.

They discuss that professional attitudes and professionalism can be used to address the domain of

interpersonal skills and attitudes in healthcare. They compare these behaviors in healthcare to

the “affective domain” in educational psychology. They also note that “education may well

address personal feelings and beliefs but assessment implies the imposing of norms, and may

thereby cross borders of the freedom of thought” (Cate & DeHaes, 2000, p. 40). However, the

authors go on to state that these noncognitive skills can be seen as behavior-in-daily practice and

should be assessed in most professions, particularly the medical field.

Noncognitive Traits and Behaviors Necessary for Educational Healthcare Programs

How can one describe the specific traits or characteristics that are necessary for

healthcare and healthcare educational programs? Are there identifiable traits that will predict the

success of a potential healthcare student? What are these identifiable traits? This review of the

literature produced many descriptions of the noncognitive personality characteristics or “soft

skills” that would be beneficial to a healthcare provider. Albanese, Snow, Skochelak, Huggett,

and Farrell (2003) state that up to 87 different traits or characteristics relevant to the practice of

medicine have been identified in literature today. They reported that their research suggests that

“there is evidence that interview ratings are predictive of subjective clinical assessments, and low

22

interview assessments are predictive of failure or withdrawal from medical school” (Albanese et

al., 2003, p. 315). Weege (2009) identified other noncognitive factors such as motivation,

decision making skills, interpersonal communication, and ability to work in a team. Multiple

authors listed empathy as being an essential skill in a strong healthcare provider (Benbasset &

Baumal, 2007; Storey, 2008; Weege, 2009). Storey mentioned compassion, emotional maturity,

and self-awareness as personal characteristics essential to the field of nursing. Salvatori (2001)

also listed maturity, empathy, and ethical integrity as contributors to good clinical performance

in the field of healthcare, and also in the area of academic outcomes. Young (1997) considered

truthfulness and honesty as fundamental values within the medical profession. She referred to

the Hippocratic Oath and pointed out that this age-old code of ethics dictates that medical

professionals will behave in a respectful manner that merits the respect of the public.

Hughes (2002) describes several specific traits that are demanded in a good doctor. She

lists honesty, integrity, fairness, conscientiousness, helpfulness, interpersonal skills, personal

welfare, and empathy. She also mentions that because there is so much emphasis placed on the

doctor to be an upstanding professional, they are more vulnerable to alcoholism, drug use, and

suicide, than other professionals.

In the healthcare field of pharmacy, Joyner, et al., (2007) listed nontraditional or

noncognitive traits of integrity, compassion, and leadership skills, among others, as essential

personality traits needed for the field of pharmacy. Albanese et al. (2003) identified with the

previous authors, that greater emphasis is being placed on assessing personal characteristics

when making selections for students in medical fields. Many different sources agree (Edwards,

Johnson, & Molidor, 1990; Joyner et al., 2007; Storey, 2008; Weege, 2009) that these areas are

hardest to evaluate. These sources did not provide information regarding data analysis to arrive

23

at these specific traits, which could prove to be a gap in the literature. The articles did not

advocate that specific personal characteristics were best at identifying a successful student, only

that noncognitive characteristics as opposed to cognitive skills were indicators that a student

would be successful in a medical career.

Emotional intelligence as a noncognitive measure. Emotional intelligence

encompasses some of those skills and behaviors that are not measured with IQ tests. The term

has become very popular when looking to measure noncognitive characteristics. Salovey and

Mayer (1990) define emotional intelligence as “the ability to monitor one’s own and others’

feelings and emotions, to discriminate among them and to use this information to guide one’s

thinking and actions” (p. 189). They write that persons who are emotionally intelligent are able

to regulate their own emotions in such a way that benefits themselves and others. Salovey and

Mayer merged emotion and intelligence as a cognitive ability and titled it “emotional

intelligence.” By controlling emotions, individuals may be able to reprioritize demands, solve

problems adaptively, motivate persistence at challenging tasks, and use their own emotions to

motivate adaptive behaviors. They argue that the emotionally intelligent person is better at

flexibility and creativity in arriving at possible solutions or alternatives to an existing problem.

Carrothers, Gregory, and Gallagher (2000) note that emotional intelligence was

introduced by Salovey and Mayer, and then later popularized by Daniel Goleman. Goleman

discusses five areas he considers to be emotional intelligence: knowledge of one’s emotions,

ability to manage ones emotions in difficult situations, motivating oneself, recognition of

emotions in others, and interpersonal skills. Brackett and Salovey (2006) add that by using

emotional intelligence, an individual is able to process emotional information which can lead to

better cognitive processes.

24

Carrothers et al. (2000) described their empirical research as showing emotional

intelligence related to educational programs which have a strong emphasis in social science and

humanities. This finding indicated an applicant’s competency in interpersonal skills and

possible fit with a medical education program. They feel that an instrument which measures

emotional intelligence will help to measure the attributes that indicate desirable personal and

interpersonal skills in these applicants.

Emotional intelligence examined in the employment context has been studied by

Kluemper (2006) who suggests that there has been an increased interest in testing the predictive

ability of emotional intelligence in the framework of employee selection and job performance.

He demonstrates the need for more evidence, but also provides research that supports the role of

measuring emotional intelligence in employee interviews and job performance. Kluemper points

out that, organizations are settings in which employees will encounter stressful situations and a

multitude of interpersonal interactions. This can also be said of the work setting of a healthcare

student or healthcare professional.

Measurement tools of emotional intelligence. Emotional intelligence has been compared

and measured in relation to job performance. O’Boyle, Humphrey, Pollack, Hawver, and Story

(2010) conducted empirical research on whether or not emotional intelligence measures predict

job performance when personality and cognitive intelligence are also included as job predictors.

Their results showed that emotional intelligence may be especially important in the service

sector as well as job areas where employees interact with customers, such as patients in

healthcare. In most all work settings, employees are required to interact with customers or

fellow employees. This research showed that this measure could also be helpful in classroom

settings due to student teamwork projects that are usually a part of classroom assignments.

25

Ultimately, there are several ways to measure emotional intelligence and O’Boyle et al. (2010)

stated that the method chosen should depend on the purpose of the project, the feasibility of

administering the test, and any other similar factors that might contribute.

The Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) was designed as a

method of measuring the various branches of emotional intelligence. According to Brackett and

Salovey (2006) the MSCEIT meets classical criteria of a standard intelligence test and is a

predictor of social outcomes. The MSCEIT measures 4 branches of emotional intelligence and

was created as a briefer version of the Multifactor Emotional Intelligence Test, or the MEIS.

Their studies showed that the relation between grades and MSCEIT was indeterminate, but the

individuals who scored high on the MSCEIT needed less cognitive effort to solve problems. The

study also revealed that higher emotional intelligence was associated with more positive social

relations.

Also, the preliminary findings of the MSCEIT across organizational settings suggest that

emotional intelligence positively contributes to job performance (Brackett & Salovey, 2006).

Their empirical studies showed that children with higher MEIS scores were rated as being less

aggressive by their peers and more social by their instructors, than students with lower scores.

Another measure of emotional intelligence which was more frequently discussed in the

literature is the Bar-On model of measuring emotional intelligence (Kluemper, 2006). This

method provides information on five composite factors that are comprised of 15 subscales. The

instrument measures emotional intelligence in five categories: intrapersonal EQ, interpersonal

EQ, adaptability, stress management, and general mood which is described as happiness and

optimism. The Bar-On Emotional Quotient Inventory (1988) is known as the EQi test. This is a

self-report inventory that measures emotional, personal, social, and survival aspects of

26

intelligence. The company asserts that this is an accurate predictor of emotional intelligence

which is also a “key predictor of life success” (Saxon et al., 2008). Table 2 is a listing of the five

scales and 15 subscales measured in the Bar-On EQi assessment.

Table 2

Bar-On Emotional Quotient Inventory (EQi) Categories

Bar-On EQi ®

Composite Scales Sub-scales

Intrapersonal

Emotional Self-Awareness

Assertiveness

Self-Regard

Self-Actualization

Independence

Interpersonal

Empathy

Interpersonal Relationship

Social Responsibility

Adaptability

Problem Solving

Reality Testing

Flexibility

Stress Management Stress Tolerance

Impulse Control

General Mood Happiness

Optimism

Critique of the emotional quotient inventory. A research study of the EQi conducted by

Derksen, Kramer, and Katzko (2002) found that when compared to a general IQ ability test, the

General Adult Mental Ability scale (GAMA) correlations between the EQi and GAMA were

very low. The GAMA is a self-administered test that measures an individual’s general

intellectual ability (Derksen et al., 2002). In a Dutch subject sample (n=873) the study indicated

that the two tests are psychometrically independent in that the EQi is measuring something other

than the GAMA. They found that not only were the correlations between the two instruments

very low, the two instruments perform differently across age. Based on their study, the authors

27

argue that emotional intelligence measured by the EQi is not equivalent to conventional

personality traits.

Mayer, Caruso, and Salovey (2000) had similar findings when comparing the MSCEIT

and the EQi. Their research showed low relationships between the two instruments with a

correlation of 0.36, which indicated they share approximately 13 percent of their variance. “The

low relationship between different EI measures raises serious questions about whether they are

all actually measuring the same construct” (Conte, 2005, p. 437).

An additional study by Grubb and McDaniel (2007) investigated the fakability of the EQi

by conducting research on a sample of 229 undergraduate students from a southeastern

university. Students were asked to respond honestly to the EQi and then asked to re-take the

assessment with fake responses. When faking their responses, respondents were able to improve

their scores on the EQi by .83 SD. The authors state this relative ease with which respondents

were able to substantially raise their scores limits the value of the EQi as an applicant screening

tool.

Conversely, across several studies (Dawda & Hart, 2000; De Weerdt & Rossi, 2012)

support the reliability and validity of the EQi and further promoted the EQi as a broad measure

of emotional intelligence. Dawda and Hart (2000) examined the EQi in a sample of 243

university students; results indicated that the EQi domain and component scales had good item

homogeneity and internal consistency. De Weert and Rossi (2012) used the Minnesota

Multiphasic Personality Inventory (MMPI), a self-report personality inventory typically used to

evaluate mental illness, as an external criterion to evaluate the EQi. Their results (n=967)

showed overall EQi scores tend to negatively correlate with the MMPI scores which indicated

that people high on emotional intelligence factors show less behavior and personality problems

28

and psychopathology (as measured by the MMPI) than people scoring low on emotional

intelligence.

One of the psychometric aims of the EQi is to predict success generally or in specific

situations, however the notion of predictive validity must grapple with both conceptualization of

the predictor instrument and the criterion behavior. One form of validation for the EQi which

some authors (Derksen et al., 2002) suggest is to correlate the EQi scores of people in

comparable positions to external indicators of success.

Assessment of Noncognitive Behaviors and Characteristics

While most authors agreed that there are specific noncognitive abilities, traits, and

characteristics which should be assessed for students entering educational programs in medicine,

most also agreed that there is great difficulty in how these characteristics are assessed. Several

authors (Edwards et al., 1990; Weege, 2009; Wright & Miederhoff, 1999) discussed the struggle

in measuring characteristics such as empathy in a non-subjective method. A particular method

that these sources felt would best determine whether or not a candidate does hold these various

traits were listed as the interview. Other areas of measurement of these traits showed to be

personal recommendation letters, personality testing, clinical observations in clinical settings,

self-report assessments. (Cate & DeHaes, 2000; Hughes, 2002; Saxon et al., 2008; Storey, 2008)

There is also discussion of the legalities of how these traits are measured as well as preservation

of a legally defensible admissions process.

Benbassat and Baumal’s (2007) research reported that there has been a shift in recent

years suggesting that more emphasis be placed on a medical school applicant’s noncognitive

traits. “Schools can concentrate on looking for candidates who have the personality

characteristics to become fine doctors, rather than concentrating on ruling out many applicants

29

who meet the academic qualifications” (Edwards et al., 1990, p. 168). Joyner et al. (2007);

Salvatori (2001); Storey (2008); and Weege (2009) agree that admissions processes for allied

health programs in secondary educational institutions should include both assessments of

cognitive and noncognitive characteristics of the applicants. Joyner et al. states there are

accreditation policies that now require noncognitive behaviors be assessed in admissions to

pharmacy programs within colleges and universities. They stated that these skills can only be

assessed through interviews. Joyner et al. found that 80% of researched schools utilized an

interview as a means to measure this area. Noonan, Sedlacek, and Veerasamy (2005) produced

research results that showed that noncognitive variables are better predictors of retention than

grades. O’Boyle et al. (2010) added that their meta-analytic procedure showed that more than

one variable predicts job performance. They mentioned that several areas should be understood

and focused on, such as emotional intelligence and cognitive variables.

The most frequently mentioned means of assessing noncognitive behaviors were

interviews (Albanese et al., 2003; Benbassat & Baumal, 2007; Cate & DeHaes, 2000; Edwards et

al., 1990; Ehrenfeld & Tabak, 2000; Goho & Blackman, 2006; Hall, et al. 1992; Joyner et al.,

2007; Kluemper, 2006; Kudlas, 2006; Salvatori, 2001; Scott et al., 1995; Stansfield & Kreiter,

2007; Storey, 2008; Weege, 2009). Interviews of prospective students can be performed as

structured, unstructured, or semi-structured. Edwards et al. (1990) reported that 99% of all U.S.

medical schools use the interview as an element of their admissions process. The interview is a

highly used selection component for higher education as well as employment. Joyner et al.

(2007) argues that within the field of pharmacy education, the interview component is more

important to admissions than the PCAT (Pharmacy College Admissions Test) and also more

important than the grade point average. Hall et al. (1992) argue that interview scores help to

30

identify applicants to medical schools who will go on to be the strongest and most competitive

performers in residency. Carrothers et al. (2000) indicated that an interview with emphasis on

criteria measuring interpersonal behaviors would produce a better student body at medical

schools. Kudlas (2006) discussed the use of interviews in a study of dental hygiene students and

how these interviews were predictive of the student’s life experiences, personal characteristics,

and community service. These traits were beneficial because they increased diversity in dental

hygiene students.

Another study performed by Eva, Rosenfeld, Reitter, and Norman (2004) showed that

because many medical schools have low rates of attrition, the admissions process is the most

important evaluation tool used by a school, but this in turn causes considerable controversy. Eva

et al. also reported that typically some form of an interview is used for this admissions process.

The Interview as a Predictor

Weege (2009) reported that panel interviews along with one-on-one interviews as well as

group interviews were used during competitive admissions to radiation therapy medical

programs. Survey results showed that the panel interview was most common. The group

interview was the least common interview tactic used for these type educational programs.

Weege, as well as Tran and Blackman (2006), discussed the superiority of a structured panel

interview as opposed to an individual unstructured interview when assessing validity and

reliability. Present findings in the Tran and Blackman (2006) study reported that “the one on one

interview format would produce significantly more accurate predictions of a candidate’s

academic potential than would the group interview format” (p. 196). Salvatori (2001) agreed that

the use of a structured interview format improves reliability and Espen et al. (2006) added that

structured interviews are more objective than unstructured. Storey (2008) also discussed

31

individual, or one on one, student interviews as the primary means of measurement in her study.

Scott et al. (1995) indicated in their research that GPA, science GPA, and the interview process

were identified as the strongest components of the entire admissions process to medical

programs.

Various ranges for different types of interviews are used for academic admissions (Goho

& Blackman, 2006). Loosely structured interviews with minimum guidelines, moderate

structured interviews with panel interviewing and question outlines, and highly structured

interviews with predefined questions and panel interviewing were discussed. Their research

showed that most medical admissions involved loosely to moderately structured selection

interviews.

Across multiple studies, interviews were reported to be the most often used admission

requirement and used in many different allied health educational programs, including

occupational therapy, physical therapy, radiography, occupational therapy and physician

assistant programs (Espen, et al. 2006; Salvatori, 2001; Storey, 2008). During their research,

Espen et al. found that more than one half of radiography programs indicated that they use

interviews as a part of their competitive admission process. Storey reported that interviews are

conducted for admission into many nursing programs and Salvatori’s research showed a large

percentage of physiotherapy programs report utilizing interviews.

In their research, Johnson and Edwards (1991) found that interviews ranked first in

importance from a list of five different selection variables when selecting candidates for medical

schools. Weege (2009) discussed that the interview is a commonly used tool for selection of

radiation therapy program students; “The interview process is a way to weed out good candidates

and determine success in the radiation therapy program” (Weege, 2009, p. 14). One of the most

32

obvious examples of the prevalent use of interviews, for healthcare education programs, was

documented in Edwards, et al. (1990) with the report that 99% of all United States medical

schools use an interview as part of their admission procedures. There is documentation of

widespread use of the interview as part of the selection process for healthcare related post-

secondary education (Goho & Blackman, 2006). What is significant across all of these studies is

the lack of real evidence indicating that noncognitive skills are actually measured in these

interviews, and that interview ratings correlate with effective performance in the profession.

A literature review table that synthesizes and compares across key empirical studies is

included as Table 3. I have selected studies to illustrate the range of data available and studies

that were most like my research study and relate to healthcare in various forms of healthcare

programs.

Table 3

Literature Review Table of Key Empirical Studies

Study Sample Methodology Findings Carrothers, R.M,

Gregory, S.W., &

Gallagher, T. J.

(2000)

147 applicants to six

year competitive

medical program

EI= not specified

Analyzed and

compared admissions

interviews of

applicants, to EI

instrument scores to

assess noncognitive

personal and

interpersonal qualities

5 dimensions of EI

indicated fair to

excellent internal

consistency;

reliability coefficients

were .66 to .95

Study showed that EI

instrument

demonstrated ability

to measure attributes

that indicate desirable

noncognitive skills in

medical students

Noonan, B.M.,

Sedlacek, W.E., &

Veerasamy, S. (2005)

263 students in health

science program at a

western community

college were

administered the NCQ

NCQ scores were

related to college

grades using Pearson

correlation and

multiple regression

Results showed

modest but

statistically significant

relationships with

community college

33

(Noncognitive

Questionnaire)

grades which was an

indicator of student

success in health

science programs

Storey, L. M.

(2008)

Participants in the

study included all 209

accepted students in

2002-2006, noted as a

sample of a larger

population of ASN

nursing students

Data was collected

from the BANER

system and entered

into an excel

spreadsheet; interview

scores were obtained

from a tool which was

developed by the

health science dept.

heads with a

maximum number

being 25 on the

interview score sheet

Student with a high

interview score is

more likely to

complete the program

than a student with a

low interview score,

but may not

necessarily have a

high nursing course

GPA

The individual

interview scores had

weak positive

correlation but no

significant linear

association with

nursing GPA,

indicating interview is

not a good single

predictor of success

Weege, M. R.

(2009)

91 program directors

at various radiation

therapy programs in

USA

Qualtrics ® link for

database and survey

Survey with open

ended questions

73% report they have

interview to measure

noncognitive

behaviors

Results concluded

program educators

value noncognitive

factors when

evaluating potential

applicants to their

program

GPA predicts

academic success but

not necessarily

clinical success

34

Multiple mini-interview. Several studies (Eva et al., 2004; Kumar, Roberts, Rothnie, du

Fresne, & Walton 2009) discussed using a MMI (multiple mini-interview) as an option for the

traditional structured interview component. Various stations are set up and applicants move

from one station to the next and are questioned by one person. Eva et al. researched this

interview type by using seven stations with various foci on areas considered critical to the

healthcare field. A few examples of the stations were communication, critical thinking, and

knowledge of the health care system. Based on a research study of 281 applicants interviewed

by the MMI technique, Lemay, Lockyer, Collin, and Brownell’s (2007) added that the MMI

offers a fairer as well as more legally defensible assessment of applicants who are applying to a

medical school. This research was in comparison to the traditional interview. Lemay et al. along

with Benbassat and Baumal (2007) claimed that interviews have strong face validity but do not

provide a standardized or defensible process for medical education applicants.

Kumar et al. (2009) conducted research on the participants’ and the interviewer

experiences and understanding of the multiple mini-interview process. Their research stemmed

from their findings that the panel interview format has disadvantages including moderate levels

of reliability, lack of predictive validity, and issues of context specificity. This led Kumar et al.

to conduct a qualitative investigation of the multiple mini-interviews in order to address some of

these shortcomings. Their belief is that qualitative rich data provides a better insight into the

understanding of their studies.

Several key themes in the perceptions of participants from the use of multiple mini-

interviews were identified (Kumar et al., 2009). Participants felt that, in multiple mini-

interviews, the group influence of one-to-one interviews is eliminated; benchmarking among

candidates is prevented; the quality of candidate-interviewer interaction is improved; and diverse

35

opinions of interviewers are captured. The themes regarding multiple assessment opportunities

in the multiple mini-interview were that candidates were provided with a second chance;

reliability of the assessment process is enhances; and multiple perspectives of a candidate are

seen. Some participants felt that a mini-interview inhibits candidate performance and limits

candidate-interviewer interaction and rapport building. Participants also voiced concerns

regarding the lack of opportunity in the multiple mini-interviews to discuss specific personal

qualities.

There was a limited amount of information regarding this particular type of interview and

there was even less information of the use of the MMI for educational and academic purposes,

thus this option was not considered for testing in this case study.

Training for panel members of interviews. Multiple authors and multiple studies

agree with the need for structured interviews and especially training for the members of the panel

who conduct the interview in the admission process (Albanese et al., 2003; Bugg & Pounder,

1993; Edwards et al., 1990; Espen et al., 2006; Ehrenfeld & Tabak, 2000; Hall et al., 1992;

Joyner et al., 2007; Kumar et al., 2009; Salvatori, 2001; Storey, 2008; Weege, 2009). There was

evidence that the use of a structured interview will improve reliability of the scoring in the

admissions process (Edwards et al., 1990; Salvatori, 2001). Salvatori (2001) found that prior

interviewing experience of the interviewer was also a means of improving interview reliability.

Tran and Blackman (2006) added that training and careful selection of interviewers are helpful

when conducting interviews.

Numerous studies (Edwards et al., 1990; Joyner et al.; 2007; Salvatori, 2001) reported

that medical school research has shown that faculty interviewers would benefit from receiving

annual training in order to assure standardization and any new panel members should receive

36

training as they are added to the committee. Their studies also indicated that forms of interviewer

training varied from program to program. Joyner et al. (2007) stated that in the field of

pharmaceutical education, “interviewer training was reported by 86% of the schools, but the

nature and extent of this training varied greatly” (p. 3). Hall et al. (1992) offered an interesting

perspective on their research “the significant relationship between interview scores and dean’s

letter ratings indicates a need to discover what qualities the interview actually measures and to

consider the methods by which interviewers are trained rather than to forsake the interview” (p.

842).

Because different types of biases are thought to be possible during an interview

component, multiple authors advocated that training for the interviewer would help to alleviate

this type of dispute (Benbassat & Baumal, 2007; Bugg & Pounder, 1993; Edwards et al., 1990;

Espen et al., 2006; Joyner et al., 2007; Salvatori 2001; Weege, 2009). Research conducted by

Bugg and Pounder (1993) found that interviewers could be biased toward attractive applicants of

either gender and Salvatori (2001) added that interviewers could be biased toward their own

demographic or educational backgrounds which could also be detrimental to the reliability of the

interview scores.

Kumar et al. (2009) recommended that targeted interviewer training in a multiple mini-

interview should be completed in order to facilitate an understanding of what is being assessed as

well as to identify areas of potential interviewer bias when interviewers are interacting with the

applicants. This approach was also deemed outside of this study.

Interview applicant’s perceptions of interviews. This review of the literature yielded

very little information regarding the interview applicant’s perception of the interview process.

Tran and Blackman (2006) examined the validity of a group interview and perceptions of

37

fairness for interviews. They compared group interview perceptions to traditional one-on-one

interview format perceptions. Seventy eight undergraduate psychology students were given a

fairness survey consisting of questions developed to assess interview selection method

favorability. They point out that there has been little empirical research on the applicant’s

perceptions of the fairness of interview processes. They also discuss the importance of this

information due to the fact that selection processes may have both ethical and legal implications

for the organization.

Participants viewed the group interview format as being significantly more unfair and

inappropriate that the one-on-one interview (Tran and Blackman; 2006). They described the

group interview as a process in which several applicants are interviewed simultaneously. This

showed that participants or applicants do have strong feelings regarding interviewing techniques

or methods. The participants listed impersonal nature, lack of face validity, and invasion of

personal privacy as reasons for the unfairness in the group interview over the one-on-one format.

Overall, Tran and Blackman stated that interviewers should use the one-on-one interview format

over the group format because their research showed that the group of applicant’s format lacked

validity when compared to the one-on-one interviews. The authors described their investigations

and their results supported their hypothesis that interviewers in a one-on-one format were

significantly better at predicting an applicant’s future academic potential.

Interviews as predictors of clinical success. This literature review revealed that grade

point average (GPA) as well as prerequisite coursework was an excellent predictor of academic

and cognitive success in medical and healthcare related educational programs (Espen et al.,

2006; Kudlas, 2006; Pritchard, 2010; Salvatori, 2001; Storey, 2008). Conversely, Salvatori

(2001) reported that “positive correlations have been found between grades and fieldwork

38

supervisors’ ratings; however, the correlations have been low and often not significant” (p. 162).

Noonan et al.’s (2005) research found a variety of problems in relying solely on GPA for

competitive admissions. One of the issues of relying so heavily on GPA was the problem of

grade inflation. Their research showed that variations in the quality of education could also be

problematic. Investigation from yet another researcher held a different perspective. Storey

(2008) who conducted a study related to structured individual interviews used in the selective

admission process reported that “but, another finding, though less significant, was new and

exciting for the author at the time: faculty predictions of success gained from interviewing the

applicant and reviewing the academic record was related to academic success” (p. 28). Storey

(2008) conducted multiple linear regression analysis and logistic regression analysis on a sample

of 209 students accepted into a nursing program at a technical college. Her results showed

interviews, when used in conjunction with cognitive assessments, do in fact predict the success

of students as well as the retention rate of students.

Healthcare professions employ noncognitive skills along with cognitive skills. These

noncognitive skills are utilized within the clinical facility in the form of patient care and

teamwork with fellow caregivers. Most noncognitive skills cannot be measured with GPA. This

literature review overwhelmingly revealed that these skills can be measured with an interview

and the interview is an excellent predictor of clinical success (Albanese et al., 2003; Benbassat &

Baumal, 2007; Edwards et al., 1990; Ehrenfeld & Tabak, 2000; Goho & Blackman, 2006; Hall,

et al., 1992; Joyner et al., 2007; Kudlas, 2006; Noonan et al., 2005; Salvatori, 2001; Stansfield &

Kreiter, 2007; Storey, 2008; Weege, 2009). Tran and Blackman (2006) point out that researchers

have shown that accurately assessing nonverbal language and behavior during an interview is a

crucial part of making correct judgments about a candidate. Ehrenfeld and Tabak (2000)

39

specifically stated that “the relationship of medical school admission interview data and medical

school performance was also evaluated, and was found to correlate with clinical performance

rather than academic achievement” (p. 102).

Interviews as “non-predictors” of success. Although most of the research reviewed

produced results that supported that interviews are predictors of success in a healthcare program,

there were a few that did not report these same findings. Even after an extensive search of the

literature, only two studies in this review, which examined the relationship between interviews

and programmatic success, reported negative results concerning overall success (Bugg &

Pounder, 1993; Benbassat & Baumal, 2007). Bugg and Pounder (1993) found gender related

biases in the admissions process, particularly the interview component, by manipulating

application profiles of hypothetical candidates for admission to a health science program. They

manipulated gender and attractiveness through a photograph which they attached to each

candidate’s application. Sixty subjects evaluated each of the applicants on certain admission

criteria. They stated, “These potential attractiveness effects suggest that students may need to

maximize their attractiveness, especially in admission interviews or at other important decision

making points in their academic and professional careers” (Bugg & Pounder, 1993, p. 237).

They also suggest that personal interviews should be deferred until initial screening of student

applicants has taken place to reduce bias based on attractiveness.

Benbassat and Baumal (2007) reported concerns that specific noncognitive skills or traits

can only be inferred in interviews. They pointed out that desired noncognitive skills can have

unclear definitions among interviewers and there can be disagreements regarding the

interpretation of the interview scores and selection information. They concluded that “the use of

noncognitive criteria is costly in terms of time and manpower, and their reliability and validity is

40

a matter of controversy” (Benbassat & Baumal, 2007, p. 516). They propose that in order to

eliminate this controversy, admission departments should be more informative in their

information regarding the strains of medical education, training, and dissatisfaction in the field.

Their opinion is that if more of this type of information is properly disseminated, then

noncognitive criteria would not need to be used for the admission process. This literature review

showed that these particular authors were the only ones who held these opinions or found these

results in their research. They also proposed that after this information was given, the final

selection could be based on academic achievements or on a lottery (Benbassat & Baumal, 2007).

They continued by writing that any applicant who is not selected for medical school based on an

interview or some other noncognitive test will feel humiliated and this could later affect that

person’s self-esteem.

Summary of Literature Review

Medical educational programs are in high demand. Educational institutions are under a

lot of pressure to fill these spaces, in these healthcare programs, with the best students.

Generally, qualified applicants outnumber the amount of accepted students into these programs

(Espen et al., 2006; Kudlas, 2006; Weege, 2009). The medical field requires many noncognitive

traits and characteristics, such as empathy, compassion, critical thinking skills, social maturity,

and self-awareness (Albanese et al., 2003; Benbasset & Baumal, 2007; Salvatori, 2001; Storey,

2008; Weege, 2009). These skills are necessary in order to relate to, and care for, patients.

These skills are sometimes referred to as “emotional intelligence” and the literature shows that

these skills are becoming more popular in medical education.

Some authors argue that emotional intelligence can be defined as non-technical

competencies, such as communication, and that more medical professionals need training in

41

these areas. Timmins (2006) mentions that emotional intelligence may look promising as a

means for teaching noncognitive abilities, however there seem to be several challenges to its

application. He discusses the need for more research in the area of emotional intelligence in

order to use it as an assessment measure and to improve student outcomes. He also indicates that

implementing emotional intelligence training without taking into account any assessment risks

can prove to be detrimental to the student and faculty as well as wasteful of resources.

Ochs and Adams (2008) stated that “interestingly, some researchers have demonstrated

that a student’s personal characteristics, such as motivation, study habits, and personal work

ethics may be strong clinical performance indicators and should be researched further” (p. 86).

What is the best way to assess and measure these characteristics in a potential candidate for an

institution of higher learning? Although the various authors who were studied in this literature

review had varying opinions on the best method of how to assess these traits, most agreed that

there needs to be further research in this area (Benbassat & Baumal, 2007; Edwards et al., 1990;

Ehrenfeld & Tabak, 2000; Espen et al., 2006; Goho & Blackman, 2006; Joyner et al., 2007;

Kudlas, 2006; Noonan et al., 2005; Ochs & Adams, 2008; Pritchard, 2010; Salvatori, 2001;

Scott et al., 1995; Stansfield & Kreiter, 2007; Storey, 2008; Tran & Blackman, 2006; Weege,

2009; Wright & Miedorhoff, 1999; Young, 1997).

The literature review generated various methods of measuring a human being’s

noncognitive characteristics. The most prevalent method in this search was shown to be the

interview. There were several methods of interviewing such as traditional interviews and

multiple mini interviews discussed in the literature. The interview was mentioned more

frequently than other measurement methods, but is this a valid tool? The literature suggested

that interviews can be subjective and possibly the reason for lawsuits regarding admissions

42

processes. The literature also proposed the use of emotional intelligence, (EI) as one measure of

medical school applicant’s noncognitive abilities. Research showed that emotional intelligence

appears to be an important set of abilities that could be expected from healthcare providers.

Several tests, or methods, of measuring EI were located in the review.

Research pertaining to grade point averages and student success is plentiful, particularly

in the healthcare field of nursing. Research suggests that interviews are a predictor of

noncognitive skills, therefore predicting the overall success of the healthcare student. This

component remains controversial. Although minimal research has been done about this outcome

based on the individual interview as part of the selection criteria; even less has been written in

the specific area of radiography, which is a healthcare area that is of particular interest to the

author. Although there is little research in this particular field, the research that has been

conducted on other healthcare professions can be applied to the field of radiography as well.

Thus, while there is considerable evidence of the prevalence of the use of interviews for

admissions processes in healthcare, and for the importance of considering non-cognitive

behaviors and traits in healthcare admissions, there is little empirical evidence linking interview

ratings with later clinical performance in the field and little evidence that the interview actually

measures the non-cognitive behaviors and traits deemed essential in this review. This action

research case study was undertaken to address these gaps.

In conclusion, there was a gap in the literature regarding the effectiveness of interviews

as opposed to the frequency of their use. In addition, there was a gap in the literature regarding

the use of alternatives to the interview particularly assessments used to measure noncognitive

behaviors and characteristics. Numerous articles mention specific noncognitive behaviors and

characteristics that they feel are essential to healthcare, but did not provide information regarding

43

how this data was generated. There were also gaps concerning documented court cases

surrounding interviews for medical admission processes. Therefore this study sought to address

some of these gaps.

44

CHAPTER 3

METHODOLOGY

This section will discuss the methodology used as the study was conducted. The

overarching methodology used for this dissertation was action research with a mixed methods

data collection approach. Action research is a systematic approach to investigation which

involves a group of people who work together to find an effective solution to some problem they

may face in their lives (Stringer, 2007). Coghlan and Brannick (2010) describe action research

as “an approach to problem solving, it is an application of the scientific method of fact-finding

and experimentation to practical problems requiring action solutions and involving the

collaboration and cooperation of the action researchers and members of the organizational

system” (p. 5), which allows involvement of others. They also describe action research as

research in action rather than about action and the members of a system being studied can

actively participate in the process.

This study employs both qualitative as well as quantitative data collection methods. The

qualitative methods include data collected from documents, notes or memos, interviews, and

surveys. Yin (2010) explains that the use of multiple sources of data collection support validity

and reliability in case study research. Because this is an action research case study, it is

grounded in a qualitative research paradigm with the purpose of gaining an understanding of a

particular research problem or issue, which according to Stringer (2007), is the fundamental

basis of action research. He does discuss that quantitative methods are not necessarily excluded

from action research, and that quantitative data can provide important information of the body of

45

knowledge that needs to be incorporated into the study. The quantitative data collection methods

employed were scores collected from various assessment tools such as interviews and clinical

evaluations.

Action Research Methodology

Action research is an established research methodology used in social and medical

sciences since the mid-twentieth century (Baskerville, 1999). Coghlan (2006) writes that

according to Shani and Pasmore (1985) the definition of action research is:

Action research may be defined as an emergent inquiry process in which applied

behavioral science knowledge is integrated with existing organizational knowledge and

applied to solve real organizational problems. It is simultaneously concerned with

bringing about change in organizations, in developing self-help competencies in

organizational members and adding to scientific knowledge. Finally, it is an evolving

process that is undertaken in a spirit of collaboration and co-inquiry. (pp. 294-295)

Coghlan and Casey (2001) add that action research is an approach to research which aims at both

taking action and creating knowledge or theory about that action. They describe the central

tenets of action research as being

Change experiments on real problems in social systems

Involves iterative cycles of identify, plan, act, evaluate

Intended change involves re-education of current patterns of thinking

Challenges the status quo from a participative perspective

Contributes to basic knowledge in social science and to action in life

As an employee of the organization where this study took place, I participated in insider

action research which provided a purpose for producing practical knowledge and was

46

exemplified in daily actions by my organization and myself. As an insider action researcher and

in contrast to traditional research approaches, the researcher is not neutral, but an active

intervener constructing and helping things happen (Coghlan & Brannick, 2010). In this type of

research, the role of the researcher along with the participants is collaborative and synergistic in

that the members of the organization being studied are actively engaged in the quest for

information which will guide their future actions (Baskerville, 1999) and is sometimes called

participatory action research. Action research is beneficial to an organization because it promotes

problem solving with members of that organization who have an interest in the issue being

studied.

According to Stringer (2007), action research methodology provides a means for which

people in health and human services may increase the effectiveness of the work in which they are

engaged. An action research team was formed within the organization in order to collaboratively

conduct research regarding the best measures for noncognitive behaviors of students preparing to

enter the field of healthcare. The initial team meetings consisted of qualitative inquiry of the

action research team member’s perceptions and understanding of the problem. Team members

worked together to determine what noncognitive measures or skills best serve the healthcare

industry and provide the best patient care.

The action research project milestones were as follows: entry, planning, diagnosis, data

analysis, intervention, and evaluation. Table 4 was created early in the project and reflects a

summary of the action research collaborative plan and outlines the stages that were to be taken

that are the intervention-the work of the action research team. Stringer (2007) describes a basic

action research routine as following the steps of “look, think, act” which encompasses gathering

47

relevant information, exploring/analyzing data, planning, implementing, and evaluating. This

table reflects this action research process.

Table 4

Action Research Process Stages

Stage Action Research Team

Process Anticipated Outcomes

Entry & Contracting Confirm team members

Set up meetings

Accept/revise goals

Construct the issue

Agreement on nature and scope of

the AR team project

Initial steps defined and

established

Make meaning and explore the

context

Planning

action/intervention

Planning based on shared

inquiry

Collaboratively determine best

intervention to measure

noncognitive skills

Taking

action/intervention

Plans are implemented and

interventions are made

collaboratively

Successful data collected in order

to evaluate the results of the

intervention

Evaluating

action/intervention

Outcomes are jointly

examined

Answer to the research question or

more planning for further

intervention(s)

During action research meetings, decisions should be made on the basis of consensus,

rather than on the basis of majority vote (Stringer, 2007). During the “taking action” stage, an

instrument was used to assess a student’s emotional intelligence which is described as skills of

empathy, self-awareness, motivation, and self-control, skills are recognized as part of effective

clinical practices (Cadman & Brewer, 2001). The Emotional Quotient-Inventory (EQ-i ®) was

administered to a purposeful sample of two different groups of healthcare students (Group A

and Group B) in training for positions in the healthcare industry. Students in Group A were

interviewed prior to their acceptance into the program and students in Group B were not

interviewed prior to acceptance and were selected for participation strictly based upon their

prior cognitive assessments. This EQi assessment was decided upon based on feedback from

48

the action research team during the “planning action” stage. The purpose of this decision was

to determine if this particular assessment could be a possible future alternative to the interview

process. In response to the information described in Figure 1, an operational framework, Figure

2, was developed to illustrate how we would test the conceptual framework. The conceptual

framework indicates that according to Gardner (2011), human beings are born with multiple

intelligences, which includes an inter/intra personal intelligence. Based on the question posited

“How can we best assess these intelligences?” figure two was created. Working with the action

research team, we decided to answer that question by comparing scores across two time periods

and three assessments with the understanding that these might indicate, collectively, an

individual’s potential for success in their chosen health care profession. Figure two depicts this

research plan.

Figure 2. Operational framework of the methodology.

49

Coghlan and Brannick (2010) describe the method of action research as consisting of four

basic steps which ultimately become a cycle of research. The cyclical steps are constructing,

planning action, taking action, and lastly evaluating the action taken. Action research team

members should seek to understand the initial problem, assess the social and political

surroundings, identify a possible intervention to solve the problem, next carrying out the

intervention, and evaluate the success of the intervention.

Participants

Schwarz (2002) defines a working group or team as a set of people with specific

interdependent roles but who are collectively responsible for making a decision. The action

research team consisted of various healthcare professionals representing numerous fields of

healthcare. This particular group of people was carefully selected because they bring an

appropriate mix of knowledge and skills to successfully complete this action research project

(Schwarz, 2002) as well as numerous years in healthcare. The action research team included

various directors and a coordinator of several different healthcare programs as noted in Table 5.

Table 5

Action Research Team Member Profile

Team

Member

Role in

Organization

Area of

Expertise Profile

Years of

Healthcare

Experience

DCR Program Director Radiography Caucasian female 22

TML Program Director Surgical

Technology Caucasian female

43

TMP Program Director Physical Therapy African American

female

28

TMR Clinical

Coordinator Nursing Caucasian female

18

TMT Program Director Occupational

Therapy Caucasian female

19

50

The expertise and professional experience that each of the selected team members have

will help to answer the initial research question regarding the skills and competencies assessed

during an interview in programmatic admissions processes. Stringer (2007) states that action

research requires a purposeful sampling when identifying stake holding groups. This process is a

conscious selection of people on the basis of a particular set of attributes. The major attribute is

the extent to which the individual is affected by the issue of interest, and since each chosen team

member is responsible for instruction of healthcare students, they are clearly affected.

I began to create an action research team by informally discussing with several co-

workers that I was working on an action research dissertation and asked if they would be

interested in meeting with me regarding this “problem.” They all immediately agreed and were

excited about the research. Prior to the first scheduled team meeting, a consent form was

created and approved by the university (See Appendix A) for each team member and a signed

copy was given to each member and an additional copy was obtained and is kept in a

confidential location.

Position of the Researcher in Action Research

As a medical professional, I know the importance of quality screening processes for

competitive admissions into health profession programs. With appropriate screening procedures,

the candidate who best fits the role of becoming a competent as well as empathetic caregiver can

be chosen. This process can help to ensure the success of the student. Healthcare programs have

a responsibility to the general public to graduate students who will find success in their chosen

field. These students not only need to have the necessary cognitive abilities to perform the job,

but they also need to have noncognitive traits relative to the profession. What are these traits and

51

what is the best method to measure them prior to a student beginning a healthcare program?

These questions led me to the initial engagement with my client system.

As Stringer (2007) explains, in action research, the position of the researcher is described

as a participant who actively engages in monitoring and directing the process of inquiry. My

role as participant of the action research team helped to shape the process and the outcome of the

study. Because I am employed at the technical college where the research was conducted, and

am considered an insider researcher, a critical skill for me was to be able to combine advocacy

with inquiry by presenting my own opinions and viewpoints while being open to the possibility

that they may not be correct.

Being an insider to the organization gave me the opportunity to engage others who I

knew had the same interest in my research questions. I was able to ask certain individuals to be

part of an action research team in order to explore the subject of the importance of noncognitive

behaviors in the medical profession. These individuals, like me, had been employed in the

medical for many years, some longer than I had. I knew from prior conversations with these

people, that they each felt that the best healthcare professional had a balance of intellect and soft

skills necessary to perform their job.

Coghlan and Brannick (2010) explain that reflection is a fundamental part of the process

of action research and enables you to integrate experiences and information with your

understanding of them which will in turn improve judgments and inform new actions. After each

team meeting, I documented my reflections of the meeting in a journal. These journal notes

became a mechanism for me to develop reflective skills. It is important to capture these

experiences and events as close to when they actually happened because as time progresses, the

perception of the event can change (Coghlan & Brannick, 2010).

52

Qualitative Methods

Qualitative as well as quantitative methods were both used to answer the three research

questions guiding the study. Action research is fundamentally grounded in a qualitative research

paradigm with the goal of gaining clarity around a problem or issue. Action research begins with

the identification of a problem and investigates the way participants describe their actual

experience of that issue and how it affect them. Alternatively, quantitative methods are then used

to analyze the relationship between a carefully defined set of variables which have been

determined throughout the qualitative method. (Stringer, 2007)

Qualitative methods were used to answer the first and third research questions: (1a) What

are the noncognitive behaviors and characteristics identified in the literature for healthcare

professions? (1b) What are the noncognitive behaviors assessed during the interview component

of a competitive healthcare program? and (3) In what ways does an action research team process

lead to individual, group, and organizational learning and change? Quantitative methods were

used to answer the second research question, “Is there a relationship between EQi scores and

ratings on interviews, mid-term assessments, or clinical evaluations?” This section will outline

the qualitative methods used here.

This action research study utilized qualitative research methods to provide thick

description of what an action research team learned about assessing noncognitive behaviors.

Data collected included transcripts of the action research team meetings, researcher memos, and

critical incident interviews. One of the strengths of qualitative data is the richness of data

collected. Such data provides “thick descriptions” that are vivid, and nested in a real context

which has a strong impact on the reader (Miles & Huberman, 1994). This research study

encompassed a time period of almost three years. Miles and Huberman also assert that a benefit

53

of qualitative data is the fact that such data are usually collected over a sustained period which

makes them powerful for studying any process and enable us to assess causality as it actually

plays out in the particular setting over time.

Multiple sources of data were used for this study. This triangulation of data provided

validity and trustworthiness. Merriam (2009) suggests that triangulation of data remains a

principal strategy to ensure validity and reliability in a study because the researcher is comparing

and cross-checking data collected from multiple sources and by several means. Each team

meeting and interview was recorded for transcription. After each transcript was merged together

with researcher memos, codes were assigned. Merriam (2009) defines coding as assigning some

sort of shorthand designation to different aspects of data which will allow the researcher to

retrieve specific pieces of the data.

Data Collection

The qualitative data obtained for this study and answers research question number three,

was action research team meeting transcripts, research memos and finally, critical incident

interviews of the action research team at the conclusion of the project. These are all elements

that comprise findings from qualitative research (Merriam, 2009). According to Ruona (2005)

the primary goal with qualitative research is to obtain participants’ perceptions and meanings

through and in the participant’s own words and then by analyzing those words you interpret the

participant’s meanings.

Document analysis. One initial source of data was the interview ratings of applicants for

Group B. The action research team analyzed these interview documents and later made the

collaborative decision to incorporate the information from these documents into the study. The

interview process conducted on the Group B applicants has been approved by administration at

54

the college. The interview panel is made up of healthcare providers of which the students in

Group B are applying, a clinical psychologist, and other healthcare professionals. The

interview questions were approved by administration and each question was asked of every

candidate. The questions measured the student’s ability to problem solve, think critically, and

demonstrate flexibility. The scores of each panelist were tallied and all are averaged together

to provide the overall interview score. This ensured inter-rater reliability. Ratings were

reported verbatim, including comments, and were analyzed by the research team collectively.

Action research team meeting transcripts. During the course of the research,

numerous action research team meetings were conducted. Members were sent an email asking

them to participate in and online poll by selecting the day(s) and time(s) that would be best for

the member. Stringer (2007) states that meeting conditions such as time and place should

always maximize the opportunities people have to attend. All meetings were held in a common

place which was easily accessible by the majority of team members. For each scheduled

meeting, I created an outlined agenda for each member along with copies of any relevant

materials to be discussed in the meeting. Stringer (2007) adds that when organizing a meeting,

careful planning and preparation must take place so members will not feel “distracted” by

poorly articulated activities, inadequate materials, or conditions that may be irritating. During

the final interviews, one team member had this to say regarding the organization of the

meetings:

“I can’t believe it, but the fact that it didn’t seem like a long odious process to support

the organization that was quite apparent that was put into it by the researcher, Ms.

Collins, you could tell that you organized, you prepared everything to make sure that

our team meetings ran smoothly, that we had all the information that we needed to

55

have, that you sent out emails to keep us informed, that we were prepared so that when

we became together as a committee we were ready to do the work that needs to be

done. It was a very smooth and organized process.”

With the verbal permission of each team member, all action research team meetings were

audio recorded. “The use of a tape recorder has the advantage of allowing the researcher to

record accounts that are both detailed and accurate” (Stringer, 2007, p. 73). The recorded

meetings were then transcribed by the researcher and professionally.

Researcher memos. During each action research team meeting, pertinent notes were

made by the researcher during the meeting and later added to researcher memos. After each

meeting, the researcher reflected back on the meeting and the participants and wrote memos

related to the meeting and what was said, or what took place. Memos capture the researcher’s

thoughts and include key concepts being discussed and then linked to the case study (Miles &

Huberman, 1994). These memos provided the opportunity to write introspective notes

regarding each meeting and the participants reactions, mine included, to the experience of each

meeting and discussion.

Critical incident interviews. At the conclusion of the research study, each action

research team member was interviewed using the critical incident technique. According to

Elllinger and Watkins (1998), the critical incident technique is a systematic method of

collecting observed incidents which are reported from memory. This technique was originally

developed by John C. Flanagan in 1954 who described an incident as observable human

activity which permits the observer to make inferences or predictions about the person or the

act. He adds that in order for the incident to be seen as critical, it must make a significant

contribution whether positive or negative, to the activity (Flanagan, 1954).

56

After critical incident interview questions were developed and approved by the major

professor, interviews were scheduled with each team member and five critical incident

interview questions were asked of each participant (See Appendix B). Each participant gave

verbal permission for the researcher to record each interview and interviews were

professionally transcribed. Tape recording the interview ensures that everything said is

preserved for analysis and also provides the researcher an opportunity to listen for ways to

improve his or her questioning methods (Merriam, 2009).

The interview questions were all open ended questions to facilitate a richer response by

the participant. Open questions provide broad parameters for an interviewee to formulate

answers in their own words and can generate detailed descriptions of the topic (Roulston,

2010). Merriam (2009) agrees by adding that good interview questions are open ended and

yield stories about the phenomenon and the more detailed and descriptive the data, the better

the interview. Occasionally, a participant was asked a follow-up or probing question for

clarification of the response.

Data Analysis

Miles and Huberman (1994) describe data analysis as a process of reducing the data,

displaying the data, and finally drawing conclusions and verifications. The purpose of data

analysis is to search for important patterns, themes, or meanings related to what the researcher

has heard or seen (Ruona, 2005). Analyzing qualitative data is an interactive process which

allows the researcher to produce trustworthy findings (Merriam, 2009).

Unlike experimental designs in which validity and reliability are accounted for before

the investigation, rigor in a qualitative research derives from the researcher’s presence,

the nature of the interaction between researcher and participants, the triangulation of

57

data, the interpretation of perceptions, and rich, thick description. (Merriam, 2009, pp.

165-167)

Data was analyzed using the constant comparative process which involves comparing

one set of data with another to determine similarities and differences; data is grouped together

on similarities and given a name which then becomes a category (Merriam, 2009). The goal is

to identify specific patterns or themes that are emerging from the data. Ruona (2005) cautions

the researcher to remember that “the process of data analysis begins with your ability to

recognize the codable moment-that is, to sense the themes emerging from the data” (p. 237)

when immersing themselves in the data. Codes were not pre-assigned to the data, as Miles and

Huberman (1994) and Merriam (2009) point out, this approach allows the researcher to be open

to what the data will reveal rather than attempting to force-fit the data into preexisting codes.

All three sets of data, meeting transcripts, researcher memos, and team interviews were

transcribed, coded for themes and categories and finally merged together. In order to conduct

the data analysis, I utilized a methodology for analyzing qualitative data using Microsoft Word

which was created by Dr. Wendy E.A. Ruona (2005). This process includes four steps: data

preparation, familiarization, coding and generating meaning.

Data preparation involved transcribing of all meetings, memos, and interviews with

minor editing of the documents in Word. Names and identifiable information were removed

from the transcripts and pseudonyms were assigned to each team member. Each transcribed

file was saved as a separate Word document and electronically stored in a folder titled “Data

Analysis.” Each document was then saved under a different file name so the researcher would

still maintain a “clean” non-manipulated copy. Each document of text was converted to a table

58

with six columns. The six columns were manipulated to specific sizes and a header row was

inserted titled: Code, ID, RQ#, Turn #, Data, and Notes.

Familiarization of the data involved an immersion into the data at a deeper level, which

means listening to tapes, reading and rereading the transcripts and adding notes and memos to

your data. The purpose of this stage is to “actively engage with the data, begin your analysis,

and record your insights about what you ‘see’ in the data” (Ruona, 2005, p. 256). For each

row, information was added based on the row heading. The row titled ID was filled in based on

the pseudonym assigned to the participant. The row titled Q# was originally created by Ruona

to function as the question number being asked in an interview of the participant, however,

because I had all data merged together, I altered this column to RQ# which allowed me to add

the number of the research question that was being answered in that row of information. The

next column, Data, which is considered the most important, was read and then divided into

meaningful segments of data by adding new rows to the table and moving that data into the

new row. The last column, Notes, was filled in with any related comments or insights attained

while reading the data.

The third step in the process was the coding of the data. Each transcript was read and

data was reduced and categorized into general categories then codes or themes were created

based on what emerged from the data. Data driven codes are codes that are created inductively

and emerge from the data you have collected and based on what you as the researcher find

interesting and significant (Ruona, 2005). I assigned a five digit code to each category,

beginning with the number 10000 and then created categories and subcategories based on this

number. This numbering allowed for the capability to sort the numbers later in the process.

59

During the last step, generating meaning, the aim is to engage in the work of exploring

how the themes which have emerged are connected to one another and any ideas you as a

researcher may have, including information obtained from the literature and prior research

(Ruona, 2005). At this stage, all transcripts have been coded and each document was merged

into a master document and then sorted using Word according to the coding system. Once all

coded information from the documents were merged into a master document, I continued to

read over the master document and reflect on the established codes which led to code revisions

and refinement. During this phase I also highlighted key themes or statements and continued to

add comments in the “Notes” column. An example of the finished coded, merged, and sorted

data is provided in Table 6.

60

Table 6

Sample of Master File of Coded Data

Code ID RQ# DATA Notes

11100 DCR 3 I think what we would have to sit down I

guess and determine is look at these and

say, “Okay, yeah. These are measuring the

same thing.” We as medical educators

recognize these … whatever traits needs…”

11100 TMR 3 Healthcare is a high stress environment

11100 TML 3 It’s not an exact science. (healthcare requires

thinking out of the box)

11100 TMR 3 We had some that responded real well, but

it was just, when it got to that point to

actually, you know, put your arm around

somebody and pat them on the back and say

… it’s going to be okay; we’re going to do

the very best we can for your husband or

whatever. Yes, none of that.

Evidence of what a

needed noncognitive

behavior in an acted out

healthcare scenario

11100 TMP 3 That’s her personality.

11100 TMP 3 I mean I could see both perspectives, inter

and intra in the sense of inter you are

concerned about another person’s well-

being, versus intra you have to know what

you need to do, to make sure that the patient

is safe.

11100 TMR 3 They’d have to have an awareness of the

patient’s safety.

Trustworthiness of the Data

Trustworthiness consists of steps taken to ensure the validity and reliability of the data

given the nature of the data, whether qualitative, survey, etc. This study included multiple data

sources which provide triangulation of data. One of the most well-known strategies to provide

internal validity of a qualitative study is through triangulation of multiple sources of data or the

use of multiple methods (Merriam, 2009). In addition to triangulation of data, a strategy of

61

member checks was used to ensure credibility of data collected. Trustworthiness of the data

will be ensured by using constant-comparative coding (Lincoln & Guba, 1985) of all transcripts

and interviews, by finding and maintaining an audit trail of all data collection and analysis

procedures (Miles & Huberman, 1994).

According to Stringer (2007) a study has credibility and trustworthiness when multiple

sources of information are incorporated. There was information obtained from the interview

component, clinical evaluations, mid-semester advisements, and from the EQi assessment. The

data of the work of the action research team includes researcher notes, transcripts from the

audio recorded meetings, and interviews with the action research team members. There was an

audit trail of the process which Stringer describes as confirmability which also adds to the

trustworthiness and veracity of the study.

Member checks. A common strategy for ensuring credibility of a study is member

checks or respondent validation. Member checks are described as solicitation of feedback on

emergent findings from the people who were interviewed (Merriam, 2009). This will help to

deter misrepresentation of the participants and give them an opportunity to validate their

comments. In this particular study, after all qualitative data was reduced and coded, then

written up in the findings, all action research team member comments used in this dissertation

were sent via email to all action research team members. Team members were then asked to

confirm all comments that may have been used from the team member and send comments and

feedback via email.

Team members were given a week in order to email any comments or revisions to the

researcher. At the conclusion of the week, three of four members had replied with feedback.

One respondent replied with several minor changes to grammar and spelling in her particular

62

quotes and stated that everything else looked “good to her.” A second respondent replied by

saying “I didn’t see anything that needed to be changed, you’ve done a great job pulling

everything together with specific quotes.”

Finally, a third member responded with “I tried to limit my comments to the quotes, but

I could not help but to add a few extra.” Stringer (2007) states that one of the values of

member checking is the opportunity for the participant to not only review the research but it

also provides an opportunity for them to extend information related to their experience. This

particular team member made numerous revisions to her quotes and elaborated in some areas to

the information she had originally stated. She deleted areas where she had been redundant and

made adjustments to certain words she used which then made the quote more meaningful. All

participant’s responses have been revised by the researcher in the final document which

Roulston (2010) describes is a result of the strategy of member checking.

Information garnered from various sources, meetings, memos, and interviews, helped

form the data which was used to evaluate the work of the action research team as a means to

explore the research questions posed in this study. Action research participants were various

healthcare professionals with an abundant amount of years spent in their respective fields.

Constant comparative coding (Ruona, 2005) was used to code and analyze the data for themes

which help to contribute to work of the study.

Quantitative Methods

A quantitative research design was used in the action research team’s chosen intervention

option. A quantitative design best examines the data related to the intervention choice and also

answer’s the research question, “is there a relationship between EQi scores and ratings on

interviews, mid-term assessments, or clinical evaluations.” As Creswell (2003) indicates, this

63

type of design method compares outcomes from two or more groups in the form of numbers to

determine the effects of some sort of treatment. Hopkins (2000) says the aim of a quantitative

study is to determine the relationship between two variables and quantify the relationship. The

relationship between the variables can be measured using effect statistics such as correlations.

The analysis of the various assessments used in this action research study used Spearman’s rank

correlation coefficient.

Sample

The initial target group (Group A) was the students at the technical college who are

currently enrolled in a particular healthcare program. Traditionally, the program has had an

average of 40 students applying each year. The mean grade point average (GPA) of the accepted

students has been 3.8-3.9. Each group of students would have completed certain college level

pre-requisite courses, taken a standardized test measuring cognitive test taking skills and

participated in an interview. Results from these three components have been averaged together

and the top 16-24 students have been accepted. The number accepted has also been dependent

on the current job market in this field.

After several meetings with the action research team, it was collaboratively decided that

another group of students (Group B) in a different healthcare program would also be invited to

participate. This program traditionally accepts 30 students and the GPA for the student admitted

is a minimum of 3.5. This group of students has also completed certain pre-requisite courses

prior to acceptance. This group of students does not have to participate in an interview as part of

their competitive admissions process; however, they are required to participate in an interview

during one of their programmatic courses. The EQi was statistically compared to interviews,

mid-term assessments, and clinical evaluations in both programs.

64

Instrumentation

This study utilized a validated measure of noncognitive abilities, titled the EQi. The EQi

was created by Dr. Reuven Bar-On who is a pioneer in the field of emotional intelligence.

According to Bar-On (2006), the EQi was the first measure of its kind to be published by a

psychological test publisher, the first measure to be peer-reviewed, and the most widely used

measure of emotional intelligence to date. According to his website, www.reuvenBar-On.org,

the US Air Force study concluded that millions of dollars would be saved by applying the EQi to

their training program. This significantly reduced mismatches and selected people who were the

best fit for the training course (Bar-On, 2010).

The EQi is scientifically validated based on more than 20 years of research worldwide

(Bar-On, 2010). Reuven Bar-On (1988) developed the Bar-On theory of emotional and social

intelligence which describes emotional intelligence as a cross-section of interrelated emotional

and social competencies and skills that impact someone’s intelligent behavior. His development

of the model proceeded over a period of 17 years and is based on various items such as his

experience as a clinical psychologist, reviews of the literature, input from experienced healthcare

practitioners, and the development of norms and validation of the instrument across cultures,

genders and other demographics. The model was developed from his EQi® or emotional

quotient inventory developed in 1988; a self-report measure of these behaviors which provides

an estimate of emotional-social intelligence (Bar-On, 2006). The EQi contains 133 short

sentence items with a 5-point response scale. The responses range from “seldom or not true of

me” to “often true of me.” The EQi is appropriate for adults age 17 and older and takes

approximately 40 minutes to complete (Bar-On, 2006).

65

This assessment is described as one of the leading approaches to assessing emotional and

social functioning. The applicants scores are computer generated; raw scores are automatically

tabulated and converted into standard scores based on a mean of 100 and standard deviation of

15. The higher the scores, the more positive the prediction for higher emotional intelligence

and the lower scores predict possible emotional, social, and behavior problems. An agreement

of findings reveals that this assessment model is consistent, stable, and reliable as well as

possesses good construct validity (Bar-On, 2006). Since construct reliability had already been

determined, the use of this instrument in this study was appropriate. Interpretation of the scores

by the action research team followed the interpretation manual developed by Bar-On.

Reliability and validity of the EQi. Reliability of the EQi is focused on two aspects of

reliability, internal consistency and test-retest reliability. Internal consistency was measured

using a statistic called the Cronbach’s alpha. The Cronbach’s alpha values for the EQi were

calculated for several different samples for the EQi and fall between .60 and .95 which indicate

the internal consistency values of the EQi are very satisfactory. This finding indicates the items

contributing to the scoring of each factor correlate highly together to measure the factor. The

test-retest correlations for the EQi are high and indicate that after a one month period, the test-

retest correlation was about .85, and the correlation was about .75 at four months. This

indicates good consistency in the findings from one administration to the next. This correlation

also indicates that the EQi provides consistent results and is sensitive to changes in emotional

functioning. (Bar-On, 1997)

According to the Bar-On Assessment user manual (Bar-On, 1997) the EQi has been

validated in many ways. Some examples of completed validity studies on the EQi which are

relevant to this research include correlations between the EQi and various personality

66

measures, comparisons between successful and unsuccessful groups in terms of their EQi

scores, comparisons between EQi scores and job performance and job satisfaction, and lastly

comparisons between EQi scores and what was theoretically expected for particular groups.

After the decision was made to use the EQi, I contacted the assessment company to

purchase the assessment. This information led to numerous forms that needed to be completed

and submitted to the company in order to receive approval to administer the EQi. The company

also required a letter from the Dean of the college where the assessment was administered, as

well as a letter from the major professor, before they would give approval to purchase the

assessment and administer. The company required proof that I was a doctoral student

conducting research. The University of Georgia required a watermarked copy of the assessment

before approval of the IRB application. I created a verbal script to be approved by the UGA IRB

and was used prior to administering the EQi to each group of students (See Appendix C).

Piloting the Assessment Instrument

After receiving approval from the EQi testing company to purchase and administer the

EQi assessments, the team was asked if they would be willing to take the EQi assessment and

later discuss the results and the perceptions of the assessment. They each agreed to take the

assessment for the purpose of the study. An email was later sent to the action research team

members, which gave them the necessary information including website address, login

information, and password for each member to take the assessment.

Results for the team were sent to the researcher as the test administrator through the

testing website. The researcher then contacted the company, purchased the results, and the

results were then sent to the researcher via the company website. Each person’s assessment

results can be opened and downloaded. This had to be done within 5 days of the purchase of the

67

assessment because after 5 days, the results were no longer available. Later, during our next

team meeting, we discussed our results, the assessment itself, and then collectively made the

decision to move forward with asking each group of students to participate.

After working with the team member involved with Group B, a date was established for

me to visit the classroom and explain my research to the students as well as invite them to take

the EQi assessment. After an introduction to the class, I explained the research and passed out

the IRB approved consent forms (See Appendix D). All students in Group B were willing to

take the assessment; however, there was one student absent and plans were made to have that

student take the assessment at a later date, which the student did. I carried out the same process

with the students in Group A the following day and again, all students were willing to take the

assessment.

Participation was voluntary and this was expressed to the students. Students in both

groups were informed of the research and asked if they would be willing to participate. It was

explained to the student that their participation in no way affected their role as a student in their

program. Students were also told that the results of the EQi were only used as a research tool

for the college’s healthcare programs and future admissions processes.

An informed consent form was developed with the guidance and consultation of my

major professor and input from the action research team. Students who agreed to participate

were given the detailed consent form approved by the University of Georgia’s Institutional

Review Board. The technical college where the research was conducted does not have an

institutional review board, so this was not an additional requirement.

68

Additional Data: Interview and Clinical Evaluation Ratings

Students interview score sheets and mid-semester evaluations from Group A were

collected and de-identified and identifiers were assigned to each sheet. A member of the action

research team had access to the assessments for Group B and after collecting these, de-identified

all personal information from the student’s information in Group B.

I created a spreadsheet and entered all scores from each group’s interviews, clinical

evaluations, mid-semester advisements, and EQi assessment results. Table 7 represents a

summary of the details of each assessment as it relates to the number of students, number of

interview panel members or clinical instructors as well as the details of each assessment. This

table was created for use when calculating the assessment scores in SPSS. The summary was

also used in the final action research team meeting.

69

Table 7

Summary of Assessment Tools and Participants

Summary of Assessment Tools and Participants

Group A Group B

18 students total (n)

EQi assessment

All 18 students took

5 subscale categories

Scale of 50-150

Interviews

Conducted by 5 panel members

10 categories (A-J)

Scale of 1-10

Mid-term advisement

Conducted by 2 program faculty

10 categories (1-10)

Scale of 1-4 (Unacceptable to

Exceeds Expectations)

22 students total (n)

EQi assessment

All 22 students took

5 subscale categories

Scale of 50-150

Interviews

Conducted by 5 panel members

8 categories (questions 1-8)

Scale of 1-5 or 1-4 or 1-3

depending on the question

Clinical Evaluations

Conducted by multiple clinical

instructors at clinical facilities

14 categories

Scale of 0-3.5 (0, 1.0, 1.5, 2.0,

2.5, 3.0, 3.5)

Because parts of some of the evaluations had missing values, due to the fact that the rater

had not provided a score for that particular section, the missing values were addressed by using

mean estimation and the mean of the scores present was added to the missing data areas. Two

new variables for each data set were created: a composite mean of the interview ratings across all

raters, and a composite mean of the clinical ratings across all raters. Group A mid-term

advisement specifically, had a lack of variance in the scores, therefore in order to create a better

fit with the EQi assessment, only averaged means from items five through nine across all raters

were used to create the composite clinical score since these items aligned most closely with the

EQi dimensions.

70

After the spreadsheet with all scores entered was finalized, the numbers were then

transferred into the SPSS system and statistical correlations conducted. Correlational analyses

correlating EQi total and sub-scores with the composite clinical and the composite interview

ratings as well as the two composite means with each other were made. Spearman’s Rho

correlation coefficient, a nonparametric statistical test was used to test for significance since this

is a smaller sample. The decision was made to use .10 for significance as it would be more

appropriate than using .05 because of the small sample size.

Group A had numerical data from programmatic interviews which were used in the

admissions process, mid-semester advisement, and the EQi self-assessment. The mid-semester

advisement was indicative of two semester’s worth of advisement; however, most all students

scored the same which was a “3” for “meets expectations.” This sameness, or lack of variance,

accounted for the decision to use the averaged means from items five through nine as discussed

above. Clinical skills and behaviors are one area which is measured in this particular assessment

for the Group A students. Didactic or classroom skills and behaviors are also assessed with this

assessment, but for this study these ratings were not used in order to have the assessment tool

better align with the EQi as well as the clinical evaluation tool used for Group B students.

Group B had numerical data from programmatic interviews which were conducted on the

students after being admitted into the program, and were conducted during a leadership course.

There was also data from the EQi and clinical evaluations which were conducted on the students

at the clinical facilities where the student was placed for their clinical assignment. Group A did

not have this type of assessment.

71

Limitations of the Study

One limitation of conducting action research within your own organization is that the

intervention’s ultimate organizational impact is often dictated by forces outside the control of the

researcher. As Coghlan and Brannick (2010) point out, any form of research within an

organization will have its political dynamics, however, conducting research in and on your own

organization is particularly political. Political forces can weaken research actions and block

planned change, which can hinder the project as a whole. The action researcher needs to gain

access and use data within the organization, which can be politically unsettling for

administration. Action research requires certain questions to be raised and judgments applied to

particular issues which can in turn have severe political implications (Coghlan & Brannick,

2010).

An insider action researcher must learn to tactfully “juggle” the role of researcher and

employee. Action research is intended to contribute to knowledge and develop learning in an

organization, but not at the expense of the position of the researcher in the organization.

Action researchers need to be prepared to work the political system, which involves

balancing the organization’s formal justification of what it wants in the project with their

own tacit personal justification for political activity. Throughout the project they will

have to maintain their credibility as an effective driver of change and as an astute

political player. (Coghlan & Shani, 2005, pp. 537-538)

Herr and Anderson (2005) add to the politics of action research that often times these

researchers are often ill prepared for the resistance, sometimes in the form of indifference, to

their efforts. They may encounter an institutional culture that places value on individual efforts

and conformity which do not describe the collaborative nature of action research. Because

72

action research is initiated by a practical problem within the organization, light must be shed on

the situation in order to answer the question to the problem and form an intervention of action.

This can cause apprehension within the organization which could lead to the inability to carry

out the research project and can be very limiting to the action researcher.

The question of legitimacy is another limitation of action research. “Although action

research is gaining increased support in the professional community, it has yet to be accepted by

many academic researchers as a legitimate form of inquiry” (Stringer, 2007, p. 191). The action

researcher may face negative comments or criticism due to the fact that some do not view this

type of research as scientific. Action research does not follow a traditional scientific method or

protocol, but instead is more of a collaborative exploration which is viewed by some as a

limitation of this type of research.

In this study, all of these concerns surfaced, the researcher and team worked together to

collaboratively explore the options related to the research and then made a collective decision

regarding which intervention option would be in the best interest of the college and it’s students.

By making this particular intervention decision the team showed an awareness of the role that

administration played in the politics of the project.

A limitation of the quantitative methods of the study was the small sample size of 40

students. Due to the high cost of the EQi being paid by the researcher, there was a limitation

regarding how many assessments could be furnished by the researcher. Future studies may seek

other funding sources to allow a larger group of students in the two program areas studied or in

various other healthcare programs to be assessed. Another limitation was that research was only

conducted in one institution rather than across multiple institutions or universities. Programs

outside of the ones housed in one institution could have had program faculty managed by

73

different administrators who may have held to different policies and procedures than found in the

institution where the study was conducted. This would allow for any regional or institutional

biases to be addressed.

Due to the small sample size and missing values for some raters for some of the students,

it was necessary to replace missing values with means in order to do the correlation analyses

included here. In addition, a computation of the mean of means was done. This data reduction of

the original multi-rater, multivariate scoring of the interviews and clinical performance may have

reduced the variance in the original data, creating fewer significant relationships. We addressed

this by using a more generous significance standard [p. =.1].

.

74

CHAPTER 4

CONTEXT AND CASE

The purpose of this action research case study was to assess the need for, and the possible

effectiveness of a noncognitive measurement tool in competitive admissions processes of

healthcare programs at the research site. Healthcare programs at this technical college have

required admissions processes; however there are a limited number of programs that are allowed

by administration to utilize an interview as part of their admissions process. These programs use

the interview to assess noncognitive skills (soft skills) of the student prior to admission to the

healthcare program. These program directors, clinical coordinators, and healthcare faculty have

expressed an interest and a desire to measure noncognitive skills or behaviors of students who

apply to their program. Another important element of the purpose of this research was the

utilization of a team of people who are healthcare professionals and educators in the college, and

have made the decision to work together to explore and identify alternative methods of

noncognitive measurement. Yin (2009) states that as a research method, the case study is used to

contribute to our knowledge of group phenomena, such as the use of an action research team.

This chapter presents the case studied and includes how the researcher and team became

initially involved in the action research project. A description of the action research team

members is also discussed. This chapter will serve to tell the “story” of the dissertation

including the course of events leading up to the intervention and analysis, which includes the

perspective of the researcher. Yin (2009) describes case study research as including the richness

of the phenomenon and extensiveness of the real-life context of the case study investigator.

75

Description of the Context

The setting for this action research project is a technical college in the southeastern

portion of the United States where I am employed. There are numerous healthcare programs

within this technical college and many of the students taking prerequisite coursework are

planning to apply to one of the healthcare programs. The college successfully prepares

individuals for employment and assists them in pursuing their educational, career and personal

goals through a variety of learning opportunities that include associate degree, diploma, and

certificate programs, as well as non-credit and public service offerings.

This technical college is committed to student-centered learning; acknowledged for its

dedication to excellence and its quest for continued improvement; recognized for its proactive

approach to quickly changing workforce development and technological trends; and responsive

to the diverse needs of its students, businesses and communities of the region it serves.

In my role at this institution, I am responsible for the admissions process of pre-requisite

students into one of the healthcare programs. The process, right now, is a competitive process

which involves the competitive average of the GPA of certain pre-requisite courses, a

standardized test, and the interview component. When I began this project, administration at the

college had not eliminated the interview component from my admissions process, but had

removed it from other healthcare programs, for liability reasons. This caused frustration from

other directors because they were not being allowed to utilize an interview or any tool to measure

noncognitive (soft) skills or characteristics of their applicants.

An action research team was formed at this technical college to work together to discuss

alternative ways to measure and assess these noncognitive behaviors and characteristics. The

76

action research team consisted of educators and healthcare professionals from various healthcare

fields including the students from each group who became the research participants.

I felt that I would have great support from these people, because they are fellow

program directors or coordinators and have expressed interest in my subject, particularly

because they conduct admissions processes for their own healthcare programs. Three of the

team members explained that they did not utilize a component that measures noncognitive

abilities, but both have expressed that they felt their program would greatly benefit from

measuring these behaviors as a part of their admissions process. A fourth member of the action

research team did utilize an interview in the admissions process and felt very strongly that these

skills need to be measured.

During the initial team meetings, one team member announced that they were going to

retire, which would leave no “voice” from this particular program or profession at the college.

Because this team member was a part of the program that had been told they could not use

interviews, I felt it would be important to include someone from this area, however, the decision

to fill this void would need to be decided collaboratively by the team and change triggers

discussion in an action research team (Coghlan & Brannick, 2010).

During discussion with the leaving team member, there was mention that the person who

would fill this role at the college might be a possibility as a new action research team member.

However, there were concerns that if this person was asked to be on the team, would they have

time to devote to the process with the new job duties they would have as a new program director.

This was added to the next team meeting agenda so the team could discuss their thoughts on how

we should proceed with the loss of this member. Stringer (2007) discusses that it is important

77

that members of an action research team include people whose lives are directly affected and

who have a vested interest in the research questions.

According to Stringer (2007) one technique for extending the people to be included in an

action research team is a process called “snowballing” which is asking the participants

themselves who they think would be a good candidate for the void. This technique was used

during the third team meeting, and the team was very forth coming in their opinion that our team

did indeed need representation from this particular profession. When the team was asked if they

felt this action research process could move forward with the representation from the four

programs that we had at the time, one team member stated,

“I have one question before I answer that, in your global plan for your research, and as I

understand it, part of the goal is to give back to the college; this profession is an integral

part of the college, so if your goal is to generate data and statistics on basically a product

that will benefit the college, then I would say that we do need to include this profession.

However, if the focus is more so just medical programs in general, then we have diversity

in our representation here already.”

Another team member stated, “It would be easy enough at this point prior to data collection to

fill this void, so I think we should invite another person from this department.”

Schwarz (2002) indicates that, additional contracting of a new team member can occur at

any time during the action research process. Contracting can be seen as a microcosm of the

larger facilitation of the team and gives team members an opportunity to make the choice of

whether or not the team wants to work with a new member. As an insider researcher, my role

was to facilitate and collaborate with the team regarding an additional team member.

78

During a meeting with the coordinator from this particular healthcare program, and after

explaining the research questions and the purpose of the action research component, she stated,

“I think this is a fantastic idea! I have always thought that the program was in dire need

of some personality type measurement tool to evaluate these students before they get into

the program on grades alone. We have had so many problems with some of these

students who have no ability to work with others or demonstrate any level of care or

compassion with their patients. It is very frustrating! I would be happy to do anything

you need in order to prove to administration that our program needs to measure these

behaviors as a part of the admissions process!! Count me in!”

This excitement to engage with the team and provide input helps to answer the research question

“How can an action research team process be used to explore and identify alternatives for

admissions to a medical program?”

This new team member became one of the most active members of the action research

team. The team member completed certain tasks without even being asked by the researcher and

became an integral part of the team in a very short time. This team member has taken charge and

has offered to help in any way needed, including proposing that her current students be a part of

the research. Just as Stringer (2007) points out, a benefit to action research is the enablement of

participants to play a role in significant levels of active involvement. Action research

encourages people to perform significant tasks as they relate to the research problem. There is

support for people as they learn to act for themselves, just as this team member has done.

During several team meetings, there was discussion from the team to possibly invite the

health science counselor to be a part of the team, or to attend one of the meetings. As the

facilitator, I wanted to allow the team to collaboratively decide if they felt this addition would be

79

helpful for the team. Action research is inquiry that is done with team members of an

organization and not “to” team members (Herr & Anderson, 2005). None of the team members

were voicing strong opinions that this person should be added or invited to the team meetings,

therefore, this person was not asked to be part of the team.

Development of the Action Research Process

My client system was the organization where I am currently employed as the director of

one of many healthcare programs at the institution. Coghlan and Brannick (2010) state that

undertaking action research in and on one’s own organization becomes a useful defining

construct regarding action in learning. I began the process with a contracting interview with the

Dean regarding my upcoming action research project. The Dean is also my supervisor and

supervises all healthcare programs at the college.

After meeting with the Dean and explaining the concept of action research, I was given

approval to be an insider researcher and conduct the research in my organization. The Dean also

became the sponsor for this project. Doing action research in your own organization is

opportunistic because the researcher can select an issue for research which is occurring anyway,

regardless of whether or not one’s inquiry takes place (Coghlan & Brannick, 2010). The Dean

seemed very excited about the prospect of researching noncognitive behaviors. When asked if

she thought noncognitive behaviors are an important area to assess prior to admission into a

medical or healthcare program, she stated:

“I would say that 80-90% of the students that are admitted into the programs are

academically prepared or have the cognitive skills to be successful academically, and

when we go back and look at our anecdotal non research data or just facts, most of the

students that are not successful are not successful because of academic reasons but they

80

are not successful because of these noncognitive reasons, such as personality disorder, or

attendance which is some type of issue that is not cognitive, factors outside of school,

such as dealing with family problems and things like that. So yes, I do think that how

students deal with stress and whether or not they are good rule followers and things like

that would be very good to assess prior to admissions to the program.”

The Dean was also asked about the interview as a means of measuring noncognitive

behaviors of the applicant at the institution and administrations view of the interview for these

programs. She had this to say:

“…..if the interview is not done properly it can become a huge liability. So we have, I

believe, two programs that continue to interview here, we have as close to a safe process

as we can by having a number of people on the interview on the panel and all of those

people have to talk to every applicant and we have a set of questions that I feel like have

kind of gotten a stamp of approval, umm, I don’t think that it’s perfect, I don’t think that

it is ever going to be perfect, I don’t think that it is ever going to be totally objective. And

there is always going to be a little bit of a possibility that you know, a student could sue

us because they felt like we didn’t like them in the interview. So I think that expanding

that process is scary to some of the administrators.”

She also elaborated by saying:

“I think if you talk to the faculty that there would be a lot of ‘oh yeah, if you can solve

this problem, you will be a hero’ but from the administrative level, higher than me, I

think that there is some resistance to implementing an interview and then I think that

there is an awareness for some programs that not every instrument is appropriate or

would work; for example, there were over 300 qualified applications for one program. It

81

would have been almost impossible to interview all of those. But if the interview came

down to the top 100, or if there were two assessments that would take that group from

300 to 100 like a stepping stone type of assessment, then I think that could be

incorporated. But I think that in most cases one or two types of assessment tools would

be appropriate and then have program directors select one or use both or whatever kind of

combination that they came up with. So in terms of assessing those noncognitive areas, I

personally am not in favor of: it’s either the interview or nothing. If there was something

other than an interview and an interview that would give two additional tools to use and

some people might use both, some might just use one.”

I began the initial data collection strategy, by developing a very short informal survey

and then asked people from different programs of healthcare to respond. I let the participants

know that this was completely voluntary and not a requirement. The Dean wanted me to

emphasize that this would be voluntary participation to anyone that participated in the research,

and that participation was not a part of their job description and participation was not required.

I worked with the Dean to create the questions, and she approved them prior to sending

an email to the participants. We discussed that the information collected in the survey would be

the start of clarifying the problem by questioning the perceptions of what noncognitive skills

really are to different healthcare providers. This preliminary data added to the initial research

questions regarding what specific noncognitive skills or behaviors are suited for healthcare

workers. The Dean wanted to be very involved in the initial stages of this action research project.

I recognized this was very helpful; however, I had reservations regarding the degree of her

influence on this participatory action research project and what impact this might have on the

82

participants. I was concerned about my role in managing expectations of both the team and of

my Dean should they conflict.

Prior to sending out the initial survey, the Dean announced my acceptance into the

doctoral program and my future research to the faculty during a mandatory faculty meeting. She

explained what my research would encompass and made it clear that no one would be required to

participate. I felt a slight sense of defeat before I had the chance to inquire about anyone’s

interest in collaborating with me on this problem. Fortunately, no one I asked declined to work

with me. I feel this could largely be attributed to the fact that they were very interested in a

solution to my research questions.

Preliminary Findings

For the initial survey, and based on feedback from the Dean, I chose ten participants from

the college who were program directors, clinical coordinators, full-time faculty, and adjunct

faculty from six different healthcare fields. The areas of healthcare included various fields such

as medical assisting, radiography, licensed practical nursing, associate degree nursing, physical

therapy assistant, occupational therapy assistant, and surgical technology.

There was a return rate of seven out of ten, and of those seven people, six indicated that

they had eleven years or more in the healthcare field. When asked “what are some important

noncognitive traits of a good medical professional,” there were over 20 different traits listed by

the seven participants. Based on their experience in healthcare, they identified these areas as

valid noncognitive behaviors required of a good medical professional: empathy,

conscientiousness, observation skills, communication, flexibility, adaptability, critical thinking,

problem solving, professionalism, interpersonal skills, time management, honesty, ability to

follow directions, responsibility, organizational skills, attendance, multitask, teamwork, patience,

83

understanding, social interaction, compassion, active listener, empathy, passion for work, self-

discipline, and positive attitude. Within these responses, communication, adaptability, critical

thinking, problem solving, professionalism, teamwork, and compassion were listed by more than

one participant. Survey question three asked “which trait do you consider the most important”,

compassion was listed the most times, followed by communication. Ten out of ten participants

felt that noncognitive traits should be measured in admission processes to medical programs.

The participants were asked to provide their opinion of a means of evaluating these

noncognitive skills. These responses consisted of: interviews, situational judgment tests, letters

of recommendation, conditional reasoning test, verifiable biodata measures, observation of

applicant in medical setting, consult with a health care professional who has worked with the

applicant previously, Purdue Pegboard test, on-site essay, and battery of written and verbal

questions. Six out of seven participants listed the interview, and two out of six participants listed

letters of recommendation as a means.

Ten out of ten participants felt any future negative clinical or classroom behaviors could

be alleviated when the measurement of noncognitive skills are a part of the admissions process;

when participants were asked if they felt that the use of a tool to measure these noncognitive

abilities has an effect on student success to complete the program, six of seven participants said

yes, and one participant stated no. The last question of the survey was “do you feel you would

benefit from this research?” Ten out of ten stated yes, one being “most definitely!” Overall, the

respondent’s perception of the importance of noncognitive behaviors in the medical field was

overwhelming.

84

Compassion and communication were listed as the most important traits, which can be

described as an individual who not only cares about their patient but does a good job interacting

with the patient and other healthcare providers as well. One participant explained it this way,

“…working with a medical professional that clearly cares about you, can make a huge

difference in the quality of their experience, it stands to reason that if a healthcare worker

or student is in a trauma situation, it would benefit everyone involved if they could

communicate with other providers and the patient.”

The survey confirmed that medical educators at the organization did feel these behaviors

needed to somehow be a part of a competitive admissions process and people other than me as

the researcher felt they would benefit from more research in this area. Although most of the

participants felt that future negative behaviors could be alleviated when a measurement of these

skills is a part of admissions, a few also pointed out this would not completely alleviate negative

behaviors.

The Action Research Team Members

My action research began with the targeted literature review in order to answer the initial

research question (1a) “What are the noncognitive behaviors and characteristics identified in the

literature for healthcare providers? (1b) What are the noncognitive behaviors assessed during the

interview component of a competitive healthcare program?” The preliminary interview and

responses of the Dean also answer these research questions. The Dean has served in a role

responsible for various healthcare programs and their admissions processes for over seven years.

In response to the third research question “In what ways does an action research team process

lead to individual, group, and organizational learning and change?” an action research team

within my own organization was formed.

85

The Dean requested a meeting with me concerning who I would ask to be on the team.

She had some very specific guidelines regarding who she would allow me to ask. For example,

when I mentioned one person, because this person had an affiliation with a particular program

which the Dean had mentioned would be a good program to use in the study, she was adamant

that I would not ask this person to be a part of the team. This struck me as odd due to the fact that

she had previously stated that she would like to see results as they relate to this program. Based

on this feedback, I did not contact the individual. I did contact the people she suggested I

contact. Herr and Anderson (2005) note that action research takes place in settings that reflect a

society with an unequal distribution of resources and power. It became apparent to me that my

role as the insider action researcher would not be without influence from others who directly

supervise me in my role at the institution and this could have a bearing on how much change I

would actually be able to generate.

After being given these suggestions for possible team members from the Dean, I met and

spoke with one person, who was also one of the initial people I asked to complete the informal

survey regarding noncognitive behaviors in the admissions process of medical programs. After a

brief explanation of how her input would be much appreciated and greatly benefit the research,

she was extremely agreeable and was excited that she was asked to be part of the team. Team

meeting reflection memos reveal that this member, although excited, later expressed concern that

she might not be “smart enough” to be on the team. This was something that, as an action

researcher, I needed to be cognizant of and make sure that she did not feel this way during the

process.

“The team members seemed very excited and interested in my research. I was most

surprised by the reaction that I received from…… program director, because I really

86

don’t know her that well, so I didn’t know what to expect. She initially shared a concern

that she may not be ‘smart’ enough to be on my team because she only has a bachelor’s

degree, and I found that surprising. I assured her that was not the case and the fact that

she has been in the field for over 30 years makes her more than competent for this

journey.”

The following week I was conversing via email with the new director of another

healthcare program on an unrelated topic and during the course of the conversation we began

discussing her first incoming class of 12 students. It was at this point that I was able to ask her

what she had decided would be her competitive admissions process for those students. Her reply

to me was this:

“We’ve already sent acceptance letters for the 12 this year, but admission was only based

on GPA, HOBET scores and 20 hours of clinical observation. We’d like to do something

more next year to weed out those who seem to be great students but don’t appear to know

much about what our field even is, or who may not have the best “people skills.” We

asked about interviews, but were told that the vice president will likely not approve any

other program to use interviews due to the potential for lawsuits, so we are trying to

figure out something else that might be helpful. If you come across anything while doing

your dissertation research, we’re open to any suggestions!”

I felt this was a wonderful opening for me to ask her if she would be interested in being on my

action research team. After several emails, she stated that she would be more than happy to

brainstorm with others regarding this topic, because she too, would like some component to

measure the soft skills of students who are applying to her program and the occupational therapy

87

profession as a whole. These two additions provided me with an action research team that

comprised five people, including myself.

Throughout the course of identifying team members and beginning the first team

meeting, one of the initial team members retired early which left an opening in the team. After

discussion with the team, the collaborative decision was made to identify someone else from that

same program. With the help of the team members, another member was identified and when

asked if she would be interested in being on the action research team, she was more than happy

to be a part of the research and action learning. Researcher notes reflect this:

Since the time of the 3rd

meeting, researcher met with the program coordinator and

discussed the possibility of the program coordinator being on the AR team in place of the

program director who recently retired. The past meetings were discussed with the

program coordinator and she was also given copies of the handouts, etc. from the past

meetings. The program coordinator was very excited and even very enthusiastic

regarding the research questions and problem, and agreed to be a part of the AR team.

Over the course of almost three years, these five team members, including me worked

together to collaboratively discuss what noncognitive behaviors are important in healthcare,

alternatives to an interview as a measurement tool, and decided on an intervention option to the

interview as a measurement. The team carried out the chosen intervention option and later

analyzed the data that had been collected from the intervention, all of which is outlined in

Chapter 5 of this dissertation. Ultimately the team members worked well together and it was

expressed to me numerous times throughout the course of the project their excitement regarding

the process and the decision to look at the problem of how to best measure noncognitive

88

behaviors. In an early team meeting, one team member opened by stating “I am so excited that

you are tackling this problem because it really is a problem and for us as well!”

The Dynamics of the Team

Coghlan and Brannick (2010) describe action research as being collaborative in that the

members of an organization actively participate in the research of that organization. The

members are part of a team and not objects of the study. In order to understand the dynamics of

the group, it is useful to examine the development of the group. Tuckman (1965) created a

group development model consisting of five stages with recurring themes that focus on tasks and

relationships of the group. Tuckman’s model explains that as a team matures and develops

ability, relationships establish and the team becomes self-managing.

In Tuckman’s model, the first of five stages is the forming stage. During this stage, the

group is just coming together and is often characterized by shyness, uncertainty, and hesitancy.

During this forming stage in the action research process, one team member stated in the first

meeting that she had concerns about being on the team because she might not “measure up” to

the rest of the team members. Ephross and Vassil (2006) discuss that when teams are developing

at the interpersonal level, a common behavior at this stage is hesitant participation because the

member may feel that other members are more competent or more powerful. Team notes show

this:

The team members seemed very excited and interested in the research. I was most

surprised by the reaction that I received from one team member in particular. She shared

a concern that she may not be “smart” enough to be on the action research team. I

assured her that was not the case and the fact that she has been in the field for many years

which makes her more than competent for this journey.

89

This team member actually demonstrated the emotional intelligence that is being discussed by

the team.

In the forming stage of group development, individuals are not clear on what their role is

in the process; the mission isn’t owned by the team; members are unfamiliar with each other; and

there is no trust yet. Also, in the forming stage, people are not committed to the team, which was

evident by the loss of a team member in the early stages of the action research process. Team

members were quiet during the initial meeting as compared to subsequent meetings. Team notes

reflect that as the goal and mission of action research was discussed and the opportunity to ask

questions was given, most team members did not ask questions.

The second stage of Tuckman’s model of group development is referred to as the

storming stage. Storming involves affective behavioral responses to requirements of the tasks

and to interpersonal conflicts. The second stage is also characterized by challenges to authority

and by isolation of certain individuals from the group activities (Weber & Karman, 1991).

During this phase of group development there appears to be a period of competing for position,

authority, and influence among the members which was evident in the action researcher’s team

meeting reflection notes.

“On one occasion, due to scheduling conflicts, the action research team meeting had to be

held in the classroom of one of the team members and not the classroom of the

researcher. This seemed to cause a shift in team dynamics as the team member who

offered the space was more authoritative in her role as a member and seemed to dominate

the conversation of the meeting. Other team members were quiet and spoke less during

this team meeting.”

90

Certain group behaviors during a meeting can reduce or increase the effectiveness of the

meeting. Schwarz (2007) states that a skilled facilitator in a group setting needs to observe the

groups interactions for signs of ineffective behavior and diagnose by inferring the possible cause

of the behavior.

As mentioned, another characteristic of the storming phase involves team members

expressing differences of ideas, opinions, and feelings. During the storming phase of this

research, the team was initially divided on the importance of the interview. One team member

contended “how can you interview someone for 10 minutes and know they are going to be a

great doctor or nurse?” The same team member stated that she felt an interview sets the college

up for a liability and another team member argued with her by saying that she did not think it

was a liability and she has used the interview at multiple colleges and has been told by an

attorney that if an interview went to court, the judge would rule in favor of the expertise of the

faculty and their professional background.

Within the action research team, it is apparent that there are definite personality

differences that have a bearing on how the team operates. Reflective memoes of the researcher

from the early team meetings state:

“The irony of an action research team working together to determine a best measure of

how to evaluate medical student’s noncognitive behaviors is that the action researcher has

to play the role of facilitator within the group as they exercise their noncognitive

behaviors in what are sometimes negative ways. I am responsible for facilitating a group

that consists of some with polar opposite differences in their noncognitive behaviors,

which is not always easy!”

91

Stringer (2007) discusses the importance of the action researcher negotiating diverse

opinions even though this can be difficult when strong and determined people try to impose their

own perspectives on the group. As an action researcher, I sometimes struggled with the

collaborative nature of this process and defining my role as researcher, facilitator, and leader. Of

the five team members, there were three team members who have shown that they are easy

going, and basically work well together by adding relevant opinions and information as needed.

The remaining two team members have very different personalities and these are the two that as

the action researcher, I had to be most mindful of during the process to ensure all views were

honored. Another example of this struggle can be found in this researcher memo:

The dynamic of the group during today’s meeting was the same; one team member was

the quietest member and I felt I needed to be sure and include her in the meeting and the

conversations. She has a very agreeable spirit and seems to quietly conform to the group.

Two other team members who are more outspoken have more input. I know I will need

to monitor this in the future, and still be a facilitator/participant rather than a

leader/dictator of the group. This can be hard at times.

According to Ephross and Vassil (2005), group climates that favor expressive participation, such

as in action research, may be comfortable for gregarious members and somewhat threatening for

more quiet members.

The third stage of Tuckman’s model is the norming stage and contains the basic theme of

cooperation obtained through the communication of shared values, opinions, and information.

Appreciation and trust builds and collaborative efforts by group members promote synergistic

outcomes while unity develops (Weber & Karman, 1991). During this phase, feedback

concerning the research was high and the team gained commitment from all members on

92

direction and goals. The introduction of a new team member was during this phase, and added to

the collaborative effort that had taken shape in the action research team. Team members were

excited about the collaborative decision made related to alternative ways to assess noncognitive

behaviors in the admissions process, and the confidence of the team was high. Comments from

researcher memo reflected my excitement:

“I explained to the team the great extent to which the newest member had become

involved which included attending every meeting since being asked to serve on our

action research team and I also recognized just how well she fit in and how well the

group was communicating and functioning.”

The fourth stage of Tuckman’s model is the performing stage. In the performing stage,

goal oriented activities are carried out and individuals begin to feel motivated to collaborate for

achievement of task which becomes evident through member interdependence of relationships

and contributions of resources (Weber & Karman, 1991). This is an effective stage in the group

process because the members have openness and support for one another and work to meet the

needs of each other. Schwarz (2007) adds that when group members feel the experience

contributes to their own personal growth and well-being, they can function more effectively as a

group or a team. Comments from team members during this stage show this:

“I think everybody getting together and getting that information to you so that it could get

analyzed was a big part of meeting your deadline. I think that probably was, for me a

great experience of a huge teamwork effort because you had not only this team member

and this team member helping and you had my interviews and your interviews, and

clinical evaluations and all that. That’s a lot of information to gather. I think everybody

worked real well to meet your deadline on that.”

93

The final stage of Tuckman’s model is the adjourning stage which is characterized by

completion and disengagement, both from the tasks and the group members. This stage involved

critical incident interviews of the team members and their reflections on the process. An

example of a summary thought one team member shared was:

“This research project allowed me to look at the interview process as a method for

assessing non-cognitive skills and to say if this is still a good way of doing it, is there a

better way, is there a room for improvement which I say there always is.”

Action Research Team Intervention Option

Over the course of the team meetings, the group took several collaborative actions

regarding the path of the action research and began to answer the initial research question

regarding the definition of behaviors assessed during an interview. The team began by

collectively defining, discussing, and agreeing on the research problem. From initial team

agenda notes, one team member stated “I am excited that you are tackling this problem because

it really is a problem for us as well.” Another team member stated,

“I am going to have to just put it out there; I am a little biased toward the interview, but I

will say that if there is no interview there has to be a way to assess the soft skills because

these skills are measured in an interview. The student has got to be able to build a

rapport with the patient and not just have cognitive skills.”

The team unanimously agreed that noncognitive skills or behaviors have got to be a part of the

medical programs’ admissions process in order for the programs to be successful. However, the

team did not unanimously agree on how these behaviors should be measured. At the time, the

team was divided regarding the use of an interview as a valid means for measuring the

noncognitive behaviors.

94

In preparation for the second meeting, two team members took action to reach out and

contact their fellow medical program directors across the country. This action resulted in one

member receiving information, rubrics and score sheets related to a group interview, which she

emailed to me for the second meeting and a second member was given the actual interview

questions and rubric she uses for interviews for her program. Stringer (2007) describes the role

of participants in action research as being engaged in processes of inquiry that provide them with

possible solutions to problems. This was evident by the actions taken by the two team members

to reach out to fellow educators for information.

The results of the team member reaching out to another program director were very

positive. The interview questions that were sent to the team member were later used to create the

interview questions for the students in Group B and later became part of the data to be collected

for the research project.

After much discussion and several meetings, the group took action and developed several

intervention options which are outlined in Table 8. These options were developed by the action

research team and included information from the literature review as well as from team members

regarding their various medical program admission dates and timelines.

95

Table 8

Team Intervention Options

Intervention Options

Option 1 We all implement the EQi with current students

We conduct a statistical analysis of comparison of scores on the EQi with admissions ratings.

Decide on an EQi tool (EI from Carrothers or EQi from Bar-On)

Decide on 2-3 medical programs to conduct the research

Each program would give its current medical students the EI test

The results would be compared with the current admissions ratings those same students

received prior to being accepted into the program

Option 2 Implement interview approach (A)

With current admitted students, we compare ratings on interview with other admissions ratings.

Decide on an interview approach

Decide on 2-3 medical programs

Interview data would be compared to other admissions ratings

Implement interview approach (B)

With potential applicants, we compare ratings on interview with other programmatic ratings.

Decide on an interview approach

Decide on 2-3 medical programs

Team would create interview questions/rubric

96

Each program would conduct interview as part of admissions process (may or

may not “count” towards the admissions process)

The results would be compared to other admissions data and programmatic data

of the admitted students (ie. program evals)

Option 3 Each uses competing options

With students from separate programs, use information from EQi and from interview data.

Decide on 2-3 programs

Administer EQi to currently admitted students in one program

Conduct interview with another program

Compare results of EQi and interviews from the programs with admissions data

and programmatic data.

During subsequent meetings the team explored the three intervention options and decided

on the second option, part B, which would be to implement an interview approach and compare

interview ratings on different programs to other admissions or programmatic data for these

programs. Unfortunately, what I failed to realize as an action researcher, was that I had allowed

my team to experience “mission drift.” What I did not notice was that my team had drifted away

from answering the original questions of the research study, in particular, the second research

question “What are alternative ways to assess noncognitive skills?” By the team choosing the

second option, there would be no alternative to the interview with which to compare.

Stringer (2007) describes action research as seeking consensus through meaningful

research. He also writes that the action researcher has the task of developing a context where

group members with divergent interpretations can formulate a decision that is a joint

97

construction and makes sense to all the members. As the action researcher, it would be my duty

to work collaboratively with the group to come to a consensus without being perceived as

manipulating the team. The team needed to still feel as though this was a group decision.

After speaking with my major professor, I met with my team and explained what had

happened. The team exercised a high level of emotional intelligence when listening to my

explanation of how we had drifted from our original research goals. The team members were

very open to the dialogue and willing to take ownership regarding the possibility of a new

option. After some discussion, the team decided on an alternative option in order to get back on

track with the initial research questions.

In response to the question, “what are alternatives to the interview,” the team decided that

option number one would be a better choice and more in alignment with the purpose of the

research team. There was also discussion regarding the significant research that had been

conducted on the Bar-On EQi assessment. The decision was made by the team to administer the

EQi in two programs, the current students in Group A as well as the current students in Group B.

The team decided for Group A, to compare the EQi results to their interview data and mid-

semester advisement data, both of which the program had already collected and had on file. This

included the 18 students who were currently in the program.

The team also decided to compare the EQi results from Group B to their clinical

evaluations as well as interview data from their leadership class, both of which were

forthcoming. There were 22 students in Group B who were asked to participate. In total, we

would have an “N” of 40 students for whom we have interview, mid-semester clinical, and

questionnaire data. Data from the two other programs was not available, so we focused on the

two that had the data we need at this time. The team also made the decision to assign a code to

98

each student so when they took the EQi, it would be anonymous, but we would still able to

compare to the other information that would be obtained.

The action research team worked together to generate initial information regarding

various methods for the measurement of noncognitive skills. Collaboratively, the team

questioned why administration would only allow certain healthcare programs to use an

interview. Of course, this was an area of concern for me due to the fact that if team members

became upset about this decision, I wondered if they would decide to approach administration,

which could negatively affect the work of the team. As Herr and Anderson (2005) point out,

action research projects are usually initiated because of some practical problem in a local setting,

however, local problems can be parts of larger problems and broader social forces that impact

how the local settings are constituted. The fact that the differences across allied health programs

were not known to program administrators was indicative of a larger organizational problem of a

lack of cross-disciplinary communication. This study addressed this by bringing together

individuals from these different programs.

The team had numerous discussions regarding the interview as a measurement tool and

members of the team started to question their own thoughts that an interview was not a valid

measurement tool. Team members also had conversations surrounding the question of whether

or not an assessment tool such as the emotional quotient inventory, EQi, could effectively

measure the emotional intelligence of a prerequisite healthcare student.

The action research team worked together for almost three years. The team maintained

representation from the same initial healthcare fields, however there was a change in the team

member who represents the program of students in Group B. After meeting with the action

research team, and after the addition of a new representative from this program, the focus of the

99

research included not only information from students in Group A, but also from the students in

Group B. This was the collaborative decision and work of the action research team.

The timeline for the action research team, occurred parallel to the timeline for the

intervention decision, and was based on the timeline for the actual intervention. As Stringer

(2007) points out, the first fundamental steps of this process are look, think, and act. The first

phase of “look” describes when the action research team will define and discuss the problem

which will be investigated. The second phase which is “think” will be the time period in which

the action research team will analyze and interpret the situation in order to develop a better

understanding. Lastly, the “act” phase will be formulations of solutions to the problem. A

timeline of the action research team was created and is shown below in Table 9.

100

Table 9

Action Research Process Timeline

Timeline of Action Research Implementation Plan Project Phase Start Date End Date

Entry

Begin formal meetings

with team after

informal discussions to

invite participants to

join the Action

Research team

November, 2011 December, 2011

Planning

Meetings and planning

with AR team

IRB’s created and

amended as needed

AR team decision of

intervention (EQi)

Data from other

assessments collected

Team creates

crosswalks

January, 2012 April, 2012

Diagnosis

EQi taken by team

members

April, 2012 May, 2012

Data Analysis

EQi results discussed

by team

May, 2012 June, 2012

Intervention

EQi administered to

students in Group A

EQi administered to

students in Group B

Interviews conducted

on Group B students

currently enrolled in

program

June, 2012 August, 2012

Evaluation

Team evaluates and

discusses intervention

during final meetings

Critical incident

interviews with team

members regarding the

process

September, 2012 February, 2013

101

The action research team made the decision to use the Bar-On emotional quotient

inventory assessment tool with the current students in each group. Various emotional

intelligence assessments were discussed, and the final outcome was to use the Bar-On EQi ®

which was purchased through Multi Health Systems Inc. Psychological Assessments & Services

(MHS). There was a student researcher discount for research purposes only, which requires

multiple documents to be completed by the researcher and the major professor. Upon approval

from MHS, the researcher may purchase the EQi at a discounted rate of $28 per student as

opposed to the original $40 per student. These documents were completed and forwarded to

MHS to apply for the discount. An IRB amendment was also completed and forwarded to the

University of Georgia IRB department along with a consent form for the students who elect to

take the EQi assessment.

After receiving the information regarding the cost of each EQi assessment, and knowing I

had an abundant amount of money in my program budget, at the advice of my major professor I

met with my Dean to request funding in order to help with the cost of the assessments. If I could

receive funding, this would also allow for a larger sample size which would enhance the overall

study. Unfortunately, the response was that although this information would contribute to the

knowledge base of the employees and the organization, this was ultimately my research project,

so there would be no help with the funding. I did mention that I had chosen to conduct my

research in my own organization with the intention of contributing the information learned to the

students within my own organization, but this did not seem to interest her. Just as Coghlan and

Shani (2005) mention, in action research, the higher the quality of collaboration and ownership

of the issues, the more successful the outcomes will be.

102

The team members worked to compile the current completed interview, advisement, and

clinical evaluation data as well as to construct new interview questions and a rating rubric for the

students in Group B. The researcher purchased the EQi and administered to both groups of

students. All students in each group consented and participated.

The Beginning of the Story; Am I in the Right Frame of Mind?

I began this process with a specific research question asking if an interview would be a

predictor of success in a competitive admissions healthcare program. By limiting my frame of

mind to research of only this tool, I was not open to the idea that there may be other methods that

could be excellent predictors of student success besides an interview. I know that my

assumptions stem from the fact that I have been a healthcare program director for the past seven

years and interviews have been a component of the programmatic admissions.

Looking back over student issues and successes, I have been able to “predict” most of

these successes and failures through the time I spent in the interview with the student. I will

admit, I have not been correct 100% of the time, but many times I have been correct. Therefore

this may have led to some of my biases and assumptions regarding an interview component.

Perhaps I have a bias with my “as a grown adult, you should know better” mentality?

I am very passionate about healthcare and have a strong desire to provide the healthcare

community with compassionate and caring individuals. Since graduating from college with a

degree in healthcare, I have been on the “giving” end of healthcare and never the “receiving” end

until my precious father suffered a heart attack right in front of me. He is so very important to

me, and to be forced to watch him undergo open heart surgery only three days later was the

hardest thing I have ever had to go through in my life. I felt completely helpless to the process

and could only hope and pray that he was receiving not only competent but compassionate

103

healthcare. This is an example of when the healthcare workers needed to be in the right frame of

mind in order to effectively and sympathetically perform their role, because during this time, I

know my family members and I were not in our right frame of mind.

Throughout the course of this project, I have studied the theory of action research and

action learning, and in doing so, have learned that as an action researcher, one must be open to

the idea that there can be no learning without action, and no action with learning. As Coghlan

and Brannick (2010) point out, those unable to change themselves cannot change what goes on

around them in the process. They also discuss the importance of the researcher framing their

analysis by becoming aware of their frames and remaining open to the consideration of

alternatives.

Based on this new knowledge, my main research question became focused on what the

best method to evaluate the noncognitive behaviors might be, rather than focusing on the

interview specifically. Ironically however, in researching the topic of measuring noncognitive

behaviors, the interview still surfaced as one of only a few known methods and one of the most

used methods. This was frustrating as an action researcher, because I felt that when I try to move

away from the direction of interview only, the information was still predominant in the findings.

I found myself having to keep an open mind and to monitor my biases regarding the interview

being the best method.

Reflection in Action

Coghlan and Brannick (2010) describe reflection in action as an occurrence that happens

when an action researcher is in the middle of an action and takes time to ask themselves

questions regarding what they are doing and what is happening around them. As I began the

process of constructing an action research team, I struggled with the thought of asking others to

104

embark on this journey for me. Why would busy individuals be interested in “helping” me solve

this problem or work with me to develop solutions and interventions? However, what I was

forgetting was that they would be embarking on this journey with me, not for me. My reflective

memos mirror this fear:

“Today was my first action research team meeting. I was extremely nervous because I

felt that I was uncertain regarding exactly what I needed from my action research team. I

had extreme reservations regarding asking these very busy professionals to take time out

of their day and meet with me regarding an action research ‘problem.’ My biggest fear

was that the team members would view my asking them to be a part of this journey as

simply a way to ‘help me with my homework’ and not necessarily as inviting them to

help me solve a problem. I have had several discussions with my major professor and

she has assured me that if these people are truly interested in the ‘answer to the problem’

they will not feel that they are just helping me with homework. I have to trust her.”

The better “frame of mind” for me was to remember that this is a problem that each team

member was very interested in, because it affects each one of them in a very direct way.

Throughout the process my assumption that this process would not work or would be a burden

on the participants was disproved. This is reflected in a memo from one of the team meetings:

“All in all, great meeting with very informative feedback from the members as well as

100% percent participation from the attending members who all brought materials to the

meeting for review by the team! Excellent!! This process might just work after all!!”

Another memo reflected my excitement that the action research team members were taking self-

initiative to supply documents which would be beneficial to the research problem:

105

“Since the last meeting, I had received information from one team member regarding a

group interview. She actually contacted a fellow director of another program in another

state and got all the paperwork and rubric for how they conduct a group interview and

then emailed it to me! I was somewhat surprised that I have actually found team members

who are ‘doing homework’ in preparation for the team meetings! This is what AR is all

about!!”

Conclusion

As the team meetings began to wind down and the project was nearing its final stages, I

noticed a closeness of the team had been formed. I observed body languages by the team

members that showed they had become more than co-workers collaborating on a project

together, but they had also formed friendships and new bonds. I foresee this group of people

collaborating with each other in the future and continuing to work together, particularly

regarding issues surrounding noncognitive behaviors and characteristics. Incidentally, almost

three months after the last team meeting, one of the members emailed several of the team

members with information regarding an upcoming webinar “Teaching Soft Skills in Health

Professions.” This is an example of action research still working even after the project is

complete.

This process has given me the opportunity to learn how to lead change through action

research even through my insecurities. I have evolved as a change leader and have clearly

become a model for others based on team actions, comments, and feedback. I realize I was

hesitant to lead this team, but I now know that my leadership was more of a facilitated lead with

a collaborative effort of some very brilliant team members who have taught me a thing or two

along the way.

106

CHAPTER 5

FINDINGS

The purpose of this study was to explore noncognitive behaviors as they relate to allied

health programmatic admissions criteria by identifying the specific skills noncognitive behaviors

and characteristics necessary for healthcare professions and how they are measured in an

interview component. An action research team explored alternatives to the interview and

compared an assessment tool with the current assessment tools being used at a technical college

in the state of Georgia. The results could indicate the possibility of a new assessment tool being

incorporated into the competitive admissions processes, or could support retention or adoption of

the interview component as the assessment tool of preference.

The three primary research questions guiding this study are as follows: (1a) What are the

noncognitive behaviors and characteristics identified in the literature for healthcare professions?

(1b) What are the noncognitive behaviors assessed during the interview component of a

competitive healthcare program? (2) Is there a relationship between EQi scores and ratings on

interviews, mid-term assessments, or clinical evaluations? (3) In what ways does an action

research team process lead to individual, group, and organizational learning and change? This

chapter presents findings from a review of the literature, analysis of various assessment tools

which measure noncognitive behaviors, and action research team meeting notes, memos, and

interviews of the team. The findings and learning’s are organized in response to each of the

three research questions.

107

What are the Noncognitive Behaviors and Characteristics for Healthcare?

A focused review of the literature as it relates to the measurement of noncognitive

behaviors for competitive admissions was conducted. Findings of the literature review include

specific behaviors and characteristics for the healthcare profession along with noncognitive

behaviors assessed during an interview component of a competitive healthcare program. As

indicated in the literature review chapter, authors found specific behaviors and characteristics

which are relevant to the field of healthcare. Table 10 provides an overview of the noncognitive

behaviors and characteristics that were identified in the literature for healthcare professions.

Throughout the literature, there was a mix of language used to “title” these terms. Some

of the various terms used were behaviors, skills, characteristics, variables, traits, qualities, and

attributes. For clarification and consistency, the terms characteristics and behaviors will be used

in these findings as umbrella terms to respond to the research questions.

108

Table 10

Findings of Noncognitive Behaviors and Characteristics in the Literature

Noncognitive Behaviors and

Characteristics Identified in

Literature for Healthcare

Professions

Specific Healthcare

Profession Contributing Authors

Empathy (general healthcare

provider) Benbasset & Baumal, 2007

Attitude (general healthcare

provider) Cate & DeHaes, 2000

Integrity

Honesty

Fairness

Conscientiousness

Helpfulness

Interpersonal skills

Personal welfare

Empathy

Doctor Hughes, 2002

Leadership skills

Integrity

Compassion

Pharmacist Joyner, et al., 2007

Maturity

Empathy

Ethical integrity

(general healthcare

provider) Salvatori, 2001

Compassion

Emotional maturity

Self-awareness

Empathy

Nurse Storey, 2008

Motivation

Decision making skills

Interpersonal communication

Ability to work in a team

Empathy

Radiation Therapist Weege, 2009

Truthfulness

Honesty

(general healthcare

provider) Young, 1997

The review of the literature yielded many different characteristics. However, of these

characteristics, empathy, a common characteristic, was listed most often and by multiple authors.

Integrity was another characteristic listed by more than one author and discussed by Salvatori

109

(2001) as being not only a contributor to successful performance in clinic, but also in the area of

academic outcomes. Hughes (2002) noted that a great deal of emphasis is placed on a doctor as

an upstanding professional who exhibits these types of noncognitive behaviors and

characteristics.

What Noncognitive Behaviors are assessed in an Interview?

In response to research question number (1b) “What are the noncognitive behaviors

assessed during the interview component of a competitive healthcare program,” the targeted

review of the literature produced results similar to the initial question related to “What are the

noncognitive behaviors and characteristics identified in the literature for healthcare professions?”

Numerous authors agreed that some of the same behaviors or traits are measurable during an

interview. A table of findings, Table 11, was created to summarize the noncognitive behaviors

identified along with the corresponding author who discussed the behaviors.

110

Table 11

Findings of Noncognitive Behaviors Assessed During an Interview

Noncognitive Behaviors Assessed During an

Interview Component of a Competitive

Healthcare Program

Contributing Authors

Personal characteristics Kudlas, 2006

Self-awareness Storey, 2008

Empathy

Benbasset & Baumal, 2007;

Hughes, 2002; Salvatori, 2001;

Storey, 2008; Weege, 2009

Emotional Maturity Salvatori, 2001; Storey, 2008

Decision Making Skills Weege, 2009

Interpersonal Skills Hughes, 2002; Weege, 2009

Leadership Joyner, et al., 2007

Motivation Weege, 2009

Teamwork Weege, 2009

Life Experiences Kudlas, 2006

Emotional Intelligence Kluemper, 2006

Integrity Joyner, et al., 2009

Compassion Joyner, et al., 2009

Most authors agreed that these types of noncognitive behaviors and characteristics need to be

assessed for students entering educational programs in medical fields. They also agreed that

these behaviors are difficult to assess due to their subjective nature (Edwards et al., 1990;

Weege, 2009; Wright & Miederhoff, 1999).

In conclusion, and in response to part A and part B of research question one, the literature

revealed the following specific terms as being listed more than once throughout the review:

empathy, integrity, honesty, interpersonal skills, compassion, maturity, teamwork, leadership and

self-awareness. With the work of the action research team, team members felt that the following

terms were most important: compassion, self-awareness, life experiences, empathy, appearance,

and maturity. These findings enabled the team to examine what assessments actually measure

the behaviors and characteristics of greatest interest.

111

Relationship between EQi Scores and Various Assessments

This section presents quantitative findings from research question number two, “Is There

a Relationship between EQi Scores and Ratings on Interviews, Mid-Term Advisements, or

Clinical Evaluations?”

The action research team made the decision to collect data from students in two different

healthcare programs. The data consisted of interview ratings from multiple raters for each of the

students enrolled in each of the programs. Data were also collected from multiple raters’ mid-

semester advisement forms (clinical component) of the students in Group A, which encompassed

10 months of time spent in the program for each student. Data were collected from multiple

raters from clinical evaluations of the students in Group B who were enrolled in their program

for 10 months. Data were collected from the individual score reports of the Emotional Quotient

Inventory assessment and compared to the interviews, advisements, and clinical evaluation

scores of both groups of students. For each group, the clinical assessment and the interview

assessment were also analyzed with each other. The results of the assessments were analyzed

using SPSS software v. 17 and comparisons were made between the assessments as well as

comparisons of each instrument with itself using correlational analyses.

Group A Comparisons

The mean interview scores and the mean clinical ratings of the mid-semester advisement

evaluation for students in Group A were compared with the five subscale scores of the EQi and

the total EQ for each student. There was an “n” of 18 students in Group A. When these

assessment comparisons were made, results showed that the EQi had no correlation with the

clinical ratings. This finding indicates the EQi is measuring something different than what is

being measured in the mid-semester advisement (clinical component) tool.

112

There was a slight positive correlation with the interview and the clinical ratings in this

program, which is of particular interest to the researcher because interviews are performed

primarily by program faculty, and mid-semester advisement scores for the clinical component

are completed based on feedback from clinical instructors and technologists at the clinical sites

where the student has rotated for internship.

None of the EQi sub-categories or total EQ correlates with clinical performance which

indicates that emotional intelligence does not appear to be captured in this data and is not

correlated with what is rated in the mid-semester assessment tool. There is some evidence of a

relationship between the sub-category in the EQi of stress management and also with the total

EQi, however the relationships are negative.

The sub-category of adaptability in the EQi is significantly correlated with the interview

and is significant at .034; however the relationship is also negative at -.501 which indicates that

as one is high the other is low. Adaptability appears to explain 50% of the variance in interview

ratings. Table 12 shows the results of the Spearman’s Rho correlations with the sub-categories

and total EQi, mean of interviews, and mean of the clinical component of the mid-semester

advisement.

113

Table 12

Correlations between EQI, Clinical and Interview for Group A

Overall findings for Group A indicate that the rating scale of the interview is closely

related to the rating scale of the mid-semester advisement and as indicated in Table 12 the two

measures correlate with each other. However, they did not correlate with the EQi. This could

indicate that these forms are more cognitive in nature and not capturing the noncognitive skills

and behaviors which are of interest to the program faculty. There was less variance overall in all

dimensions for these assessments, less variance in EQi scores, interviews, and mid-term

assessment for Group A, thus more data with greater variance is needed to fully test the

hypotheses in this study. Mean scores on the total EQi (108) were higher than Group B (104),

indicating a higher average emotional intelligence among those students selected in Group A and

none of the students in Group A had a total EQi below the published range. This was an

interesting finding regarding the mean of the overall EQi scores for each program because the

program which included an interview as part of their admissions process had a higher overall

114

EQi mean than did the program that did not use any measurement other than cognitive based

admissions tools.

Group B Comparisons

The mean interview scores and the mean clinical evaluation scores for students in Group

B were compared with the five subscale scores of the EQi and the total EQ for each student.

There was an “n” of 22 students in Group B. All of the EQi sub-categories which included

intrapersonal, interpersonal, stress management and adaptability were significantly correlated

with the clinical evaluation except for the sub-category of general mood. Thus, variance in

clinical ratings was at least partially explained by higher emotional intelligence scores. Both

total EQi and the interpersonal dimension explained 50% or more of the variance in clinical

scores. This relationship confirms the initial hypotheses of the action research team that

emotional intelligence or noncognitive behaviors were important determinants of success in

professional practice.

Interestingly, unlike the Group A correlation results of interview with clinic, in the

correlation of interview and clinic ratings for Group B, there was no correlation. This could be

due in part to the difference in the rating form used by this program; i.e. what was assessed in the

interview as part of the class was a different set of constructs from those assessed in the clinical

ratings.

Table 13 shows the results of the Spearman’s Rho correlations with the sub-categories

and total EQi, mean of interviews, and mean of the clinical evaluations for Group B.

115

Table 13

Correlations between EQI, Clinical and Interview for Group B

What is of particular interest for this study is that the results for Group B most closely conform

to the hypotheses of the action research team. It was the belief of team members that

noncognitive behaviors and traits are what are essential to success in practice, and indeed they do

correlate with the assessments of effectiveness in practice. On the other hand, the team was

exploring the use of the EQi as an alternative to the interview to assess noncognitive behaviors

and characteristics in the admissions process. Since there was no correlation between the

interview and the EQi dimensions, we have to conclude that the interview may capture

something different than the skills and traits measured in the EQi.

In conclusion, the results of Group A showed that the EQi had no correlation with the

clinical assessment and each seemed to be measuring something different. The slight positive

correlation with the program interviews and clinical assessment can be viewed as either the tools

are measuring the same behaviors, or the tools were too closely related based on the fact that

they are very similarly created assessments. However, each assessment was filled out by

116

separate groups of people, program faculty and clinical instructors, who both found correlations

in the student’s behaviors prior to admissions and then later during their clinical rotations at the

clinical facilities.

Contrastingly, the results of Group B suggest just the opposite as the results of Group A.

In relation to these results, the EQi could be an excellent addition to the measures currently used

at admissions for this group of students. This is an interesting finding because this group of

students did not participate in an interview or noncognitive assessment of any kind prior to

admission into the program. They were admitted based on cognitive factors which included a

GPA and a standardized cognitive test. Because the clinical assessment for this program had

correlations with the EQi, they appear to measure the same set of behaviors or characteristics

which this profession deems important for a healthcare student. Further research with a larger

sample is needed to more definitively affirm this set of relationships.

Action Research Team Process

This section presents findings from transcripts of action research team meetings,

researcher memos, and critical incident interviews of the team members in response to research

question number three, “In what ways does an action research team process lead to individual,

group, and organizational learning and change?” At the conclusion of the study, team members

were asked five primary questions to identify and explore key informal and incidental learning

experiences regarding the action research process. See Appendix B for the critical incident

interview questions.

The findings are organized by major themes and categories which emerged during the

analysis of the data. Table 14 provides an overview of the findings and categories for research

question number three, which is: “In what ways does an action research team process lead to

117

individual, group, and organizational learning and change?” The findings are divided into three

major themes which are “Individual change focused on working collaboratively and trying out

new or adapted approaches to assessing non-cognitive behaviors and characteristics,” “Team

learning focused on finding shared experiences, shared concerns,” and “Organizational or system

learning focused on developing a model for evaluating impact and predicting student success.”

At the individual and group level, team members generated actions and words which

showed their enthusiasm through participation and collaboration. They also shared best practices

and connected with one another illustrating the overarching practice of action research. At the

organizational or system level, team members discussed noncognitive behaviors on predicting

student success and how facilitation of changes could impact this success. Discoveries were

made regarding inconsistency in administration which directly affects organizational changes

and learning.

118

Table 14

Table of Findings

Research Question

Findings from Data

RQ3: In what

ways does an

action research

team process lead

to individual,

group, and

organizational

learning and

change?

Individual change focused on working collaboratively

and trying out new or adapted approaches to assessing

noncognitive behaviors and characteristics

Participation and Initiative of Team

Alternatives Explored

Teams Perception of their EQi results

Teams Opinion of Best Measure

Team learning focused on finding shared experiences,

shared concerns

Noncognitive Behaviors and Characteristics for Healthcare

Professionals

Student’s Lack of Awareness or Dedication to Profession

Organizational or system learning focuses on developing a

model for evaluating impact and predicting student

success

Noncognitive Skills Measured in an Interview

Facilitation of Change

Inconsistency in Administration

Individual Learning and Change

Throughout the course of the project, themes emerged related to the individual learning

of each action research team member. The team members worked collaboratively to discover

new or adapted approaches of assessing noncognitive behaviors and characteristics. Findings

119

show that the individuals expressed feelings of excitement and appreciation of action research,

along with gaining an appreciation for the experience of each other and their respective

healthcare roles. This provided an opportunity for team members to explore alternatives and

learn more about themselves as they participated in an alternative method.

Participation and initiative of team. There were numerous actions taken by the team

members which displayed a major theme of participation and initiative of the team members

during this process. Members showed immediate individual initiative beginning with the first

team meeting by taking responsibility for the project. During the first meeting, two separate

team members offered to contact their fellow program directors across the country in order to ask

about information related to the issue of alternatives to measure noncognitive behaviors. At the

second team meeting, these team members brought the information they received from other

healthcare professionals regarding their admissions process. Researcher notes reflect the

excitement of the researcher in relation to the team members “doing their own homework” in

order to prepare for each meeting:

Since the last meeting, I had received information from one team member regarding a

group interview. This member actually contacted the director of another program in

another state and got all the paperwork and rubric for how they conduct a group interview

and then emailed it to me! I was somewhat surprised that I have actually found team

members who are “doing homework” in preparation for the team meetings!

Team members also showed personal initiative by researching the EQi assessment, emotional

intelligence, and journal articles related to noncognitive assessment. One team member stated,

“That’s what I did yesterday when I came in because there’s a lot of data out there when

you go on the internet. I googled, ‘How do you measure compassion?’ I did this late

120

last night and stayed because I’m like, ‘I don’t want to come here without doing

something.’ But this is a great article and I made a copy for you.”

Other team members offered to bring their programmatic assessments and evaluation tools for

the other team members to look at and learn from. This participation created knowledge for the

team members during this process. One team member stated: “Gosh, the only thing to add is

how much I realized I don’t know. It’s definitely helpful to participate in things like that so I can

enhance my knowledge up front and hopefully not have to figure things out alone.”

Team members also showed interest and initiative in regards to conducting research on

their particular groups of students. Several team members were very willing to offer their time

and resources related to their program and students. One team member in particular joined the

action research team after another team member retired from the college. This team member

showed immediate initiative and became an instrumental part of the process. Researcher memos

reflect this:

I was excited to see that the new team member is very involved and seems to be very

excited to work with us on this process. She was offering to get evaluation tools from her

program and bring to the next meeting as well as began to offer her help to the team by

incorporating the group of students that she currently has as part of the research! This

became the new direction of the research by using her current students as part of the

intervention option! She turned out to be a very valuable addition to the team!

Team and researcher collaboration. Another recurring theme at the individual level was

the team’s collaboration with each other and with the researcher. As a researcher in action

research, the goal of the researcher is to collaborate with the team rather than dictate or “lead”

121

the team’s decisions. This was evidenced by numerous comments from the team members,

particularly during the final interviews. One team member stated,

“I think based on the fact that you had four disciplines involved and we all were equally

involved in the research….we equally had the opportunity to provide input to add little

pieces that were pertinent to our discipline, because even though we’re all health care we

all have different specifics or things that are specific to our discipline that makes it

unique. I think by being a team that we were able to collectively perform the research or

participate in the research that could be useful across all discipline and the action research

allows you to be more involved in the process as a part of that team, which you

traditionally don’t always get especially if you’re not the primary researcher or in

traditional research.”

Another team member recognized the collaboration of the team and the collaborative decision of

the intervention option as being a critical turning point for her in the action research process.

“The effort of everybody coming together, because it seemed like everybody had

something to contribute to that. It is just like this is coming together, and everybody has

an opinion, and everybody is working together to see what would be best, the best way to

move forward, so all of that. I think we had a good team all along. I think what happened

in that meeting was again we researched the EQi test. All members were involved in that

and I remember all of us discussing the test and you had written the options down. We

reviewed the options and think that maybe might have been one of the critical or turning

points.”

Team excitement about practical significance of action research. Throughout the course

of the project, there were numerous statements and actions which reflected the true excitement of

122

each individual team member during this process. The team members repeatedly expressed

genuine excitement regarding the opportunity to work together on an issue and experience the

outcomes of their hard work. During a critical incident interview, one team member stated initial

hesitation for the project, which later turned to excitement for the research.

“In all honesty, in the very beginning when you had asked me, I’m not one for ever

saying no. I tried to say, ‘Okay, can I contribute anything to this?’ In the beginning I was

wary, but then after speaking with you and going to the first meeting that was, I was

excited about it. I actually, and I’m not just saying this, I started looking forward to

going to the meetings!”

Another team member stated: “The fact that I contributed in very small part to the research that

add some evidence in this area, I am just excited that that could be a part of it!” And yet another

member stated: “Basically when I was asked to join the team I was very excited because of the

project that was going on, because of the fact that it was the interview process that was being put

under the microscope. I can’t wait to see this!” This comment reflects this individual’s

excitement to learn more about the issue being researched.

A researcher memo mentioned observable body language, which showed true excitement from

one team member during the meeting of the discussion of the results:

There was excitement from one team member (which was evident by her bouncing

around in her seat and her high-pitched voice) to learn that her clinical evaluation tool

had some high correlations; however there were some areas that would need some

tweaking in order to make it an even better tool. She stated numerous times how excited

she was to learn this and even more excited to take the results back to her program

director for future use.

123

Teams appreciation of action research. During the critical incident interviews, there

was a theme of appreciation for the process of action research which most members admitted

was a new approach to them and not only gave them an opportunity to investigate the issue, it

also gave them an opportunity to learn more about the process of action research. Team

members acknowledged that they had never been a part of action research nor were they aware

of this method of research; however, they showed a true appreciation at the conclusion of the

project.

“Not only am I a health care educator, I’m also a student. I think it’s going to help me a

lot going forward because I plan to pursue a doctorate of some sort. Of course, there’s

always research that’s involved. It just helped me to understand how to choose a topic

that is actually going to impact the way things are done. Processes that need to be

changed. That’s what happened with us, with action research. This process for me has

helped me to see if you choose the right topic that it can definitely be a huge benefit to

not only your program but to every other program in the college.”

Another team member showed an appreciation for the collaborative inquiry that is action

research by stating:

“The action research is quite different and I actually enjoyed it. It was new for me. I

enjoyed it because it brought a team together of experts if you will experts in their area,

in their particular discipline that we were able to come together and immediately look at

a tool and see how it could be generalizable across professional disciplines, but you

typically wouldn’t get this in traditional research.”

124

An additional member stated: “I think I like the process because I think getting people

involved and the action part is better than just strictly having your own study and gathering the

data from groups or whatever.”

Experience of the team members. There was a common theme among team members

regarding the opportunity to learn to appreciate each other and the various healthcare

professions with which each team member had experience. Team members echoed their

thoughts that this process was very effective in part due to the fact that each team member had

so much experience in their field and with that experience came many shared practices. Team

members seemed to value the different backgrounds of each other which added to the richness

of contributions they were able to make to the research and for each other’s benefit. One team

member stated:

“It didn’t allow me, as a therapist, to only focus on therapy but to look at how this type

of research or this project in particular could be beneficial to multiple health care

programs. In terms of working best, I think that we as a team came together as various

professionals, as health care providers who genuinely believe in the research that was

being done.”

Another team member pointed out in the interview, “I was glad to work with definitely with

other disciplines. I think it’s good to see how other programs do things because everyone is so

very different. I think that was more helpful maybe than it would have been if it had just been

other programs exactly like my own.” And another added, “As far as the team members,

everybody was very knowledgeable, and brought a lot to the table because all of us deal with

students on a daily basis.” And finally, another added this statement, which indicates the

learning and benefit of learning from others:

125

“I think having the point of view of other colleagues that the health care professionals

brought a different perspective to it, it allows you to think outside the box, it allows you

to entertain options that you might not have originally thought of. Again I can’t

emphasize the significance I believe in having the opportunity to interact with other

health care professionals in getting their point of view.”

Alternatives explored. Part of the individual change and learning was the

collaboration regarding the exploration of various assessments to measure noncognitive

behaviors for medical programmatic admissions processes. The team discussed the use of

interviews as well as the emotional quotient inventory as an alternative to the interview. There

was discussion around the concern for liability of an interview and discussion regarding the

concern of validity and fakability of the EQi assessment. One team member discussed her

single finding of an additional alternative, after she reached out to a fellow program director in

another state. During one of the meetings, this team member shared this information:

“One place did something truly different, but to me it sounded [like] it could be more

liability with it than interviews. They haven’t had a problem with issues. They’ve been

doing [it] about 10 years. They called it a group interview… She said, they have never

had any sign of someone wanting to sue or whatever in the 10 years.”

This reaching out of a team member is indicative of the self-directed learning of an individual

and the effectiveness of action research.

Team members discussed the fact that some programs at the college utilize some form

of measurement of noncognitive skills, and there are some programs that do not. Based on this,

the team agreed that they do have similarities in their admissions processes; however, not

everyone uses an interview, and throughout this discussion, team members recognized there are

126

other tools available which may also measure noncognitive skills. A team member stated: “I

guess just hearing others talk about some of the different measures that were even there that

they were interested in looking at, it made me realize oh, wow, great, maybe there is something

we can use or an interview.”

An example of the team’s discussion around exploring different options was captured in

one of the team meetings:

“I think that noncognitive skills or measuring noncognitive skills for admission for the

health care providers is important and that the interview process is one good method of

doing that and the EQi can also add to that, because basically what I got out of our last

meeting was that there was little overlap between the two tools. Therefore both of them

may be needed to look at various noncognitive skills, so we could use the EQi as one

assessment tool to look at certain things versus an interview which again looks at

noncognitive skills, but it looks at some other things in a different way.”

Another area of discussion that was brought up during the exploration of various

options to measure noncognitive behaviors was the team members views that these behaviors

are not measured in any way by cognitive type tests or assessments. This was a comment

from one team member:

“Because in my opinion the GPA or a standardized test is not the best way to measure

noncognitive skills because GPA is not predictive of performance in terms of clinical

skills. I would say it is not unusual for the majority of med students to be a 4.0. There

is research out there that basically confirms that the GPA is not the best predictor for

clinical performance or assessment of soft skills or ability to perform soft skills. I

don’t see GPA as being the greatest assessment tool.”

127

Another team member pointed out the need and importance of cognitive assessments and the

role they play in measuring the intellect of the student by stating:

“You’re looking at medication errors, the eighth leading cause of death in America. Just

like you have to have those Math skills, you have to. With the drugs and everything, it’s

the science part of it and it’s kind of important, really important. It’s a known fact that

tests, pen and paper that definitely measures competence.”

Team members agreed that both cognitive and noncognitive behaviors need to be measured

during the discussion of various options.

Interviews and liability. During the collaborative discussion around alternatives, the

panel interview was discussed by the team. Team members commented on the value of an

interview as well as the liability associated with this subjective type of assessment. One team

member who currently uses interviews in their admission process mentioned the value of an

interview as a teaching tool from the perspective of the student.

Just the other day I had a graduate say to me. “The only time I’ve ever really had an

interview was the interview to get into the program and I am so glad I had that because

it gave me such good practice for the interviews that I now had for my job. I pulled out

the same suit that I bought to wear to the interview…do you know it was the exact same

thing and I am so glad that we went through those interviews because I had an idea of

what to look for and how to act in the interview and it was the exact same way.”

Another team member added to this comment by relating to job interviews being done at the

college: “You still have job interviews. I mean, I don’t understand… They wouldn’t hire

somebody to teach at this college without having seen them and have an interview.”

128

One team member in particular began the action research team meetings disagreeing

with the use of interviews. She stated: “I did not initially agree with an interview because I

was like how can you interview somebody with 10 minutes and know that they’re going to be a

great doctor or nurse or … but I think with the questions …” She later admitted to the fact that

after reaching out to other programs and receiving a copy of interview questions they use in

their program admissions, she had changed her mind about the use of an interview and felt that

she had learned something new. She was now of the belief that the interview could be a

valuable tool used to assess these behaviors, particularly if the questions were created to “fit”

the program and profession. She stated:

“I sent and I gave you a copy, I sent emails out to a few of the [discipline] programs in

the country that did interviews and I heard back from one and the questions are on the

next page. They’re specific to [discipline] applicants. My colleague and I love the

questions and we would like to start interviewing! We would love to…..now that I have

seen the interview questions we would love to incorporate all of these! I’m excited

about the … I think the interviews which are another tool that we could use.”

One team member talked about the interviews that their program used as part of a leadership

course the students had to take. She explained that the program faculty had discussed how the

interviews during that course gave them the opportunity to “see” and “hear” how a student

responds to the questions. She also mentioned that the program faculty felt that if they had an

opportunity to interview the students prior to admission, they would have recognized some of

the problematic students that were currently in the program and would have been able to make

the decision that those particular students would not have been a fit for that program. She

stated:

129

“We were doing our interviews in the interim for part of their leadership course but we

did learn a lot about these students through that interview process. All of us on the team

reflected that if we had been able to use this interview prior to them getting it, it

probably would have affected their acceptance. When you see something in hindsight,

for instance there have been several problem students that we feel like we could have

weeded out of the program based on the interview that we couldn’t because we weren’t

allowed to do the interview beforehand. Because I really believe that with this

particular person, and the interview that he presented, we would’ve picked up on that, I

really do. There were actually several who wouldn’t have gotten into our program

because of that.”

During the discussion of the interviews, there was a recurring theme surrounding the fear of

liability of an interview. Team members were occasionally in disagreement regarding the fact

that an interview could be a liability. One member stated: “The perception is that the interview

process is a liability… We have thought about it. We have just kind of tossed it out there and

talk about it. I think it does something for lawsuits. Students are going to come and say well,

I’ve got a 4.0 and I talked to this person for 10 minutes and they took somebody else, it’s just a

liability.” Another team member countered with:

“We’ve actually had an attorney to come in to talk to us about admission’s processes

and things like that. Basically, if it ever went to court it would be a judgment call based

on the expertise of the faculty is what they told us. Nine times out of 10, the judge is

going to rule in favor of the expertise of the faculty and the individuals who are on that

panel. Because it’s not like that interview is the stand-alone thing or the only thing that

the admission is based on. Every applicant should be informed of what is included in

130

the admissions process. It consists of these things’…. It’s not like that 4.0 GPA is the

predictor or that the interview by itself is the predictor in terms of who will be

admitted.”

During the course of the discussion around interviews and the possible liability, another

theme developed regarding multiple admission criteria as well as diversity of panel members in

order to curtail a possible litigation from an interview score. Team members also shared their

thoughts on the importance of using the same panel members and asking the exact same

questions of each student. This is reflected in a comment by one team member’s discussion of

who is on their interview panel: “I mean we have the program faculty which is basically the

two of us and then a psychologist represented. We also have clinicians who participate and

we’ve actually had former students participate.”

Another team member explained that she always asks a male to be on the panel since a

male student was not accepted into the program and went to administration stating the reason

he was not accepted was because there were no male panel members. Another panel member

stated that it is also a good idea to have someone who is of a different race or ethnicity because

it adds more diversity to the panel members which may help with liability issues. During the

course of this discussion, one team member excitedly stated: “I have never called on another

diverse person to be on the panel. Look there! I’m learning stuff already!” There was also

discussion that in regards to the program faculty, their numerous years of experience in the

profession may give validity to the panel as professional expertise of the panel members.

Two team members brought up the idea that success of a student could be predicted

collaboratively by the expertise of the panel. One person stated: “That’s what we found when

we did our interviews. That once a student would leave, we would do just a small bit of

131

collaboration and nine times out of ten, we all came to the same conclusion without even

talking to each other. I mean just by looking at what we had written on our interview sheets”

and another team member added: “You have different perspectives from each person on the

panel. When we're sitting on the panel … it’s uncanny sometimes that all of us do come to the

same conclusion and get the same perception. Every now and then, there’s a little difference

but commonly we all kind of hear and see and feel the same thing.” As the team members

viewed this collaboration as a benefit to the interview process, could it not also be viewed as

inter-rater bias? Are the team members influencing one another’s decisions? This was not

discussed.

EQi as Alternative and Validity of EQi. During the discussion of alternatives to the

interview, as the researcher, I introduced the Emotional Quotient Inventory, the EQi, to the

action research team. I explained that this assessment is supported by 17 years of research and

is used internationally as well as multi-culturally. During the critical incident interviews, one

team member showed appreciation for the knowledge and the introduction of the EQi as an

additional resource. She stated:

“This process allowed me the opportunity to look at other options for assessing

noncognitive skills. The EQi is a valid tool that I was unaware of before entering into

this project. Having the chance to take the EQi assessment myself, seeing how

relatively easy it is.”

One theme surrounding the discussion of the EQi was the idea that college

administration would be glad to see an objective measurement of noncognitive skills over a

more subjective measure like the interview. One team member stated: “I think that would be

better than the interview. How can they fight that? They couldn’t dispute it because it’s a pen

132

and paper, graded or whatever test.” Another added, “Now if this works, I think the school

would be absolutely tickled.” An additional statement was made by yet another team member,

“Administration wants it more objective, too. This would be more objective than an

interview.” One team member also added that the EQi would be much easier to carry out than

interviews are. She stated, “Because you’re right, if you can do it this way, it would be a lot

easier than the interviews.”

A theme emerged regarding the cost of the EQi. Some team members felt that asking

the student to pay for the assessment would not be a good idea; however, others disagreed

stating that if a student is asked to pay, it may add more value to a student’s program decision.

One member indicated:

“And I think that gives some students pause to think about “Is this what I really want?”

Because I think some students, particularly what I saw initially in my program, were

students just applying for the heck of applying just to see what would happen. I think

now, I’m a believer that if there is no monetary attachment to something, a lot of times

students don’t see the value in it.”

While one team member showed interest in the EQi for her programmatic admissions during a

team meeting by stating that she would like to use the EQi in her specific program, another

team member expressed hesitation during her interview, regarding its effectiveness by stating:

“I think the student’s perception of themselves is probably not as astute as it should be

because when you go back and look at the results of the EQi but then you look at the

results of the clinical evaluation grades, they don’t always reflect each other. I think that

was the biggest thing to me is that they, their perception sometimes of themselves are a

little higher than what it actually is.”

133

Researcher memos reflect an additional discussion and team agreement regarding the EQi as an

alternative:

There was also a great conversation around the fact that several team members felt that

this self-assessment would be answered totally different depending on the age of the

person and how much “life experience” the person would have. When that comment

was brought up, I noticed most of the team agreed. However, it would be difficult to

assess this given the fact that our ages were so similar.

Themes also emerged concerning the possible validity or fakability of the EQi. Team

members felt that a student could intentionally misrepresent themselves in order to be a viable

candidate for the program for which they were interested in applying. After the team members

took the EQi themselves, one team member stated:

“It’s the one that they want you to have. Because I mean seriously, if I went back and

took this again, if I wanted to like go back and take it, if I wanted to misrepresent

myself, I could easily, easily. And if you’re using it for admission criteria, it kind of

makes you think they’re going to steer you in that direction. They’re going to make

themselves really, really good so that they can get in. When in an interview, you can

actually see… the interaction between everybody in the entire team and that person.”

I reminded the team that the instrument does have a positive and negative impression scale in

place to protect against this and detect respondents who may be giving an exaggerated

impression of themselves. One team member added this, “the EQi has been proven to be valid,

therefore it should measure what it is meant to measure; therefore, it would be difficult to

“fake” results.”

134

However, another added:

“I did have the same concern that you guys had… If I’m thinking, I’m trying to get into

a medical program and I know that you need to have affection for your patients or

whatever, I’m going to put ‘always’ in those categories or questions.”

One team member shared this:

“I’m always leery and I guess just from a research perspective, of self-reporting things,

which can create a huge bias and really determine or have an effect on the external

validity of any research. That was my question about that one. I even found an article

where a guy actually set out to disprove that …whole tool.”

Teams perception of their EQi results and assessment. As part of the collaboration

and exploring of options, I purchased the EQi assessment for myself and all action research

team members. After we took the assessment, we had a meeting to discuss our results. Themes

emerged during that meeting and meetings that followed, related to how well the team

members felt the EQi had accurately assessed them along with thoughts that their age and life

experiences might have had an effect on their scores. Opinions were varied as to how well the

EQi measured each person’s noncognitive behaviors and competencies. Of the five team

members, only two felt the instrument had scored them according to the way they view

themselves. Would this mean the instrument had in fact scored them incorrectly, or are our

perceptions of ourselves not quite as accurate as the way others truly view us? This was not

discussed.

One team member stated, “I think for me, it measured these areas well. I have to admit,

there was one I was surprised I didn’t do as well in but…” and another member stated “But I

135

think this…with the exception of one or two things, it’s fairly accurate.” On the contrary,

another team member said:

“I was surprised at parts of it I guess. I didn’t quite understand why the score would

have been what it was. I couldn’t see why. I don’t feel like the scores said…. Just odd.

Wow, looking at my score, I wonder if I’m more fit to work with test tubes than people.

With my results, I think I should have taken more time [to take the assessment]

apparently, because either I really shouldn’t be admitted to any program (Laughing) or

maybe I got some of them switched around somehow, I don’t know. I just kind of

wonder like some areas I think are probably accurate. Then, there’s other areas seem

like that’s weird.”

Themes reflected the team’s feelings that they answered the EQi questions based on their life

experiences, their age, and their years in the medical field. A team member stated, “Twenty-

three year olds are not going to get the same thing because they don’t have those life

experiences to pull from.” One statement made by a team member indicated this:

“If I had to look at certain things that I’m encountering in my life right now, my

response would be different than how I responded or how I would expect that I would

respond if circumstances were different. That played a major role for me and one of the

reasons why I know it took me so long. I had to tell myself I’ve got to take myself out

of this situation and some of the stressful things that I’m experiencing right now and

look at it holistically in terms of how I would generally respond to this.”

Teams professional opinion of best measure. During team meetings, a recurring

theme seemed to be the professional opinion of the team members that the interview is the best

way to measure noncognitive behaviors. This theme continued even after the team had taken

136

the EQi. The consistent reason that the team members stated they felt the interview was best

was due to the face-to-face interaction that you have during an interview. One team member

stated,

“I’m thinking I would favor an interview over this just because of the actual face-to-

face interaction. And that’s why interviews are good because you can see the mature

adult person as opposed to the young person that comes in, cell phone on, tattoos and all

that.”

A team member added, “I would agree with that. I’m not saying that this could not have value

but if I’m making a choice between the two, I would still favor the interview.” Another added

“You know employers are still doing interviews. To me nothing beats an interview. Some

people test good.” And finally another team member indicated “You can’t beat an interview. I

mean you just can’t beat an interview. I wasn’t losing students academically. I haven’t lost

students academically since we started using the grade point average and the HOBET.”

Group Learning and Change

Team members not only had the opportunity to learn and collaborate at the individual

level, but also at the team or group level. This experience gave team members the chance to

share same experiences and concerns they each had around the issue of noncognitive behaviors

and characteristics for healthcare professionals. Findings indicated the team members

discovered how student’s exhibit lack of awareness or dedication to their chosen profession in

more than program which led to discussions around the possibility that an interview has a

potential to assess a student’s knowledge of the profession. Team members also focused on

sharing knowledge regarding what they felt to be specific noncognitive behaviors and

characteristics for success in healthcare and why they need to be measured.

137

Noncognitive behaviors and characteristics for healthcare professionals. Themes

evolved around what noncognitive behaviors team members considered to be beneficial and/or

necessary for healthcare professionals. Team members agreed that healthcare is a high stress

environment and is not an exact science. As a group, team members identified some of the

same behaviors necessary for any healthcare program: compassion, attitude, work ethic, self-

awareness, professionalism, adaptability, and integrity. They shared stories related to these

various behaviors in the scope of their programs. For example, one team member discussed

sending students home for failing to follow the programmatic dress code which shows a lack of

accountability on the part of the student:

“But we have a handbook, and it has a dress code we have to abide by that, and our

instructors know to send them home if they come in with a wrinkled uniform, or even if

they have a missing piece, oops, I forgot my stethoscope, I didn’t bring my watch, I

don’t have a pen, can I borrow a pen? No, you’re not prepared, go home.”

When the team members began discussing the importance of communication, several

comments were shared based on personal experiences, for instance, “In our field, you’d have

to be able to describe what’s going on with your patient to the doctor over the phone, so

communication is paramount” another added, “For us, it’s adaptability in the communication, I

mean, to be able to talk to this group, and this group, and this group, and get your point across

without making them feel underwhelmed or overwhelmed.” One team member shared a

particularly interesting point related to communicating with different groups of people:

“With communication, especially when you’re dealing with patients you have to make

sure that it’s culturally appropriate. First of all, is English their first language or not?

How do you deal with that? And then you have to look at, other, cultural

138

considerations. I mean if we were talking about the black community, and I said

hypertension, the average black person wouldn’t know what I was talking about, but

high blood pressure they might know. If I said diabetes, they may not know. But if I

said ‘sugar in the blood’ they might know. So you have to look at communication from

a cultural perspective as well.”

Even as a healthcare professional myself, I was unaware of these differences which was a

moment of professional development for me. The knowledge that we obtained during our

interactions as a team will benefit us in each of our different professions and can easily be

applied in various healthcare related situations as well as passed along to our students. After

that statement was made, I found myself sharing that information with my own students the

following week during my lecture.

Relationship to success and application to various medical programs. As the group

focused on finding shared experiences, themes emerged regarding noncognitive behaviors and

their relationship to success and how they can be applied to all healthcare programs. One team

member stated: “What we find should be able to be applied to any medical program because

every program needs a compassionate person because in the end they’re still dealing with the

patient.” Another team member stated, “Even though we’re in totally different professions,

they’re still all under that umbrella of medicine and still skills that I think we all agree that the

students need to show.”

Several team members agreed that the majority of students who have been dismissed

from their programs were dismissed for poor noncognitive behaviors or low emotional

intelligence.

139

One member stated,

“I’ve never failed anybody academically. Period. It’s all been…clinical, immaturity,

attendance, work ethics. I don’t lose students academically anymore. I don’t. They fail

because of their work ethics. They fail because of this, what you’re calling soft skills.

That’s what's failing them.”

A team member added to this by remarking that this was a problem in her program as well:

“Yes, these behaviors affect me as an educator….. there are a couple of students that I

have in class right now and I have to reign them in. Pat (pseudonym) my clinical

coordinator and I have these discussions all the time about what we see in the classroom

and how it’s [lack of professional behaviors] going to be difficult in terms of clinical

placements.”

Another team member shared an example of poor emotional intelligence which was causing a

student to be unsuccessful in the classroom:

“There’s one young lady that I have in my class who has a tendency to say anything that

comes to her mind (laughs). She now has a 10-second rule. She has to take 10 seconds

and think about what she has to say before she says it and she has to determine if her

comment is appropriate and if it is going to contribute to the conversation. Her outbursts

in class have reduced significantly.”

One team member shared her perspective of how important it is for healthcare

professionals to show initiative in order to be the best practitioners possible. She also shared

that this is something she relays to her students all the time, and it is a good example of a good

noncognitive behavior like responsibility:

140

“As a professional you are expected to be competent and self-directed in your area of

practice.….even to the point of being a lifelong learner, that’s one of the things that I

emphasize to my students a lot, you may learn something today, it may be obsolete

tomorrow, it may have changed tomorrow, you need to make sure you stay on top of

your game as a professional to know what’s new out there, and take the initiative to

learn about it, even if it’s not required. For example, to renew your professional license,

you have to have 30 continuing competency hours every two years, but that’s not to say

you can’t go beyond that, that you can pursue something else, or maybe what’s required

isn’t what you need.”

The team agreed that this same noncognitive behavior of taking responsibility and showing

initiative is also beneficial for all of us as healthcare educators and adds to our professional

development.

Healthcare educators belief these behaviors need measuring. One specific story told

by a team member was an example of why she felt soft skills should be measured and are

important in healthcare:

“We did scenarios yesterday on the campus. I was the wife of a patient who was in a

C.O.P.D. exacerbation and had been hospitalized. A little old man, 67 years old, and

we’d been married for 30 years, and I was the upset and crying wife. I had one set of

students (they were going in by two’s) and I had only one pair of students to offer me

support when I got upset thinking that my husband was going to die from this situation.

Only two students exhibited compassion for me as the upset wife!”

141

Another team member stated:

“I mean, I can see the administration’s desire to be consistent across the board, but I

hope that desire doesn’t mean that they’re going to get rid of our interview, because

there has to be and hopefully our research will show the best method; I’m going to have

to just put it out there. (Laughs) I’m biased toward the interview, I’m going to have to

say that, but I will say that if not the interview then there needs to be at least some way

to assess what I call soft skills, because they are so essential in health care.”

She also added:

“It’s something that I think adds to the validity of trying to determine if a student is

going to be successful in the program and more importantly be successful as a health

care provider because so much in what we do is noncognitive in nature so I really feel

like those are skills that we really need to assess as a part of the admission process and

to a health care program.”

All team members agreed that there needs to be some measure of noncognitive skills for these

students. One member added that it can be difficult to measure them.

“I myself have always felt that your noncognitive abilities should be part of the selection

process. Of course the challenge is always, always been, how do you measure

compassion? How do you measure critical thinking, empathy, communication, how do

you measure those?”

Student’s lack of awareness and dedication to their profession of choice. A very

evident theme emerged regarding the frustration of the group as a whole related to student’s

lack of knowledge about the healthcare profession and program they had made a decision to

apply to. Team members agreed that although numerous noncognitive behaviors for various

142

medical fields are the same, the actual profession itself is not. Team member’s discussions of

various job duties for each of the diverse professions revealed that they are extremely different

and there seems to be a theme at the college of students not researching their career choice, or

simply hopping from one program to another until they are accepted into one. One team

member stated:

“We’re having them observe, but our biggest problem I think with a new program is

that so many students have just tried to get into any program and so we get those that

really don’t care (laughs) and once they get into the program at some point, they are

probably going to be, ‘I don’t know if I really like this’ because that wasn’t their first

choice.”

Another team member echoed these sentiments by stating:

“But they just want to get into a program. So many people have said, I’ve tried to get

into the physical therapist assistant program, I’ve tried to get into the nursing program

or I thought about the radiography program or whatever. They’re just going to go down

the line until they finally get in somewhere.”

One team member shocked the team by telling of a student who got up during the middle of her

class and walked out. The student later told the team member that she got accepted into

another healthcare program that she preferred, so she just left. The team member was very

frustrated because this student had taken a “seat” in her program which could not be filled by

another student at that point.

Another theme evidenced by the group was the lack of knowledge regarding the

intensity of a healthcare program, or the details of what duties are expected of the student in

that particular healthcare field. Two separate team members both indicated they had issues

143

with former students who did not “realize they would have to touch an actual patient,” and

couldn’t stand the sight of blood or didn’t know they would actually see blood.

During one team meeting, one team member stated:

“They would say well, ‘You didn’t say it would be this stressful.’ Oh, we tell them that

the first day. We do. You need to get someone to help you with the kids and the family

and all those other variables that are going to distract you because this is tough.”

One team member mentioned the possibility that a disconnect between student and faculty may

be a reason the student doesn’t realize expectations of the program,

“Yeah, they think they can do it all, but I think there may be a misconception. As

faculty we know what they’re getting into, and the rigors of the program and this whole

thing. They really don’t, I don’t care how much you tell them, or how much you try to

prepare them, they don’t know.”

Interview can assess knowledge of profession. During discussion of the student’s lack

of awareness of the profession and expectations of the program, a theme emerged from the

team regarding the capability of an interview to assess this knowledge. Team members

discussed questions in the interview they felt could be asked in order to assess the student’s

awareness. One team member stated:

“That to me it is scary when they come in and we ask questions about: what they know

about the profession and they don’t have a clue because then you think, if you really

don’t know what you’re getting into, how are you going to be committed for two years

to this program?”

144

Another added: “In the interview, some of them answered, they didn’t have any [experience]

in healthcare. We kind of like to know if they’ve had some experience in healthcare so they

don’t get freaked out when they see blood and stuff. So we ask about it in our interview.”

Organizational Learning and Change

A final overarching theme which emerged during the action research process and in

response to research question number three, was organizational or system learning by focusing

on the development of a model for evaluation of impact and to predict student success. They

also had conversations regarding what skills they believed to be measured in an interview and

how this could potentially predict student success. Team members worked together to discuss

changes regarding the competitive process and how inconsistencies in administration has

affected the different processes at the college. Team members felt that these inconsistencies

have also affected retention and attrition rates within some of their programs which in turn have

affected the organization as a whole.

Noncognitive skills measured in an interview and success predicted. Throughout the

course of the meetings, the team members echoed their similar thoughts regarding specific

noncognitive skills that can be assessed during a face to face interview. Team members also

expressed concern and disappointment because sometimes they would witness a great candidate

in the interview, however knowing that the candidate’s grades weren’t high enough would likely

mean the candidate would not be accepted. One team member stated, “They may be the better

candidate, and that’s the sad thing.”

Some recurring behaviors mentioned by the team were communication, assertiveness,

personality, affect, appearance, professionalism, body language, eye contact, grammar, attitude,

145

stress tolerance, and commitment to the program. An example of a comment from a team

member is:

“I’ve had some come in, and they were just as bubbly and friendly, like, they were just

happy. And we observed that right here. And I have had others come in who didn’t even

smile…just a flat affect through the whole interview.”

Another team member added: “Attire and appearance; eye contact, when they’re looking at you,

and you can also listen to their grammar. All of that you can hear and see.”

Appearance was described by several team members as being a professional behavior that

could be measured in an interview. One team member stated:

“Observation was basically what we saw when they walked into that interview.

Remember, we talked about this one time where the attire that the student wore they

thought was acceptable based on their level of what’s considered appropriate professional

attire. But what we saw was the shirt un-tucked, not ironed or a disco dress that was too

short, and fingernail polish.”

Another added:

“Or blouses cut down too low. Khaki pants with tennis shoes. And for them they don’t

see that this is inappropriate attire for this sort of event. They figure, okay, I’m just

coming in to talk; I don’t have to worry about what I wear.”

Researcher notes reflect that team members were amazed to hear some of the other stories that

were told regarding how some students have presented themselves in interviews for a

professional medical program. It was another opportunity for the team to feel that this issue is a

problem for all of us and it was also beneficial for the team members who have not used

146

interviews to learn that these types of behaviors have happened and could possibly be potential

problems in their upcoming interviews of students.

One other team member described the ability to assess body language in an interview by

stating: “As well as body language…that’s another key thing we look at. That’s when we’re

grading them on their eye contact, attitude, smile; those that have the nervous giggles.” Another

team member discussed the ability to get a sense of whether or not a student will be

compassionate: “We want our profession to continue to be a caring profession and you can

definitely get a sense of that through these interviews.”

One last theme that emerged during the meetings and interviews with the team members

was the ability of an interview to predict the success of a student in the program. One team

member described it this way:

“I can already see that there’s a person in the program and I know if she had gone

through an interview process, she would not have gotten in. Yeah. Just because things

kept coming out, she’s very scattered and she’s gone through a million different

programs. She finished the cosmetology program but never did cosmetology. She did

that with probably four other programs. I’m like, if that had all come out in the interview

we would have been like, ‘Why is this now the program for you?’ Her GPA was good,

her HOBET was good, so she got in.”

Another team member discussed how she feels an interview would have helped her poor

retention rate, which also directly influences the college as well:

I’ve got 15 students left out of 22. I know by looking at this interview process there’s a

lot of them right off that I knew weren’t ready for my program … I mean, it’s amazing

because I’ve been doing this for 40 years. The 15 students that I have left would pass this

147

interview with flying colors. Of the six students that we’ve lost, if we would have had

those interview questions I bet you we wouldn’t have lost them because we wouldn’t

have admitted them. ”

One team member brought up how she feels the interview may also be beneficial for the student

who may not have the highest grades but who would make an excellent healthcare professional.

The interview may give a student who does not have a 4.0 GPA, an opportunity that they may

not have had if there was no interview component. She stated:

“I think on a flip side to that student whose barely making the grade, they meet the

criteria, but barely making the grade sometimes that interview or whatever tool we just

have to use, that interview can show us that they have the ability to be successful. Maybe

academically they struggle which could be for various reasons, but the interview could

give us that insight to say, well yes, they do have the potential to be successful, so I think

it works both ways.”

Facilitation of change. There were several themes that emerged related to the

facilitation of change for the team members throughout this process. This facilitation of change

is an indicator of the effectiveness of action research and group collaboration as well as impact

on the system. Numerous statements were made to point out changes that were being made or

implemented as a result of the project. For example,

“Early on when you had asked me to be on the committee, I had told you about our

retention and that we wanted to add another tool to assess the students. Yes, most

definitely the interview is going to be incorporated… so that we’re not getting a 4.0

student and just saying, this is going to be our best student. Now that you have done

this study and it is an option and it appears that administration is going to let us go

148

forward with this, I’m excited about it. I’m excited because now like I said 35% is

going to be measuring these soft skills.”

Prior to completion of this research, administration made the decision to allow all healthcare

programs to utilize an interview as part of their programmatic admissions process. It is not

known whether this research influenced that decision, but this team member’s comment implies

a perceived connection to this change in the organization.

After the assessments were analyzed and statistical correlations made, one team

member had this to say:

“Absolutely, we are going to make changes. Our clinical evaluation tool did prove to be

valid for the analysis part. Our interview questions need some work. We’ve gotten some

input from you as far as how to word those questions, how to fashion those questions to

get the information we need. As we speak, the program director and I are in the process

of tweaking our interview questions to make sure that we get the information that we

really need to make good decisions about who to accept into the program.”

Team researcher memos reflected this, which is another indicator of possible future changes:

“The biggest surprise for the team was the high correlation of the interviews in one

program and the extremely low correlations (or lack of) in another program. This led

certain team members to decide immediately that they were going to make changes in

their assessment tools, because according to the statistical data, their current assessment

tool was not valid! I also overheard comments being made by the team members asking

each other if they would mind sharing assessments and if they would help each other

with the revising of their assessments.”

149

One team member in particular who currently uses an interview as their measurement tool

stated this:

“I feel like in this particular research process helped me to identify some areas or some

interview questions that could be refined, removed, reworded, you know, those sorts of

things to make it a little bit better to make sure that it correlates with the skills that I’m

trying to identify. I think it helped in that regard. I feel now I could go back to my

interview questions and look at some things in terms of what’s good, what’s not so

good, what needs to be added, what needs to be deleted that sort of thing, and possibly

look at the EQi as an additional option for measuring noncognitive skills.”

Inconsistency in administration of admissions processes. Some action research team

members who were involved in this research learned that several programs were allowed by

administration to use an interview in their programmatic admissions and others were not. This

theme of inconsistency in administration was evident by numerous comments and statements

that were made by the team members during the meetings. One researcher memo reflected my

“nervousness” regarding possible problems that could arise from this information:

I stated to the team that currently there are only 2 programs that utilize an interview

component in the admissions process. This was ‘news’ to some team members, who

were unaware that any program still utilized interviews since they had been told by

administration that they could no longer use interviews. I have always gotten nervous

when discussing the status of interviews with the team because I was aware that some

programs could and were using an interview, however, others had been told they could

not. Apparently not all the team members were aware that some can use interviews and

others cannot. I worried that this would start a problem in the departments.

150

In the first action research team meeting, one team member stated:

“I was told we couldn’t do the interview. I didn’t know anybody else was doing the

interview. My advisory committee a year ago said, ‘let’s start doing the interviews.’ I

didn’t think that we were even allowed to, because they had quit doing … some of the

programs had quit doing interviews. I didn’t even think it was an option.”

Another team member added this statement in relation to administration, “They need to clarify

it. It’s not consistent across the board.” One team member who had recently started a new

healthcare program at the college had this to say:

“We weren’t allowed to use interviews in the program. Our program was basically told

that no other program would be allowed to use it at this time at least. Every time I ask

I’ve just been told, ‘well, they’re making some changes. Just hold off. You don’t really

need to know yet’.”

Throughout the course of the meetings, this theme of inconsistency in administration was

further evident by informal discussions the team had which were not related to the action

research project. Team members also discussed inconsistencies in some of the college policies

and procedures and their confusion around some of these issues.

During the research process, the team members gained knowledge related to healthcare

professions which enhanced their job growth along with their personal development. The

continuous improvement process of the functioning action research team allowed for team

members to acquire new knowledge from each other, which they can later use in their own

programs. Team members soon realized they were not aware that each dealt with this kind of

problem. This bonding of the team members led to professional development, on a personal as

well as group level, which is another example of the power of action research.

151

Conclusion

The research questions for the study were answered by mixed methods which comprised

both qualitative and quantitative date including a review of the literature, statistical correlation

testing on various assessment tools, and coded transcripts of team meetings, researcher memos,

and interviews of team members. The review of the literature produced information related to

interviews and emotional quotient inventory testing. An action research team worked together to

decide on an intervention option which was the decision to compare various assessment

instruments that measure noncognitive behaviors.

Major themes which emerged from the coded data were learning and changes at the

individual, group and system level. Team members discussed alternatives to an interview which

included pros and cons of an interview and the EQi instrument. Team members also enhanced

their professional development by sharing knowledge, stories, and information related to a

common issue they all face. Throughout the action research process, many new interesting

questions arose which will be discussed in Chapter 6 and could give rise to future studies and

research in this area. Implications of the study will also be discussed.

152

CHAPTER 6

CONCLUSIONS AND IMPLICATIONS

The purpose of this action research study was to explore alternative measurements of

noncognitive behaviors and characteristics essential for success in the healthcare field. Solely

relying on cognitive and objective type tests will not improve the selection of students for

medical programs (Cate & DeHaes, 2000; Salvatori, 2001). Reliance on these methods may

predict the academic success of a medical student, but how is the clinical success of a student

predicted? How can we as adult educators do a better job of supplying the professional field

with the most qualified and superior caregivers?

Three research questions were created and have guided this study; they are as follows:

(1a) What are the noncognitive behaviors and characteristics identified in the literature

for healthcare professions?

(1b) What are the noncognitive behaviors assessed during the interview component of a

competitive healthcare program?

(2) Is there a relationship between EQi scores and ratings on interviews, mid-term

assessments, or clinical evaluations?

(3) In what ways does an action research team process lead to individual, group, and

organizational learning and change?

This is an action research case study using both qualitative and quantitative methods.

An action research team of five members was created within the researcher’s institution

and team members worked collaboratively over almost a three year period to discuss alternatives

153

and intervention options regarding the assessment of noncognitive behaviors and characteristics.

Just as Herr and Anderson (2005) point out, action research is a process in which the action

research participants are in control of the design and methodology. As a medical professional

and instructor of programmatic healthcare courses, my research stems from an interest in how to

best avoid disruptive clinical behaviors of students in allied health programs. Yin (2009) states

that a case study investigator should have a strong grasp of the issues being investigated which

made my years as a healthcare professional and educator beneficial to the study. This was also

another benefit of an action research study because each team member had many years of

experience in both areas as well.

Summary of Findings

(1a) “What are the noncognitive behaviors and characteristics identified in the literature

for healthcare professions” (1b) “What are the noncognitive behaviors assessed during the

interview component of a competitive healthcare program?”

Data for research question one included an extensive review of the literature related to

these behaviors and traits. Findings from this study included a list of noncognitive behaviors

and traits believed to be important in the healthcare field. These behaviors and characteristics

were identified by analysis of team rating forms, and in the literature. Empathy was identified

more often in the literature than any other characteristic or trait (Benbasset & Baumal, 2007;

Hughes, 2002; Salvatori, 2001; Storey, 2008; Weege, 2009) however, team meetings and initial

data collection from healthcare professionals at this institution found that compassion was

mentioned the most and identified as being the most important trait for a medical professional.

Integrity was also a characteristic that was discussed by more than one author in the literature

(Hughes, 2002; Joyner, et al., 2007; Salvatori, 2001).

154

Of particular interest in this study, in response to “What are the noncognitive behaviors

assessed during the interview.” empathy was cited by the majority of studies, just as it was most

commonly listed in the literature as a necessary noncognitive trait for healthcare professionals.

In addition, most authors agreed that noncognitive behaviors and characteristics must be assessed

prior to students entering a medical program (Hughes, 2002; Salvatori, 2001). Although

controversial, the interview was cited as the most prevalent means of assessing noncognitive

behaviors (Edwards, et al., 2007).

Interestingly, during action research team meetings, team members felt that

professionalism, appearance, and body language were important traits or behaviors that could

only be measured in an interview; however, these were not as frequently mentioned in the

literature. Professionalism is similar to emotional maturity cited by Salvatori (2001) and Storey

(2008). Appearance and body language are similar to Kudlas (2006) who found that personal

characteristics are assessed during an interview component of a competitive healthcare program.

(2) Is there a relationship between EQi scores and ratings on interviews, mid-term

assessments, or clinical evaluations?

Correlational analyses were conducted to answer research question two. Results showed

that the EQi did in fact correlate with the clinical evaluation tool in one program and appeared to

measure skills not assessed in the interview in the other program. There was no positive

correlation with the EQi and interview in either of the two programs. Indeed, the only significant

relationship with the interview in one program was negative. This confirms that this particular

program has captured noncognitive skills and behaviors in their clinical rating. When both

programmatic assessments for each program were compared, the interview correlated with the

clinical evaluation in only one of the two programs. Additionally, in one of the programs, the

155

subcategory “adaptability” had a significant negative correlation with that program’s current

interview tool and no other correlations.

This negative relationship of the assessments came as a surprise to the researcher. The

EQi user’s manual (Bar-On, 1997) describes their adaptability subscale score as comprising

areas of problem solving, reality testing, and flexibility. This score helps to reveal how

successful someone is at coping with environmental demands and dealing with problematic

situations. High scores in this area identify people who are flexible, realistic, and effective in

understanding problematic situations and can arrive at an adequate solution. The manual also

asserts that people who score high in this area might do well in research and development as well

as technical support departments and are generally open to and tolerate different ideas, ways and

practices.

As the researcher and having a solid foundation regarding the job duties and

expectations of the profession of study in Group A, a speculation I have regarding this

correlation is that in Group A, the profession requires someone who has to follow specific

protocols and usually with no exceptions. This element of the job does not allow for different

ways or practices solely for the purpose of arriving at the highest level of patient care which

can be received. Someone in this particular profession does need to “think outside the box” but

only in minimal situations, while a particular set of protocols still must be followed.

Surprisingly though, Bar-On (1997) indicates that someone who scores high in this area would

do well in a technical support area and this particular program utilizes technology a great deal

in their profession. So, if a student scored high on the interview during the admissions process

to this program, why would they score low in an area that the EQi deems necessary for this

156

type of profession? Further research is needed to determine whether or not this relationship

holds up.

(3) In what ways does an action research team process lead to individual, group, and

organizational learning and change?

Qualitative data was collected from transcripts of action research team meetings,

researcher memos, and critical incident interviews of the team members. As a researcher in a

case study, the researcher must ask good questions; listen to what the participants have to say;

and be flexible and adaptable (Yin, 2009). In an effort to make sense of the data, team meetings,

researcher memos, and interviews were transcribed, consolidated, and inductively analyzed for

common themes and categories using the constant comparative method (Ruona, 2010). As

Ruona (2005) points out, qualitative research is about analyzing data by interpreting participant

meanings.

Team members worked together for almost three years to discuss what noncognitive

behaviors and characteristics are important in healthcare, specifically competitive healthcare

programs and explored various ways to measure these skills and traits. The team made a

collaborative decision to compare various assessments for the purpose of identifying the best

measure. Interview data, mid-semester/clinical data, along with data from the EQi were

compared. Reactions regarding the interview as the best method were strong, regardless of the

outcome of the statistical correlations.

Throughout the course of the research study, the process of action research allowed for

team members to learn and change at the individual, group, and organizational level. At the

individual level, change focused on working collaboratively by testing new and adapted

approaches to assessing noncognitive behaviors and characteristics. Team members displayed

157

a real enthusiasm to participate in the research and showed great initiative for the process by

contributing to the meetings in the form of researching articles, offering their students as

participants, de-coding information for the team, and assisting in the administration of the

assessments. Team members also learned about themselves when taking the EQi assessment as

well as learning about the assessment itself.

At the group level, team members focused on finding shared experiences and concerns

regarding the problem of how to measure these behaviors and characteristics. During these

meetings, team members discussed what they felt to be important and necessary behaviors and

characteristics required of a healthcare professional. As a group, they also discovered that

student’s lack of awareness or dedication to their profession of choice was a problem in most

all of the programs and the team members found they were all dealing with this in some

fashion. This gave rise to the opportunity for members to learn from each other how they

might be dealing with this problem in their own programs and how it is affecting each of them

in their respective roles something that does not happen across these disciplines in the ordinary

course of events. Thus the action research process introduced a cross-disciplinary professional

development opportunity targeted at a level of conversation that participants found useful.

At the organizational level, learning focused on developing a model for evaluating

impact and predicting student success which could ultimately affect the system as a whole.

One key point regarding system change was the decision that administration at the college

made to allow all healthcare programs to use an interview in their competitive process. The

interview questions and rubric collected by one action research team member were later used as

a guide for all programs deciding to create their own programmatic interview questions.

158

Team members also began to make changes in their program assessment tools in order

to refine their assessments and began to share ideas and tools with each other which then

created organizational changes. Team members shared experiences regarding how

inconsistency in administration can sometimes impact programs and student success. This

inconsistency in the system was an impetus for the original research questions and by studying

this problem, raising consciousness within the organization, gathering information, the

organization changed and its members learned.

Benefit of Action Research in One’s Own Organization

Processes resulting from the team member’s exploration of alternatives included:

participation, initiation, professional development and value of fellow team members’

experiences. Interactive research is another term for action research in which the relationship

between participants is emphasized as equals and a high degree of participation is stressed

(Coghlan & Brannick, 2010). Team members gained an appreciation of each other’s

backgrounds and realized they shared common problems and issues related to the lack of

noncognitive behaviors and characteristics in their students. They also were able to share an

understanding of the importance of certain types of noncognitive traits and skills in order to have

a successful student in their allied healthcare program.

At the completion of the study, team members were excited about the opportunity to have

worked together to explore alternative methods of assessing noncognitive behaviors; and gained

a sense of what some of these methods are. One team member commented by saying this

experience had given her an opportunity to recognize there are alternatives to an interview, and

this new knowledge would help her moving forward with her admissions processes. An example

159

of system impact from this study was the statement made by this team member that she would

look more closely at the EQi as a possible option for her admissions process.

Study Conclusions

Conclusions from this action research study center on the contrasting findings between

the two groups studied, and on the political ramifications of doing this study as an insider action

researcher. Both groups are future healthcare professions, however one group was accepted into

their program based on both cognitive and noncognitive assessments while the opposing group

was accepted based on cognitive measures only. Research closely related to this type of study

indicate mixed results in that some researchers did find correlations with noncognitive

assessment and student success, while others did not find correlations.

Political implications, which may be found in any type of research, seem to be more

prevalent in action research within the researcher’s organization (Coghlan & Shani, 2005). This

type of collaborative research relies on a system change which can be difficult to facilitate when

met with opposing forces such as administration and organizational culture and biases.

Multiple Assessments are needed in Healthcare to Capture Multiple Intelligences

Howard Gardner’s (2011) theory of multiple intelligences indicates that people are born

with multiple intelligences that predict how successful they will be in life. According to

Gardner, certain intelligences predict certain areas of success, for instance someone born with a

high “logistical/mathematical intelligence” may become a successful engineer, or someone born

with a high “musical intelligence” may become a successful composer. This theory formed the

basis for my study regarding the personal (or intra/inter personal) intelligences of a human being

and how these intelligences can predict the success of a healthcare student and professional. As

noted in my conceptual framework figure in Chapter One, if a student has strong personal

160

intelligences, then they will be more successful in healthcare, however, what is the best way to

assess these intelligences in order to accept the most qualified student for a healthcare program?

The operational framework, noted in Chapter Three, “interview ratings plus clinical

ratings plus EQi equals success in healthcare” became the basis for the work of the action

research team to determine a response to our initial research questions. Based on this research,

interviews from both programs did not correlate with the EQi which is a valid and reliable

assessment (Bar-On, 1997) and interviews did not significantly correlate with clinical

assessments in each program which could indicate success in clinic was not predicted by

interview data. These findings are similar to Conte (2005); Derksen et al. (2002); and Mayer et

al. (2000) who conducted correlational studies of the EQi with other “personality type” measures

and found that the two assessments were not measuring the same characteristics or skills.

On the other hand, the findings differ from Storey (2008) who found in her study that

interviews did in fact predict success of a student in a nursing program, and Hall, et al. (1992)

who found that interview scores of applicants to medical school had a positive correlation with

program completion and recommended the interview as a valid predictor of success. These

studies differ from this research in that neither of these authors incorporated the EQi assessment

in their correlations, which may have accounted for different results. Also, both authors looked

at program completion as their indicator for success which was not something this study used as

an indicator. Both groups of students were currently enrolled in the college and no research was

conducted on these students at the completion of their programs. If given more time for this

research, an additional research question could be framed around how scores from the various

assessments correlate with program completion of each student.

161

As an action research team, we learned that when assessing noncognitive behaviors and

characteristics, various assessments can be used to measure different variables. Our findings

indicate that the EQi could be used in addition to an interview which may be helpful for

assessing appearance, body language, and most importantly knowledge and dedication for the

profession. We also learned that just as Lewis, Rees, Hudson, and Bleakley (2005) point out, are

we attempting to measure what can be viewed as “un-measurable” based on this research and

others?

Perhaps what comes through strongest in this research is that, as the operational

framework indicated- it is the combination of interview ratings, EQi, and clinical performance

that sufficiently captures the complex capacities needed in modern healthcare settings. As we

reflected on our findings, it was clear that each assessment was capturing something unique. Just

as the HOBET assesses cognitive capacity, the EQi certain personal intelligences, and the

interview assesses professional fit; the clinical evaluations help healthcare educators gain a

preview of future performance of their students. We wonder if the better question provoked by

this research is not whether or not there is an alternative to the interview for measuring non-

cognitive behaviors and characteristics, but rather what array of assessments will best capture the

complex behaviors and characteristics, the multiple intelligences required of these students as

they enter the healthcare profession? Figure three illustrates a revision to the original conceptual

framework with additions from the operational framework as well as information from key

findings.

162

Figure 3. Conclusive findings figure representing multiple noncognitive behaviors and

characteristics need measuring by multiple assessments.

Gardner’s

Personal

Intelligences

(Inter/Intra)

Empathy

Integrity

Compassion

Honesty

Maturity

Self-

awareness

IF

Strong

…Then Successful

Performance

in Healthcare

Professions

Combination of

Assessments:

Interview Ratings

Clinical Ratings

EQi

163

Insider Action Research has Less System Impact when the Key Dilemmas Cannot be

Resolved or Managed

From the perspective of conducting action research in one’s own organization, various

dilemmas are faced by the action researcher. Coghlan and Shani (2005) discuss that action

researchers are confronted with the issue of role duality by having to deal with role expectations

as well as behaviors of the role senders. Role expectations are standards that are applied to the

behaviors of the focal person who occupies a given organizational role and sent-roles consist of

communications arising from role expectations and then are sent to the focal person in order to

influence behavior. During action research, the researcher has to be cognizant of their role in the

organization and recognize that the system may not have unified expectations of the action

research project which can create role ambiguities and in turn affect the research. In regards to

the dual role of action research, the researcher will need to be careful when making promises to

clients, need to balance their research-oriented actions with action-oriented activities, recognize

the audience when presenting themselves and realize that if an organizational action is headed

for a possible failure, a researcher may have to decide which priority is more important, quality

of the action or quality of the research (Coghlan & Brannick, 2010). In this study, there was a

bias toward the quality of the research at times over organizational action.

Action research comes with a set of political dynamics which must not be overlooked

during the research process. The researcher must be politically astute as they balance the role of

being active in the change process while intervening in the political and cultural systems by

influence and negotiation as needed. They must be prepared to work the political system, by

balancing what the organization wants from the research with their own personal reason for

political activity (Coghlan & Shani, 2005). This may not always be easy for the researcher as

164

they will need to continually assess the roles and influence that relevant stakeholders may have

in the project. Sponsorship of this project was initially provided by the individual whose

decisions initially problematized the interview. This made it difficult for the team to navigate

when they found evidence that contradicted the individual’s earlier decisions. The individual had

also influenced the selection of the team. In retrospect, the team may have ceded more authority

to the individual than the individual expected, but it is certainly true that their felt sense of the

individual prevented the team from pushing for further organizational change. Change occurred,

but almost subversively.

One last area of importance for the action researcher to recognize and deal with is

constraints around pre-learned biases on the part of the action researcher. The knowledge and

experiences of a person can apply to the theoretical understanding of the organization dynamics

as well as their own lived experiences in the organization (Coghlan, 2007). The researcher must

be careful not to make judgments and be open to the idea of re-framing their current way of

thinking. This is of particular importance to the action researcher when conducting action

research in your own organization and can be a significant challenge.

In this study, each of these dilemmas posed a challenge. As the researcher and an

employee of the organization, it was important for me to carefully maintain both roles as I

strived to complete the research. At the start of the research, I worked with the sponsor, my

Dean, to determine the course of action related to the study. She had specific expectations

regarding who I would invite to the action research team, as well as which programs would or

would not be participating. The Dean mentioned that she hoped we might find another

assessment to potentially use in place of an interview, due to the liability. My role as the

researcher was to determine the best method, regardless of the outcome.

165

The audience in this research seemed to be contradictory at times. The Dean preferred an

alternate to an interview, while the action research team collectively showed preferences towards

the use of an interview as the best measure of noncognitive behaviors and characteristics. As the

researcher, recognizing this was important because information discussed within the team

meetings would need to be balanced, and unfortunately at times, this was not the case.

As the primary researcher, as well as the director of one of the healthcare programs in the

organization, I had to deal with my own pre-learned biases related to the idea that an interview is

the only and best way to measure these behaviors and characteristics. I struggled with the idea of

re-framing my way of thinking due to the fact that I felt I had been able to recognize problematic

students in the interview process who later proved to be problematic in the classroom or clinical

setting. This was challenging for me as the researcher because these were “lived experiences” in

my own organization. Many of the team members struggled with the same challenge; they

expressed concerns that the interview would be the best measure due to their experience in the

admissions process and then later in the program.

Implications and Recommended Next Steps for the Organization

Findings produced by this study can be valuable to the program directors of the students

in Group A and Group B and the program faculty and instructors of each program. Clinical

instructors who are located at the clinical facilities where these healthcare students complete

their required clinical hours and training, program directors of the other healthcare programs at

the college, and finally the students themselves will also benefit from this project.

Administration of the college can also use the information gleaned in order to generate better

competitive admissions processes for future healthcare programs by creating better interview

questions and rubrics which could later add to the creation of better clinical assessment tools.

166

Administration can also determine if additional assessments, such as the EQi can or should be

used in the competitive process in order to enhance the process and determine the most

successful students for the program and the profession.

Recommendations for Further Study

Several recommendations could be considered based on this research and the results of

the study. This research was limited to a very small number of participants in one institution.

Future studies using a larger number of students as participants and across varying colleges or

institutions can be generalized with confidence. There were also financial constraints which

only allowed for the purchasing of a small number of EQi assessments, which also limited the

number of participants. If adequate funding were available it would be very beneficial to the

researcher to administer the EQi to more students in multiple healthcare programs or tracks.

This study was only conducted on two different programs which limits the diversity of

healthcare professions. Research on multiple professions would also be an advantage for future

research of competitive admissions processes for healthcare programs.

Because this study was completed in a relatively short period of time, over the course of

three years, future research could use a more longitudinal approach and acquire a larger set of

data over a longer period of time. This larger set of data could help strengthen the correlations

and help to determine why we didn’t getting stronger relationships especially between

admissions and clinical assessments. If the college is admitting who they consider to be the

“right” people for the program, why didn’t the EQi have a stronger correlation if it is a valid

and reliable tool for measuring high emotional intelligence? Is there something in our

pedagogy that could be looked at in future research?

167

An additional implication and finding for that the institution was the relationship

between interviews and performance was not clear possibly due to a lack of variance in the

clinical assessment measures. The interview and clinical assessments in the Group A program

ratings are very closely related and the questions are very much alike. This calls for the need to

repeat this analysis with more nuanced scoring of the performance measures or more data.

These clinical measures compared with the interviews are a key test of the validity of the

interview and any other metric, so there is a need for a way to get better data here.

We don’t know which of the assessments was more effectively tapping into

noncognitive behaviors and traits; and we also know the assessments do not relate to each other

very well. The EQi did not correlate with the interview from either program under study and

this correlation was of particular interest to the researcher and the team. Closer analysis of

team comments and data analyses indicates one possible reason for the lack of relationship

between the interviews and the EQi is the fact that the interviews focused more strongly on

professionalism, work ethics, and knowledge of the field which are constructs not measured by

the EQi. These are areas that could be assessed in addition to the use of the EQi if a program

chose to use the EQi as part of their admissions process.

One last area recommended for future research studies within the institution would be to

evaluate the emotional intelligence of students who are dismissed from allied health programs

for disruptive behaviors or characteristics, particularly in the clinical environment.

Correlations and comparisons can be made regarding their pre-admission data such as interview

scores along with programmatic data such as clinical evaluations while in the program. This

would also be a very interesting study if the results of the EQi for these particular students

could be assessed. Bar-On (1997) suggests retesting respondents; this is an excellent method of

168

scientifically examining changes that may occur in a student’s emotional skills over time.

Students who show obvious signs of disruptive behavior could be retested prior to the action of

dismissal and possible interventions could be put in place focused on developing noncognitive

skill in order to retain the student. This would be very beneficial to the institution and retention

rates.

Implications for Practice

The literature indicated that interviews are a subjective means (Cate & DeHaes, 2000)

nonetheless; they are used in most United States medical schools admissions processes

(Edwards, et al., 1990). The literature revealed enough evidence to validate a need for more

research surrounding competitive admissions processes, specifically in the area of effectiveness

of interviews. Within the literature, as well as in discussions among the action research team

members, a very strong bias towards the interview was evident. Yet the data suggests not all

interviews are equal, and some do not appear to measure emotional intelligence. Clearly given

the strong prevalence of and preference for the interview in allied health programs, future

directions are more likely of success if tied to improvements in the assessment of noncognitive

behaviors and traits within interviews and/or the use of the EQi as a supplement to admissions

processes rather than as a replacement for the interview.

Improvements in the admissions process can affect the success of the student within the

program, which can in turn affect the overall success of the program itself. A successful

program is beneficial for a number of reasons, one being accreditation. Accreditation agencies

hold programs accountable for meeting certain standards and benchmarks. Among other things,

this can be achieved by the success of a student, both in cognitive and noncognitive behaviors.

Successful programs are beneficial for the college as well as administration, for example

169

retention and completion. A high retention and completion rate for a healthcare program is a

financial benefit for the organization which bodes well for administration at the organization.

Given the large number of students who apply to healthcare programs and the small

number who are actually accepted, colleges or institutions need to identify the best candidate

for the program. One area of great importance to the team members who are healthcare

professionals as well as healthcare educators was described as “knowledge of the field.” This

included work ethics as it related to each profession and job duties. In this study, action

research team members felt that, in theory, students who have an awareness and understanding

of what the job requirements entail for the program they were applying to typically seem to be

more successful in the program and the profession once graduating. This awareness prior to

program admission would allow for a student to be somewhat aware of the upcoming

requirements and expectations of the program and clinical component. This would in turn

mean that a student would not be committing to a program for which they were not suited, and

then in turn, ultimately decide to withdraw from the program or be unsuccessful. Future

research may expand the literature by describing best ways to specifically measure a student’s

knowledge of what the profession entails. This information could in turn help to determine

future admissions plans and processes for numerous types of healthcare programs.

Leader of Change on the “Inside”

How has action research affected me becoming a leader of change in my own

organization and system? How have I evolved as a leader during this process? These are

questions I have reflected on throughout this process of action research within my own

organization. This research gave me and fellow action research team members the opportunity

to gain insight related to noncognitive behaviors and skills necessary in each of our own

170

healthcare programs. The purpose of action research is to assist people in extending their

understanding of their situation which can in turn assist in resolving problems that confront

them (Stringer, 2007).

This was an opportunity for me as well as the team to grow professionally and become

more familiar with each other. These new relationships will have an impact on the organization

because as members became closer and more comfortable with each other, there was a

noticeable shift in the dynamic of the team and team members began asking each other for

advice and recommendations regarding assessments. There was an obvious commitment to

learning by the team members which is a defining construct in the practice of action research

(Coghlan & Brannick, 2010) and a dedication within the team to explore the issue.

As Stringer (2007) states, community based action research results not only in a

collective vision but also functions by providing a sense of community and the role of the

researcher is more facilitative and less directive. This journey provided me with the

opportunity to “grow my skills” as a facilitative leader who work with a group of people to

determine the need for change and implement that change for the betterment of the college and

co-workers as well as future students. With this new information I can contribute to upcoming

procedures related to admissions processes.

I realized the impact I had on the organization when team members decided to make

changes to their own admissions processes and comments made by team members implied a

perceived association with the research and the fact that administration did make the decision to

allow all healthcare programs to use an interview. Another significant outcome of this project

which I initiated was the awareness generated about the problems in interviews and clinical

rating scales which team members identified from our collected data.

171

In order to be an effective leader of change and action research in your own

organization, one must balance research-oriented activities with action-oriented activities,

present yourself as a consultant with academic interests, and be ready to resolve conflict

between the qualities of the organizational action with the quality of research (Stringer, 2007).

In short, I learned how to balance my research needs with the organizational needs at hand

while facilitating a team of people who all had a vested interest in the process. I found myself

learning to remain open to changes and to the prospect of new knowledge of how to be a better

leader. These skills will be beneficial to me moving forward.

Conclusion

Ralph Waldo Emerson stated that “Character is higher than intellect…a great soul will

be strong to live, as well as to think,” and Vincent Van Gogh stated “Don’t forget that little

emotions are the great captains of our lives.” Both statements beautifully reflect the

importance of emotions and noncognitive behaviors or characteristics which are an important

element in the makeup of a healthcare provider. Competitive admissions processes for allied

health programs can be daunting and time consuming, however, programs can use the findings

from this study to begin to create better measures for assessing these necessary traits and skills

in order to improve the quality of patient care, which should be the ultimate goal of any

medical profession.

172

References

Akhouri, D., & Singh, A. R. (2010). Emotional intelligence. Indian Journal of Clinical

Psychology, 12, 170-180.

Albanese, M. A. (1999). Rating educational quality: Factors in the erosion of professional

standards. Academic Medicine, 74(6), 652-658.

Albanese, M. A., Snow, M. H., Skochelak, S. E., Huggett, K. N., & Farrell, P. M. (2003).

Assessing personal qualities in medical school admissions. Academic Medicine, 78(3),

313-322.

Bar-On, R. (1988). The development of a concept of psychological well-being (Doctoral

dissertation). Retrieved from http://eprints.ru.ac.za/1994/

Bar-On, R. (1997). EQ-I Bar-On emotional quotient inventory user’s manual. North Tonawanda,

NY: Multi-Health Systems, Inc.

Bar-On, R. (2006). The bar-on model of emotional-social intelligence (ESI). Retrieved from

http://www.ressourcesetmanagement.com

Bar-On, R. (2010, July 24). A new US air force study concludes that applying EI will save them

millions [Web log post]. Retrieved from http://www.reuvenBar-On.com

Baskerville, R. L. (1999). Investigating information systems with action research.

Communications of the association for information systems, 2(19), 2-32.

Benbassat, J., & Baumal, R. (2007). Uncertainties in the selection of applicants for medical

school. Advances in Health Science Education, 12, 509-521. doi: 10.1007/s10459-007-

9076-0

173

Birks, Y., McKendree, J., & Watt, I. (2009). Emotional intelligence and perceived stress in

healthcare students: A multi-institutional, multi-professional survey. BMC Medical

Education, 9(61), 1-8.

Brackett, M. A., & Salovey, P. (2006). Measuring emotional intelligence with the Mayer-

Salovery-Caruso emotional intelligence test. Psicothema, 18, 34-41.

Bugg, N., & Pounder, D. G. (1993). Gender-related biases in admission decisions. Radiologic

Technology, 64(4), 232-238.

Cadman, C., & Brewer, J. (2001). Emotional intelligence: A vital prerequisite for recruitment in

nursing. Journal of Nursing Management, 9, 321-324.

Carrothers, R. M., Gregory, S. W., & Gallagher, T. J. (2000). Measuring emotional intelligence

of medical school applicants. Academic Medicine, 75(5), 456-463.

Cate, J. T., & DeHaes, J.C. (2000). Summative assessment of medical students in the affective

domain. Medical Teacher, 22(1), 40-44.

Coghlan, D. (2006). Insider action research doctorates: Generating actionable knowledge. Higher

Education, 54, 293-306. Doi: 10.1007/s10734-005-5450-0

Coghlan, D., & Brannick, T. (2010). Doing action research in your own organization. Thousand

Oaks, CA: Sage Publications, Inc.

Coghlan, D., & Casey, M. (2001). Action research from the inside: Issues and challenges in

doing action research in your own hospital. Journal of Advanced Nursing, 35(5), 674-

682.

Conte, J. M. (2005). A review and critique of emotional intelligence measures. Journal of

Organizational Behavior, 26, 433-440. doi: 10.1002/job.319

174

Creswell, J.W. (2003). Research design: Qualitative, quantitative, and mixed methods

approaches. Thousand Oaks, CA: Sage Publications, Inc.

Dawda, D., & Hart, S. D. (2000). Assessing emotional intelligence: Reliability and validity of

the Bar-On emotional quotient inventory (EQi) in university students. Personality and

Individual Differences 28(4), 797-812.

Derksen, J., Kramer, I., & Katzko, M. (2002). Does a self-report measure for emotional

intelligence assess something different than general intelligence? Personality and

Individual Differences 32(2002), 37-48.

DeWeerdt, M., & Rossi, G. (2012). The Bar-On emotional quotient inventory (EQi): Evaluation

of psychometric aspects in the dutch speaking part of Belgium. doi: 10.5772/38875

Edwards, J. C., Johnson, E. K., & Molidor, J. B. (1990). The interview in the admission process.

Academic Medicine, 65(1990), 167-177.

Ehrenfeld, M., & Tabak, N. (2000). Value of admission interviews in selecting of undergraduate

nursing students. Journal of Nursing Management, 8(2), 101-106. Retrieved from

http://wfxsearchgalileo.webfeat.org/wfsearch/search

Ellinger, A. D., & Watkins, K. E. (1998). Updating the critical incident technique after forth-

four years. Published paper.

Ephross, P. E., & Vassil, T. V. (2005). Groups that work: Structure and process. Chichester,

NY: Columbia University Press.

Espen, D., Wright, D. L., & Killion, J. (2006). Admission requirements for radiography

programs. Radiologic Technology, 77(5), 366-372.

175

Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004). An admissions OSCE: The

multiple mini-interview. Medical Education, 38, 314-326. doi: 10.1046/j.1365-

2923.2004.01776.x

Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327-358.

Gardner, H. (2011). Frames of mind the theory of multiple intelligences, New York, NY: Basic

Books.

Goho, J., & Blackman, A. (2006). The effectiveness of academic admission interviews: An

exploratory meta-analysis. Medical Teacher, 28(4), 335-340. doi:

10.1080/01421590600603418

Goleman, D. (1995). Emotional intelligence why it can matter more than IQ. New York, NY:

Bantam Dell.

Grubb. W. L., & McDaniel, M. A. (2007). The fakability of bar-on’s emotional quotient

inventory short form: Catch me if you can. Human Performance, 20(1), 43-59.

Hall, F. R., Regan-Smith, M., & Tivnan, T. (1992). Relationship of medical students’ admission

interview scores to their dean’s letter ratings. Academic Medicine, 67(1992), 842-845.

Herr, K., & Anderson, G. L. (2005). The action research dissertation. Thousand Oaks,

California: Sage Publications.

Hopkins, W.G. (2000). Quantitative research design. Sportscience, 4(1), 12-21.

Hughes, P. (2002). Can we improve on how we select medical students? Journal of the Royal

Society of Medicine, 95, 18-22.

Johnson, E. K., & Edwards, J. C. (1991). Current practices in admission interviews at u.s.

medical schools. Academic Medicine, 66(7), 375-427.

176

Joyner, P. U., Cox, W. C., White-Harris, C., & Blalock, S. J. (2007). The structured interview

and interviewer training in the admissions process. American Journal of Pharmaceutical

Education, 71(5), 1-5.

Kluemper, D. H. (2006). An examination of ability-based emotional intelligence in the structured

employment interview (Doctoral Dissertation). Oklahoma State University, Stillwater,

Oklahoma.

Kudlas, M. (2006). Effects of radiography program admissions practices on student retention.

Journal of Allied Health, 35(3), 162-168.

Kumar, K., Roberts, C., Rothnie, I., du Fresne, C., & Walton, M. (2009). Experiences of the

multiple mini-interview: a qualitative analysis. Medical Education, 43(4), 360-368.

Retrieved from: http://wfxsearchgalileo.webfeat.org/wfsearch/search

Lemay, J., Lockeyer, J. M., Collin, V. T., & Brownell, A. K. (2007). Assessment of noncognitive

traits through the admissions multiple mini-interview. Medical Education, 41, 573-579.

doi: 10.1111/j.l365-2923.2007.02767.x

Lewis, N. J., Rees, C. E., Hudson, J. N., & Bleakley, A. (2005). Emotional intelligence in

medical education: Measuring the unmeasurable? Advances in Health Science Education

Theory and Practice, 10(4), 339-355.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Thousand Oaks, California: Sage

Publications, Inc.

Mayer, J. D., Caruso, D. R., & Salovey, P. (2000). Selecting a measure of emotional intelligence.

In R. Bar-On & J. Parker (Eds.), The handbook of emotional intelligence: Theory,

development, assessment, and application at home, school, and in the workplace (pp.

320-342). San Francisco: Jossey-Bass.

177

Merriam, S. B. (2009). Qualitative research a guide to design and implementation. San

Francisco, CA: Jossey-Bass.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage

Publications, Inc.

Noonan, B. M., Sedlacek, W. E., & Veerasamy, S. (2005). Employing noncognitive variables in

admitting and advising community college students. Community College Journal of

Research and Practice, 29, 463-469. doi: 10.1080/10668920590934170

O’Boyle Jr., E. H., Humphrey, R. H., Pollack, J. M., Hawver, T. H., & Storey, P. A. (2010). The

relation between emotional intelligence and job performance: A meta-analysis. Journal of

Organizational Behavior. Advance online publication. doi:10.1002/job.714

Ochs, L. L., & Adams, R. D. (2008). The admissions process and student success in radiation

therapy education. Radiation Therapist, 17(2), 85-88.

Pritchard, T. L. (2010). COMPASS and prerequisite course scores as predictors of success in

practical nursing school. (Unpublished doctoral dissertation). Walden University,

Minnesota.

Rothstein, M. G., & Goffin, R. D. (2006). The use of personality measures in personnel

selection: What does current research support? Human Resource Management Review,

16, 155-180.

Roulston, K. (2010). Reflective interviewing a guide to theory and practice. London, EC1Y:

Sage Publications, Ltd.

Ruona, W. E. A. (2005). Analyzing qualitative data. In R. A. Swanson & E. F. Holton (Eds.),

Research in organizations: Foundations and methods of inquiry (pp. 223-263). San

Francisco, CA: Berrett-Koehler.

178

Salvatori, P. (2001). Reliability and validity of admissions tools used to select students for the

health professions. Advances in Health Science Education, 6(2), 159-175.

Salovey, P., & Mayer, J. D. (1990). Emotional intelligence. Imagination, Cognitioin and

Personality, 9(3), 185-211.

Saxon, D. P., Levine-Brown, P., & Boylan, H. R. (2008). Affective assessment for

developmental students, part 1. Research in Developmental Education, 22(1), 1-4.

Schwarz, R. (2002). The skilled facilitator: A comprehensive resource for consultants,

facilitators, managers, trainers, and coaches. San Francisco, CA: Jossey-Bass.

Scott, A. H., Chase, L. M., Lefkowitz, R., Morton-Rias, D., Chambers, C., Joe, J., Holmes, G., &

Bloomberg, S. (1995). A national survey of admissions criteria and processes in selected

allied health professions. Journal of Allied Health, 24(2), 95-107.

Shani (Rami), A. B., & Pasmore, W. A. (1985). Contemporary organization development:

Current thinking and applications. Glenview, IL: Scott, Foresman.

Stansfield, R. B., & Kreiter, C. D. (2007). Conditional reliability of admissions interview ratings:

Extreme ratings are the most informative. Medical Education, 41, 32-38. doi:

10.1111/j.1365-2929.2006.02634.x

Storey, L. M. (2008). The effect of structured interviews on predicting the success of associate

degree nursing students (Doctoral dissertation). University of Georgia, Athens, Georgia.

Stringer, E. T. (2007). Action research. Thousand Oaks, CA: Sage Publications, Inc.

Timmins, R. P. (2006). How does emotional intelligence fit into the paradigm of veterinary

medical education? Journal of Veterinary Medical Education, 33(1), 71-75.

179

Tran, T., & Blackman, M. C. (2006). The dynamics and validity of the group selection interview.

The Journal of Social Psychology, 146(2), 183-201. Retrieved from

http://wfxsearchgalileo.webfeat.org/wfsearch/search

Weber, M. D., & Karman, T. A. (1991). Student group approach to teaching using Tuckman

model of group development. Advances in Physiology Education, 261(12), 12-17.

Weege, M. R. (2009). Predictors of success in applicants to the radiation therapy program at

university of Wisconsin-la crosse. (Master’s thesis, University of Wisconsin-Stout).

Retrieved from http:// http://minds.wisconsin.edu/handle/1793/43381

Wilson, S. C., & Carryer, J. (2008). Emotional competence and nursing education: A New

Zealand study. Nursing Praxis in New Zealand, 24(1), 36-47.

Wright, S. S., & Miederhoff, P. A. (1999). Selecting students with personal characteristics

relevant to pharmaceutical care. American Journal of Pharmaceutical Education, 63,

132-138.

Yin, R. K. (2009). Case study research: Design and methods. Thousand Oaks, CA: Sage

Publications, Inc.

Young, T. A. (1997). Teaching medical students to lie, the disturbing contradiction: Medical

ideals and the resident-selection process. Canadian Medical Association, 156(2), 219-

222.

180

APPENDIX A

CONSENT FORM Measurement of Noncognitive Skills in Competitive Programmatic Admissions Processes to

Healthcare Programs

I, __________________________________________, agree to participate in a research study

titled “Measurement of Noncognitive Skills in Competitive Programmatic Admissions Processes

to Healthcare Programs” conducted by Deanne Dotson Collins, MSRS, R.T.(R)(ARRT), a

doctoral student in the University of Georgia’s Adult Education Program, and supervised by Dr.

Karen Watkins, Professor in the University of Georgia’s Department of Lifelong Education,

Administration, and Policy, Adult Education Program (706-542-4355).

I understand that my participation is voluntary. I can refuse to participate or stop taking part

without giving any reason and without penalty or loss of benefits to which I am otherwise

entitled. I can ask to have all of the information about me or provided by me returned to me,

removed from the research records, or destroyed. I understand that an expected duration of

participation would be, at most, until May, 2013.

Purpose of the Study: I understand that the purpose of this study is (a) to determine if there is a

need for the measurement of noncognitive skills in competitive admissions processes to

healthcare programs (b) determine the best method of measuring these noncognitive skills for

admission into a healthcare program and for the success of the student both clinically and

academically.

Procedures: If I choose to participate, I will be asked to do the following things:

Participate in an action research team relating to measurement of noncognitive skills in

competitive admissions processes to healthcare programs

Participate in several meetings conducted with the researcher and other members of an

action research team at an agreed upon location and time to discuss what tasks to

undertake and identify and collect existing data in order to implement some strategy

pertinent to the research

Benefits: The benefits for me are knowledge, support, and learning regarding what constitutes

noncognitive skills in the medical profession as it relates to a competitive admissions process.

The researcher also hopes to determine a valid measurement tool for use in competitive

admissions processes for healthcare programs at a local technical college in the state of Georgia.

Confidentiality: I agree to have the meetings and discussions audio recorded for transcription

purposes, in order to maintain accuracy and clarity in the conversations. No discomforts or

stresses are expected as a result of these meetings. No risks are expected. No individually-

identifiable information about me, or provided by me during the research, will be shared with

others without my written permission. AUDIO TAPES WILL BE STORED IN

RESEARCHER’S LOCKED HOME OFFICE, SEPARATELY AND CONFIDENTIALLY,

FOR USE BY THE RESEARCH TEAM. AUDIO RECORDINGS WILL NOT BE PUBLICLY

181

DISSEMINATED, AND WILL BE DESTROYED PROPERLY AT THE CONCLUSION OF

THE STUDY.

I understand that the researcher or her doctoral supervisor will answer any further questions

about this research, now or during the course of the project (Deanne Dotson Collins, at

ddcollins7@gmail.com or Dr. Watkins at 706-542-4355 or kwatkins@uga.edu).

I understand that I am agreeing by my signature on this form to take part in this research project

and understand that I will receive a signed copy of this consent form for my records.

_________________________________ ____________________________ ____________

Name of Participant Signature Date

_________________________________ ____________________________ ____________

Name of Researcher Signature Date

Telephone: _________________________ Email: _____________________________

Please sign both copies, keep one and return one to the researcher.

Additional questions or problems regarding your rights as a research participant should be addressed to The

Chairperson, Institutional Review Board, University of Georgia, 629 Boyd Graduate Studies Research Center,

Athens, Georgia, 30602-7411; Telephone (706) 542-3199; Email address: IRB@uga.edu

182

APPENDIX B

Critical Incident Interview Questions

Action Research Project

Deanne D. Collins

Telling Your Story

“The purpose of this interview is to identify and explore key informal and incidental learning

experiences regarding the Action Research process.”

1. When you think about what you have experienced in the action research process, in regards to

the problem of assessing noncognitive skills and possible alternatives to the interview, what was

the most significant incident or turning point in the project? Describe for me:

The context

What stage of the project this occurred

What happened

Who was involved

What you and others said or did

What it was about this incident that made it critical

2. Thinking back over our work together, how do you now view the way your particular program

assesses these skills and are there any changes you plan to make?

3. How was this learning experience important to your career as a healthcare educator?

4. When you think about how the action research team has functioned together, can you tell me

about a particular time when the AR team seemed to work best? Describe for me:

The context

What stage of the project this occurred

What happened

Who was involved

What you and others said or did

What it was about this incident that made it critical

5. Is there anything else that you would like to add regarding what you have learned in this

journey with others?

183

APPENDIX C

Measurement of Noncognitive Skills in Competitive Programmatic Admissions

Processes to Healthcare Programs

Project # 2012-10059-0

Verbal script to be used by Deanne Dotson Collins, MSRS, R.T.(R)(ARRT), a doctoral student in

the University of Georgia’s Adult Education Program, and supervised by Dr. Karen Watkins,

Professor in the University of Georgia’s Department of Lifelong Education, Administration, and

Policy, Adult Education Program (706-542-4355).

“Hello, my name is Deanne Collins and I am a doctoral student at the University of Georgia. My

major professor is Dr. Karen Watkins and she is supervising me in this research project. She can

be contacted at kwatkins@uga.edu if needed.

The purpose of my research is to assess the effectiveness of a noncognitive measurement tool

in a competitive admissions process for allied health programs.

I will be administering an emotional quotient inventory assessment online which consists of statements that provide you with an opportunity to describe yourself by indicating the degree, to which each statement is true of the way you feel, think, or act most of the time and in most situations.

If you are 18 years of age or older, you are eligible to participate in this research study.

I would like to invite you to participate in this voluntary study. If you would not like to

participate, the instructor will have an alternate activity for you, while the voluntary

participants complete the online assessment.

Thank you for your consideration.”

184

APPENDIX D

CONSENT FORM Measurement of Noncognitive Skills in Competitive Programmatic Admissions Processes to

Healthcare Programs

I, __________________________________________, agree to participate in a research study

titled “Measurement of Noncognitive Skills in Competitive Programmatic Admissions Processes

to Healthcare Programs” conducted by Deanne Dotson Collins, MSRS, R.T.(R)(ARRT), a

doctoral student in the University of Georgia’s Adult Education Program, and supervised by Dr.

Karen Watkins, Professor in the University of Georgia’s Department of Lifelong Education,

Administration, and Policy, Adult Education Program (706-542-4355).

I understand that my participation is voluntary. I must be at least 18 years old to participate. I

can refuse to participate or stop taking part without giving any reason and without penalty or loss

of benefits to which I am otherwise entitled. I can ask to have all of the identifiable information

about me or provided by me returned to me, removed from the research records, or destroyed. I

understand that an expected duration of participation would be, at most, 2 hours.

Purpose of the Study: I understand that the purpose of this study is (a) to determine if there is a

need for the measurement of noncognitive skills in competitive admissions processes to

healthcare programs (b) determine the best method of measuring these noncognitive skills for

admission into a healthcare program and for the success of the student both clinically and

academically.

Procedures: If I choose to participate, I will be asked to take an online emotional quotient

inventory assessment titled the EQi ®.

Benefits: The benefits for me are knowledge, support, and learning regarding what constitutes

noncognitive skills in the medical profession as it relates to a competitive admissions process.

The researcher also hopes to determine a valid measurement tool for use in competitive

admissions processes for healthcare programs at a local technical college in the state of Georgia.

Anonymity: The results of participation will be anonymous.

Risks/Discomforts: No more than minimal discomforts or stresses are expected as a result of this

assessment. You will be asked to answer questions regarding your emotions and personality.

Some participants may feel slightly uncomfortable answering questions regarding emotions and

personality. Therefore, participants will be able to skip any questions they do not want to answer.

I understand that the researcher or her doctoral supervisor will answer any further questions

about this research, now or during the course of the project (Deanne Dotson Collins, at

ddcollins7@gmail.com or Dr. Watkins at 706-542-4355 or kwatkins@uga.edu).

I understand that I am agreeing by my signature on this form to take part in this research project

and understand that I will receive a signed copy of this consent form for my records.

185

_________________________________ ____________________________ ____________

Name of Participant Signature Date

_________________________________ ____________________________ ____________

Name of Researcher Signature Date

Telephone: _________________________ Email: _____________________________

Please sign both copies, keep one and return one to the researcher.

Additional questions or problems regarding your rights as a research participant should be addressed to The

Chairperson, Institutional Review Board, University of Georgia, 629 Boyd Graduate Studies Research Center,

Athens, Georgia, 30602-7411; Telephone (706) 542-3199; Email address: IRB@uga.edu