1 hiring fast and slow: lens of bounded rationalitym044c9256/fulltext.pdf · 1 hiring fast and...
TRANSCRIPT
1
HIRING FAST AND SLOW:
A QUALITATIVE STUDY OF TEACHER HIRING PRACTICES VIEWED THROUGH THE
LENS OF BOUNDED RATIONALITY
A doctoral thesis by
Michael Hinman
To
The School of Education
In fulfillment of the requirements for the degree of
Doctor of Education
in the field of
Education
College of Professional Studies
Northeastern University
Boston, Massachusetts
March 2019
2
Abstract
This examination of the hiring practices of K-8 educational leaders in a particular county in a
mid-Atlantic state is viewed through the theoretical framework of bounded rationality, which
holds that rationality is limited by the availability of information, time, and mental capacity. In
this study, a review of the literature explored the applicability of bounded rationality to the work
of school principals, what is known about the decision-making processes by experts in a number
of fields, and what is known about the decision-making process involved in teacher hiring. Of all
the decisions made by school principals, the hiring of teachers may be the most important task;
nevertheless, the bounded rationality involved in decision-making unchecked by the supports of
protocols such as research supported screening criteria or interview procedures engenders an
ongoing problem of practice: sub-optimal decision-making in the process of teacher hiring. This
research examines the question: How does bounded rationality affect administrators’ decision-
making processes in the process of hiring new teachers among comparable school districts in the
mid-Atlantic region? There were eight findings identified in this research: bounded rationality
affects the decision-making of K-8 educational leaders; the teacher characteristics listed as most
important were not consistent among all principals; fast thinking through biases and heuristics
including fit, missing data, overconfidence were identified; slow thinking through the use of
protocols, question sets and self-awareness were also identified; and principals stated the
addition of a decision-making tool, such as a checklist, might improve their processes.
Key terms: biases and heuristics, bounded rationality, decision-making, and teacher
hiring.
3
Acknowledgements
My family. Through my mother’s limitless belief that I was the cleverest of all the Michaels and
my father’s checks on my tendency to believe her, I generated the level of imposter syndrome
that motivates one toward educational pursuits. From my father, I learned to consider my words
and to see a problem, go to a hardware store, and fix it. From my mother, I learned very early
that an audience, even of one, is all the creation one really needs. Well, that and maybe music.
And something to eat, of course. Do you want a sandwich? My brother, two years and two light
years ahead of me in all pursuits, also fed my desire toward mastery, but I realized that there
were decades required for us mere mortals to come close to his example. It was actually my
sister’s gifts of the best sellers Thinking Fast and Slow and The Checklist Manifesto that sparked
my dissertation inquiry. She has always been the model to live life as our best selves, and in
doing so, she is the best of us. Ask anyone. If you met my children, you would understand I will
need a whole heap of education to keep up with them. Talented, driven, and quick-witted (the
ultimate sign of intellect in our house), they make me want to be, and are each examples of how
to be, a better person. For the three of us, truly, it is my wife who is the team’s consistent
manager, coach, and entire front office. All of our accomplishments were largely due to her
unwavering and relentless support of our best interests, spending more time considering and
solving our problems than we did. The decades of that workload was compounded by the
addition of my graduate work pursuits. And how did she react? If you don’t know the answer,
you aren’t following along here. Sue is an efficacy and achievement expert. Here is just one
example: she once suggested we run a marathon; five years later, with me dragging along
through seven, she had run thirteen, finishing by qualifying for and running Boston. For the
better part of my life, I have been impressed by and drawn to her determination and success. The
completion of this project, and all that is good in my life, is owed to her.
4
The participants in my study. Had these principals not volunteered their time, this study and
the years of its preparation would have been lost. As a researcher, I sought to avoid evaluative
thinking, but could not help but leave each interview impressed by these dedicated educational
leaders, their knowledge, and their respect for their students, staff, and communities.
The educators I have known. Much of who I am and where I am can be directly sourced from
the individuals that have been my teachers since September 1970, in which my kindergarten
teacher, Mrs. Maxim, was tasked with the unenviable job to curtail my early first attempts to
organize opposition groups. I believe she retired at the end of that year. My love/hate
relationship with the American education system presented a mixed bag for those adults I met as
a student, as a teacher and administrator, and as a faculty colleague. Daily, I have access to
outstanding educators from those in my office, Betty, Hope, and Alison, to those in each office
and classroom around the 90 square miles of the school district. These professionals face
herculean tasks with great joy and skill. Stop in and you might find yourself welling up as you
witness the most caring and touching of moments. To name individuals would be to miss
innumerous others. But risking that, I would like to thank a few, now doctoral, colleagues: the
late Dr. Jeanne Carlson for dedicating her life in support of gifted education and for her
mentorship; Dr. Annette Giaquinto for being the model of moral and rational leadership; Dr.
Pamela Vaughan for her years of guidance in the field; Dr. Chris Unger for inspiring the
improvement of systems for those they serve; and my advisor, Dr. Peg Dougherty, for her
specific and clear direction through this writing process.
5
Table of Contents
Table of Contents .......................................................................................................................... 5
Chapter I: Introduction .............................................................................................................. 10
Statement of the Problem .......................................................................................................... 10
Significance of the Problem ...................................................................................................... 13
Positionality Statement .............................................................................................................. 15
Research Question ..................................................................................................................... 17
Theoretical Framework ............................................................................................................. 18
Chapter II: Literature Review ................................................................................................... 22
Bounded Rationality .................................................................................................................. 22
Identified biases and heuristics. ............................................................................................. 23
Bounded rationality’s effect on decision-making. ................................................................. 25
Critics of the framework. ....................................................................................................... 28
Decision-making Tools ............................................................................................................. 29
Decision-making tools used in business. ............................................................................... 30
Decision-making tools used in medicine. .............................................................................. 32
Decision-making tools used in education. ............................................................................. 34
Subtle decision-making influence. ........................................................................................ 35
Summary. ............................................................................................................................... 37
Hiring Practices ......................................................................................................................... 39
Screening. .............................................................................................................................. 39
Interviewing. .......................................................................................................................... 41
Conclusion ................................................................................................................................. 45
Chapter III: Research Design ................................................................................................... 47
Methodology ............................................................................................................................. 47
Research Design ........................................................................................................................ 47
Research Tradition .................................................................................................................... 49
Participants ................................................................................................................................ 51
Recruitment and Access ............................................................................................................ 52
Data Collection .......................................................................................................................... 54
Data set 1 – Survey of principals (Appendix D). .................................................................. 55
Data set 2– Principal interviews (Appendix E). .................................................................... 55
Paper review. ......................................................................................................................... 56
Candidate interviews. ............................................................................................................ 56
General processes. ................................................................................................................. 57
Data set 3 – Focus Group (Appendix F). ............................................................................... 57
Data Storage .............................................................................................................................. 58
Data Analysis ............................................................................................................................ 59
6
Table 1 ....................................................................................................................................... 60
Table 2 ....................................................................................................................................... 61
Data set 1: Survey ..................................................................................................................... 61
Table 3 ....................................................................................................................................... 62
Table 4 ....................................................................................................................................... 63
Data Set 2: Interviews ............................................................................................................... 64
Table 5 ....................................................................................................................................... 67
Data set 2 - Interviews first cycle coding .................................................................................. 69
Data set 2 - Interviews second cycle coding ............................................................................. 70
Data Set 3: Focus Group ........................................................................................................... 71
Data Set 3 – focus group cycle 1 coding ................................................................................... 72
Data Set 3 – focus group cycle 2 coding ................................................................................... 72
Table 6 ....................................................................................................................................... 72
Summary ................................................................................................................................... 73
Trustworthiness ......................................................................................................................... 73
Credibility. ............................................................................................................................. 74
Transferability. ...................................................................................................................... 74
Dependability......................................................................................................................... 74
Confirmability. ...................................................................................................................... 74
Chapter IV: Findings and Analysis ........................................................................................... 76
Introduction ............................................................................................................................... 76
Review of Research Problem .................................................................................................... 76
Table 7 ....................................................................................................................................... 78
Review of Research Question ................................................................................................... 79
Data Collection .......................................................................................................................... 80
Data Analysis ............................................................................................................................ 81
Table 8 ....................................................................................................................................... 82
Analysis of Survey Responses .................................................................................................. 82
Participant experience. ........................................................................................................... 83
Applicant data valuation. ....................................................................................................... 83
Table 9 ....................................................................................................................................... 83
Committee use. ...................................................................................................................... 83
Table 10 ..................................................................................................................................... 84
Protocols. ............................................................................................................................... 84
Survey Summary. .................................................................................................................. 84
Hiring rates............................................................................................................................ 85
Data valuation. ...................................................................................................................... 85
Committee use........................................................................................................................ 85
Protocols. ............................................................................................................................... 86
Analysis of Data from 1:1 Interviews ....................................................................................... 86
7
Table 11 ..................................................................................................................................... 86
Identified Fast Thinking. ....................................................................................................... 88
Expertise/confidence. ............................................................................................................. 88
Intuition/gut/fit. ...................................................................................................................... 88
Time constraints. ................................................................................................................... 88
Likeability/background of candidate similar to administrator. ............................................. 88
Identified Slow Thinking. ...................................................................................................... 89
Protocols. ............................................................................................................................... 89
Cross checking. ...................................................................................................................... 89
Self-reflection......................................................................................................................... 89
Second Cycle ............................................................................................................................. 89
In vivo coding of transcripts...................................................................................................... 89
Expertise. ............................................................................................................................... 89
Confidence. ............................................................................................................................ 90
Intuition/gut. .......................................................................................................................... 90
Time Constraint. .................................................................................................................... 90
Likeability/Similarity to the administrator. ........................................................................... 90
Table 12 ..................................................................................................................................... 91
Summary of interviews. ......................................................................................................... 93
Analysis of Focus Group Data .................................................................................................. 95
Table 13 ..................................................................................................................................... 96
Cycle 1 Pattern matching coding. ............................................................................................. 97
Administrator Expertise/Confidence. .................................................................................... 97
Confirmation Bias/Likeability/Beliefs Regarding Characteristics of Successful Teachers. . 97
Fit........................................................................................................................................... 97
Cycle 2 In vivo coding. ............................................................................................................. 97
Prediction Overconfidence - Administrator Expertise/Confidence. ...................................... 97
Gut/Intuition. ......................................................................................................................... 98
Confirmation Bias/Beliefs Regarding Characteristics of Successful Teachers/Fit. .............. 98
Summary of focus group analysis. ........................................................................................ 99
Table 14 ..................................................................................................................................... 99
Emergent Themes from the Three Data Sets .......................................................................... 100
Identified fast thinking......................................................................................................... 101
Affect bias/likeability. .......................................................................................................... 101
Confirmation bias/beliefs regarding characteristics of successful teachers. ...................... 101
Fit......................................................................................................................................... 102
Missing data bias. ................................................................................................................ 102
Identified slow thinking. ...................................................................................................... 103
Self-awareness. .................................................................................................................... 103
Table 14 ................................................................................................................................... 104
8
Data Triangulation................................................................................................................... 105
Trustworthiness ....................................................................................................................... 106
Credibility. ........................................................................................................................... 106
Dependability....................................................................................................................... 107
Confirmability. .................................................................................................................... 107
Summary of Findings .............................................................................................................. 107
Finding one: ......................................................................................................................... 107
Identified fast thinking. ........................................................................................................... 108
Finding two: ......................................................................................................................... 108
Finding three: ....................................................................................................................... 108
Finding four: ........................................................................................................................ 108
Finding five: ........................................................................................................................ 108
Identified slow thinking. ......................................................................................................... 108
Finding six: .......................................................................................................................... 108
Finding seven:...................................................................................................................... 108
Finding eight: ....................................................................................................................... 108
Conclusion ............................................................................................................................... 108
Chapter V: Findings ................................................................................................................ 110
Introduction ............................................................................................................................. 110
Review of Themes from Research .......................................................................................... 110
Discussion of Key Findings .................................................................................................... 111
Finding one: ......................................................................................................................... 111
Finding two: ......................................................................................................................... 112
Finding three: ....................................................................................................................... 113
Finding four: ........................................................................................................................ 113
Finding five: ........................................................................................................................ 115
Finding six: .......................................................................................................................... 116
Finding seven:...................................................................................................................... 116
Finding eight: ....................................................................................................................... 117
Linking Findings to Theoretical Framework .......................................................................... 118
Linking Findings to Literature ................................................................................................ 121
Study Limitations .................................................................................................................... 124
Alternate explanations ............................................................................................................. 126
Implications for Educational Practice ..................................................................................... 127
Implications for Future Research ............................................................................................ 128
Conclusion ............................................................................................................................... 130
References .................................................................................................................................. 132
Appendix A ................................................................................................................................ 147
Appendix B ................................................................................................................................ 148
9
Appendix C ................................................................................................................................ 149
Appendix D ................................................................................................................................ 151
Appendix E ................................................................................................................................ 153
Appendix F ................................................................................................................................ 155
10
Chapter I: Introduction
The purpose of this qualitative case study is to understand the teacher hiring practices of
educational leaders in K-8 public schools in a particular county in a mid-Atlantic state. In this
research, teacher hiring practices are generally defined as the screening, interviewing, and
selecting of teacher applicants. In this case study, the researcher seeks to understand
administrators’ beliefs and actions related to hiring and examine what they choose to do as well
as what they choose not to do, and their explanation as to the reasons for those choices.
Knowledge generated may inform the practice of educational leaders in the important work of
staffing schools.
Statement of the Problem
This examination of the hiring practices of K-8 educational leaders is viewed through the
theoretical framework of bounded rationality, which holds that rationality is limited by the
availability of information, time, and mental capacity (H. A. Simon, 1982). According to
bounded rationality, the mental shortcuts that allow humans to survive in times of crisis requiring
quick, decisive action do not also generate predictable rational decision-making. As a result of
this natural mindset, the answers that come to mind first are deemed by the thinker to be the best
answers. Of all the decisions made by school principals, the hiring of teachers may be the most
important task (Mason & Schroeder, 2010; Rowan, 1994a); however, the bounded rationality
involved in decision-making unchecked by the supports of protocols such as research supported
screening criteria or interview procedures (Cranston 2014) engenders an ongoing problem of
practice: sub-optimal decision-making in the process of teacher hiring.
From the tender age of 6 until at least 16, American children are compelled by law to
spend their days under the tutelage of school employees. Students don’t pick these teachers,
11
school administrators do. Yet, it is the students who benefit or suffer from those selections
(Haskins & Loeb, 2007). The importance of the selection of teachers should not be
underestimated for as Hanushek (2011) suggests, “Replacing the bottom eight percent of
teachers with an average teacher would bring the U.S. up to the level of Finland” (p. 475). The
hype and debate over teacher evaluation rages on, but where is the discussion of the selection
process that results in teacher hiring in the first place?
Teaching is one of the most complex and demanding professions (Haskins & Loeb, 2007;
Rowan, 1994b; Schumacher, Grigsby, & Vesey, 2015). It is the task of teacher education
programs to prepare teacher candidates to teach effectively, but as in all fields of endeavor, these
novice professionals range in ability (Gentry, 2012). It is imperative effective teachers are within
the group of employed teachers as the selection of new teachers has a profound and long-
reaching impact on schools (Cranston, 2012; Rutledge, Harris, Thompson, & Ingle, 2008;
Stronge, Ward, & Grant, 2011). According to the latest (2016) Educator Provider Performance
Report form the New Jersey Department of Education, only a portion of trained teachers are
employed in the field; of the 9,284 New Jersey 2014 graduates with a certificate of eligibility
with advanced standing, only 65% were employed in the profession in 2016 (NJDOE). Given
that the pool of teacher candidates is greater than the available openings (Clement, 2013), and
the importance of the selection (Engel & Finch, 2015; Stronge et al., 2011) it would be expected
that hiring practices would be deeply researched, carefully and consistently practiced. This is not
the case (Liu & Johnson, 2006).
Tasks such as teacher evaluation, curriculum development, or student discipline are often
conducted with expectations and procedures for consistency and interrater reliability; however,
the actual practice of teacher hiring varies greatly, is often “information poor,” and the criteria
12
for selection is often based on feeling over evidence (Liu & Johnson, 2006 p. 331). Improvement
in the hiring process could produce significant improvement in student achievement (Heneman
& Milanowski, 2004). This study expands the available research on teacher hiring practices and
could lead to the refinement of present interview processes.
It is widely accepted that teacher effectiveness is the greatest predictor of student success
(Darling-Hammond, 2010; Stronge et al., 2011). The belief was expressed in the opening
statement of the Obama Administration’s Educational Plan of 2011 (Our future, our
teachers.2011). The importance of a quality education is undisputed; however, the best manner in
which to judge teacher effectiveness has been the subject of polarizing debates in education. The
U.S. Department of Education’s $4 billion dollar Race to the Top initiative of 2009 focused the
country on the process for evaluating employed teachers through value added measures which
calculate teacher effectiveness in part by comparing student test scores to those of previous
years.
Less discussed are the practices involved in initial teacher recruitment so that the most
effective teachers are selected in the first place. Little is known about the nuances of the process
of teacher hiring (Mason & Schroeder, 2010), defined for this study as the paper review, sifting
applications and resumes, and the live selection, conducting interviews. Yet, targeted research by
Engel & Fitch and Kayes (2015; 2006) has recently examined the steps in K-8 as well as higher
education, and some conclusions for better hiring processes have been offered. Effective hiring
practices are required to arm the teaching profession with the caliber of teachers needed to face
the onslaught of challenges (Peterson, 2002; Rutledge et al., 2008; Whitworth, Jones, Deering, &
Hardy, 2016).
13
Significance of the Problem
The complexity of and the importance of effective teacher hiring generates the potential
for problems. The hiring process begins with screening applicants based on static criteria such as
certification, education and experience. Screening is usually followed by individual interviews. It
is the impressions from interviews which are given the most weight in hiring decisions
(Cranston, 2012; Kersten, 2010; Rutledge et al., 2008). Specifically, principals focus on
interview skills and stated expressions of caring and work ethic rather than content knowledge or
pedagogical skill (Engel, 2013). The impressions made during an isolated meeting are given
more value than the information gleaned from the years of performance reported through records
reviewed in the screening process (Peterson, 2002; Rutledge et al., 2008).
One manner in which the hiring process may be less than optimal is caused by the
comparison to the employee being replaced. Information gleaned is often skewed, consciously or
unconsciously, in comparison to the teacher the candidate may be replacing. Candidates are often
measured by the fit to the staff already in place (Rutledge et al., 2008). Thus new hires may
replace a retiree or fill a new position not based on the researched characteristics of effective
teaching, but on similarity to the present staff, the principal’s sense of his/herself as a teacher, or
just general likeability. This raises concerns not only about replicating the status quo in teaching
styles, but also about diversity, or the lack thereof, among the school faculty.
A problem generated by poor hiring relates to the aftermath burden of the district. For
teachers who are hired, schools focus a good deal of time and money in order to generate
professional development. An entire section of federal funding, Title IIA: Preparing, Training,
and Recruiting High Quality Teachers and Principals, is allocated for this effort (USDOE,
09/15/2004). Additionally, monies are dedicated to teacher preparation at the higher education
level (USDOE, 8/14/2008). Through careful hiring, districts could allocate the requisite time and
14
energy of professional development to advance effective teachers rather than attempt to repair
ineffective ones. A good hire has the potential to impact generations of students positively,
whereas a poor selection has a direct negative effect on students. Additionally, a poor selection
can be a drain on the limited resources available for professional development and school-wide
improvement. The selection of staff can be one of the most lasting decisions made by an
educational leader (Mason & Schroeder, 2010).
Other aspects of poor hiring may be generated by availability of time and ability to apply
rational decision-making. Areas to be examined include possible overconfidence based on
snippets of information such as colleges attended or references mentioned to being inattentive to
missing information. Predicting the future behavior of an individual is extremely difficult
(Courtney, Lovallo, & Clarke, 2013; Schumacher et al., 2015; Venkatraman, Payne, & Huettel,
2014); the simple fact that the interviewee knows she/he is being judged may make accurate
assessment by the interviewer that much more difficult (Cain-Caston, 1999). Coupled with the
demands for time on the part of the school administrator, the interview process is ripe for
misunderstanding and erroneous conclusions (Rutledge et al., 2008; Shen et al., 2012). Schools
engage in ongoing professional development, observation, and evaluation, all with the goal of
ensuring the faculty is capable of meeting the challenges of the profession. However, ongoing
professional development or the addition of graduate degrees offer little impact on teacher
effectiveness; as it turns out, the relatively few hours spent screening and selecting good teachers
are far more important than the years of training that occur after hiring (Chingos & Peterson,
2011). Therefore, more time, effort, and focus is needed on the system of hiring teachers.
Research on teacher hiring is limited, but researchers such as Clement; Rutledge, Harris,
Thompson, and Ingle; and Schumacher (2013; 2008; 2015), offer possible improvements. One
15
example of a researched technique to improve rationality and consistency in teacher hiring
practices is the use of structured interviews specifically aligned to the domains of teaching --
planning, management, instruction, and reflection (Clement, 2013). These protocols are helpful
in selecting effective teachers, but structured interviews are rarely used (Clement, 2009; Liu &
Johnson, 2006; Rutledge et al., 2008). Other research central to the theoretical framework of
bounded rationality suggests that the use of structured protocols that address pitfalls in human
thinking would improve the rationality in hiring decisions (Bingham & Haleblian, 2012; Guerra-
López & Blake, 2011; M. Toplak, West, & Stanovich, 2011). The impact of bounded rationality
on even the best case scenario for teacher hiring practices requires the attention of educational
leaders. Suboptimal decision-making caused by missing data, the misinterpretation of data, or
overconfidence generated by the presence of some data could undermine the crucial work of the
hiring process and, therefore, the quality of instruction provided to students.
Positionality Statement
The researcher of this study brings to this work experiences and subsequently biases. As
Machi and McEvoy (2012) state, “Preconceptions, personal attachments, and points of view
present both strengths and weaknesses for the research effort” (p. 18). This positionality
statement examines the researcher’s biases and perspectives as these may influence this research.
An author’s position should be examined and considered in relation to others (Briscoe, 2005).
This researcher is a middle aged, middle class, mid-career, mid-Atlantic, monolingual, married
male. As such, his notion of the institution of education is influenced by experience of it from his
position within the larger society.
16
Further, this researcher is a product of public schools, was a teacher for over a decade,
and during the past two decades has served as a director of curriculum as well as instructor and
director of teacher education programs.
His perspectives on education were developed over unique philosophically oriented times
and within differing roles. These could influence his beliefs about education in general and also
the notion of effective teaching specifically. This researcher was a K-8 student in public schools
from 1971 to 1984. Hargreaves & Fullen (2009) term the period of 1970-1990, the Interregnum,
as it was a time that maintained the optimism and innovation of the 1960s, but also infused
market-driven competition brought on by a declining economy. This researcher was a teacher
from 1990-2002. This era, Hargreaves (2009) terms the Second Way, was a period of increased
prescriptive curriculum, outcome-based measures, and state-led reforms. This researcher serves
as an administrator during the years of the Third Way, from 2002-Present, a time of autocracy
and technocracy. Therefore, it should be noted that his desire to seek an improved decision-
making process to identify effective teachers, may be in the context of this Third Way or a
combination of experiences in all three. It is to be seen if this research fits the inclusive,
supportive, sustainability focused Fourth Way which Hargreaves (2009) prescribes for the future
of education.
This researcher’s demographic and ideological positioning impacts his views on the role
of education and therefore his views of the decisions made by those within that system.
Interestingly, unlike many of his doctoral program colleagues, his inquiry does not focus on
individuals or groups served by education; rather, in the study of the educators themselves. It
should be noted that as an insider within the group of administrators, he may see the decision-
making processes of educators differently than an outside observer might. Briscoe (2005) asserts
17
that perspectives are set by one’s position within a culture and enmeshed in the language of the
culture. Therefore, it may be this researcher’s relationship to educational leaders, not to the
demographics of the students they serve, that should be most closely inspected for bias.
Beyond this researcher’s position as an administrator and his intent to research the hiring
practices of schools in which some colleagues are known to him through his years in working in
education in the area, additional biases and perspectives germane to his specific area of study
include his beliefs which may be the product of the theoretical and political influences on
education during the years as a student and as an employee. These beliefs include: the child’s
right of access to quality education supersedes the preferences of the employees of a school, the
possibility that quality education can be established for all, and that available knowledge coupled
with willingness can make this possible.
This study is feasible at the target site of K-8 schools within a mid-Atlantic region of
school districts. Participants volunteered via reply to a brief survey of interest. Interviews were
conducted based on the pool of responses. Even though the researcher is a member of a specific
district within the larger target group, the study’s design, the confidentiality offered, and the
voluntary nature of the participation offer an appropriate setting to conduct this work.
Research Question
The purpose of this study of as educational leaders in a mid-Atlantic region of school
districts is to understand their experience of teacher hiring. This research explores the challenges
involved in identifying effective teachers during the hiring process which include the paper
review, sifting applications and resumes, and the live selection, conducting interviews.
Therefore, the researcher has elected to utilize qualitative research. In this study, the researcher
seeks to understand administrators’ beliefs and actions related to hiring and examine what they
18
choose to do as well as what they choose not to do, and their expression as to the reasons for
those choices.
The present qualitative study was designed to answer the following research question:
How does bounded rationality affect administrators’ decision-making processes in the process of
hiring new teachers among comparable school districts in the mid-Atlantic region? As described
in the next section, bounded rationality is a framework this researcher utilized to frame the
examination.
Theoretical Framework
Bounded rationality, first coined by Simon (1955) offered a response to the dominant
theory of the time, neoclassical rationality, which held that human decisions were calculable,
rational responses with the most utility as the result. Simon suggested humans responded not
with the most logical solutions, but the most readily satisfying (H. A. Simon, 1982). Additional
Nobel awarded work was conducted by Kahneman and Tversky (1984) which identified the
mental shortcuts or biases and heuristics most involved in fast thinking. Present work in bounded
rationality continues to define the fields of psychology and economics and the understanding of
human rationality focusing on areas such as predictions and forecasting (D. Lovallo, Clarke, &
Camerer, 2012), confirmation bias (Rabin & Schrag, 1999), and brain and addiction research
(Bernheim & Rangel, 2004).
Despite the human shortcomings involved in processing data demonstrated through the
decades of research building the theoretical framework of bounded rationality (Botterill &
Hindmoor, 2012; Kahneman, 2003; H. A. Simon, 1982), in the field of education, there remains
a widely held belief in the possibility of rational, data-driven decisions in schools (Cooley et al.,
2006; Marzano, Waters, & McNulty, 2005; Reeves & Flach, 2011). The best practice era that has
19
dominated the world of teaching, which assumes a set of techniques can be applied to all cases to
garner the same effect, mirrors the zeal in which data-driven decisions are prescribed for
educational leaders. However, bounded rationality holds that regardless of the clarity of the
available data, intuitive, or fast thinking, undercuts rational thought (Kahneman & Tversky,
1984). When individual perceptions filter the interpretation of data, even carefully organized
data-driven decision-making settings are jeopardized (M. E. Toplak, West, & Stanovich, 2014).
In this view, it is not the just the use of data to drive decisions that cause school leaders to
navigate the problem solving required, but rather understanding their own unconscious filtering
of that data that improves decision-making. This critique of neoclassical economics suggests the
emphasis of data-driven decision-making is overly simplistic in its cause and effect view of
gathering and viewing of data and logical choice selection (H. A. Simon, 1955). Viewing teacher
hiring processes through this lens may inform and guide the practice toward improved efficacy.
Biases and heuristics play a significant and often undetected role in the outcomes of
decisions (Hoppe & Kusterer, 2011; Hutchinson, Alba, & Eisenstein, 2010; Kahneman, Lovallo,
& Sibony, 2011). Biases are the natural tendency to think in certain ways (Kahneman et al.,
2011). These patterns of thinking impact the rationality of decisions. Heuristics are simple
practices that quickly produce seemingly adequate answers to difficult questions. Answers that
are derived quickly are more likely to be confidently perceived as accurate by both the thinker
and those around him or her (Kahneman & Tversky, 1984). In daily life, these quick thinking
patterns are useful. It is the impact on situations that require more rational thinking that is
concerning (Kahneman, 2011; Thaler, 2009). Furthermore, when beliefs and data conflict,
people, even analysts, tend to reinterpret the data so that it conforms to their thinking rather than
updating their beliefs (Hutchinson, Alba, & Eisenstein, 2010). Other specialists, certainly
20
educators, possess a lower percentage of leaders fully cognizant of the rules of statistical
analyses. Bounded rationality (Kahneman, Lovallo, & Sibony, 2011; H. A. Simon, 1991) holds
that it is human nature to prefer cognitive ease and therefore for mental biases and heuristics to
generate a choice preference of the first decision that comes to mind. Affect (Tversky &
Kahneman, 1974), confirmation bias (Tversky & Kahneman, 1974), the halo effect (Tversky &
Kahneman, 1974) and other bias identified within the bounded rationality framework may be the
mental shortcuts that limit the consistency and rationality of the hiring process in lower
achieving schools. Affect biases are decisions based on emotions, on likes/dislikes or how one
feels about something replaces the more difficult question of what one thinks about something.
Anchoring biases are the weighing of one piece of information, often the first information
encountered, too heavily. Confirmation bias causes the searching for evidence that supports the
original belief while ignoring evidence that contradicts it. Understanding the degree to which
heuristics and biases alter the decision-making process of individuals and teams in educational
settings is needed.
This study applies this framework of bounded rationality as studied in areas of business
and medicine to an overlooked group, educational leaders. Any advancement in the area of data-
driven decision-making in educational leadership settings requires a clear understanding of
educational leaders’ data filtering processes and the resulting impact on their rationality.
Individuals’ biases and heuristics can be identified, as can the resultant impact on choices
(Kahneman et al., 2011; Schlag, 2010; H. A. Simon, 1982). The specific biases and heuristics
identified in bounded rationality are exactly those involved in hiring: e.g. affect, anchoring, and
confirmation.
21
Therefore, the researcher seeks to understand the decision-making process involved in the
hiring of teachers through the lens of bounded rationality. Beneficiaries of this investigation
would include educational leaders, principals, specifically, as well as students, teachers, other
administrators, and the community.
In Chapter Two, the researcher examines three streams of literature: the decision-making
as viewed through the lens of bounded rationality, techniques utilized in the attempt to increase
the rationality of decisions in the practice of various fields such as business and medicine, and
teacher hiring practices. The review of this literature examines the applicability of the framework
of bounded rationality to the understanding of administrators’ experiences in teacher hiring.
22
Chapter II: Literature Review
In this literature review, the researcher examines three streams of literature. The first is
the decision-making as viewed through the lens of bounded rationality, which holds that
rationality is limited by the availability of information, time, and mental capacity (H. A. Simon,
1955). The second stream includes techniques utilized in the attempt to increase the rationality of
decisions in the practice of various fields such as business and medicine. Finally, the researcher
examines research on teacher hiring practices in order to understand whether teacher hiring
decisions may also be similarly affected by bounded rationality and ameliorated by awareness
techniques or decision-making tools such as those utilized in other fields.
Bounded Rationality
Bounded rationality, first coined by Simon (1955) offered a response to the dominant
theory of the time, neoclassical rationality, which held that human decisions were calculable,
rational responses with the greatest utility as the result (Jevons, 1888). Simon suggested humans
responded not with the most optimal solutions, but the most readily satisfying (H. A. Simon,
1982). Additional Nobel awarded work conducted by Kahnemann and Tversky (1984) identified
the specific mental shortcuts or cognitive biases and heuristics most involved in fast thinking.
Their work, compiled in Thinking: Fast and Slow (Kahneman, 2011) establishes a view of the
human mind as one that defaults to quick, subconscious choices, but with awareness able to slow
and shift to the more rational abilities. Quick thinking allowed humans to survive through the
immediate dangers of the millennia and continue to be useful for eminent danger such as the
demands of driving. However, the important decisions of modern man do not entail the
immediacy of sabretooth tigers, but the complex and long range strategies required for such
endeavors as building enterprises, educating youth, or curing illnesses. Heuristics are simple
23
mental functions that allow for quick decisions and cognitive biases are the illogical results that
often are caused by utilizing heuristics (Kahneman, 2011; Medin & Bazerman, 1999). Present
work in bounded rationality continues to define the fields of psychology and economics and the
understanding of human rationality focusing on areas such as forecasting (Frederick, 2005)
confirmation bias (Rabin, 2013), and brain and addiction research (Bernheim & Rangel, 2004).
Identified biases and heuristics. The ability to avoid errors through the check and
balance of the thinking System 1 and 2 is limited by time, mood, and mental distractions
(Kahneman, 2011). Central to this area of research is the specific identification of cognitive
biases and heuristics and the manner in which these impact decisions. Of the many biases and
heuristics identified by the seminal work of Tversky and Kahneman (1974), most relevant to the
study of decision-making in educational settings are: affect, anchoring, availability,
confirmation, halo effect, missing data, prediction overconfidence, and sunk cost.
Affect bias causes decisions to be based on emotions or on likes/dislikes. How one feels
about something replaces the more difficult question of what one thinks about something
(Tversky & Kahneman, 1974). This could cause administrators to select candidates based on
adult to adult likeability rather than teacher efficacy.
Anchoring causes the weighing one piece of information too heavily, often the first
information encountered. (Hoppe & Kusterer, 2011; Kahneman, 2011; Sunstein & Thaler, 2003)
When evaluating people, such as in the hiring process, this results in a Halo Effect (Lovallo, D.
& Sibony, O., 2010; Singh, 2008; Timmons, 1998) which occurs when someone is liked, due to
one characteristic such as attractiveness, and his/her negative characteristics are therefore
overlooked. This could cause administrators to select candidates with shortcomings related to the
abilities as teachers that should have prevented their selection.
24
Availability, also known as saliency, is the tendency to overestimate the likelihood of
events when similar past events come to mind (Cioffi & Markham, 1997; Tversky & Kahneman,
1974) An example of this would be to fear the ocean due to likelihood of shark attacks when an
event can be recalled and to overlook the much more likely event of drowning due to unseen, but
common riptides. This could cause administrators to select candidates based on the
administrator’s specific knowledge base. If the candidate is a graduate of a college from which
the administrator had a previous positive experience, the teacher may be overly favored to
candidates that attended superior, but lesser known colleges regardless of other compelling
information.
Confirmation bias causes the searching for evidence that supports belief and ignoring
evidence that contradicts it (Beshears & Gino, 2015; Tversky & Kahneman, 1974). This could
cause an administrator to focus on evidence that supports a preconceived preference or aversion
to a candidate. Rather than comparing all candidates on their demonstrated abilities in all
domains of teaching, confirmation bias may cause an administrator to focus on the one domain
his/her preferred candidate most demonstrated and ignore the evidence that suggest another
candidate to be more suitable.
Missing data bias is the tendency to ignore the lack of available data needed for rational
decisions (Henningsen & Henningsen, 2007; Tversky & Kahneman, 1974). The interview
process is riddled with opportunity to miss important data. If an administrator ignores missing
information, candidates about whom little is known may be favored over those better known.
This could cause an administrator to neglect to conduct thorough reference checks.
Prediction overconfidence is the subjective confidence level that is not supported by
objectivity (Campbell, Goodie, & Foster, 2004; M. Toplak et al., 2011; Tversky & Kahneman,
25
1974). This could cause an administrator to mistake confidence in a decision for evidence of
rationality of the decision and thus offer a contract to a teacher who has not provided evidence to
support predictions of effective teaching. One example might be the assumption that a successful
youth league sports coach would be an equally successful physical education teacher.
Sunk cost effect, also known as loss aversion, is the phenomenon in which people justify
increased investment of time or resources into a decision, based on the cumulative prior effort
(Tversky & Kahneman, 1974). This could cause administrators to select candidates who at the
end of a lengthy screening process may not be strong candidates, but the administrator is loath to
reopen the hiring process due to the time already invested in finding the first candidate.
Bounded rationality’s effect on decision-making. Sterman (1989) extended research on
the impact of biases and heuristics in individual decision-making to the implications for
organizations. Work since continues to explore the construct between the benefits of heuristics
that allow for decision-making and their potential to shortcut rational decisions (Kahneman,
2003; Kahneman et al., 2011; Waddell & Sohal, 1994). Sterman’s (1989) empirical study
measured business leaders’ heuristics including anchoring and adjustment. Anchoring is the
establishment of mental frames of value that impact subsequent choices. Adjustment is the term
used to describe the failure to shift mentally from anchoring values, such as basing the value of a
stock on the original purchase price. Sterman identified individuals’ biases and heuristics and
their resultant impact on decisions. In doing so, he demonstrated the weakness in traditional
decision-making theorists (Jevons, 1888) who viewed decisions making as a rational outcome of
data input.
As organizations become increasingly complex, so do the decisions. However, empirical
studies (Eisenhardt & Zbaracki, 1992; Kahneman et al., 2011; Nobre, Tobias, & Walker, 2009;
26
H. A. Simon, 1982) demonstrate that subjects are insensitive to the feedback from their
decisions. This impacts the function of the organizations tremendously. Insensitivity to the
complex surroundings distances the decision makers from reality. Insensitivity to data impedes
the very essence of data-based decision-making. The findings which support the framework of
bounded rationality have been applied to decision-making in organizational settings and have
forwarded the importance of understanding the construct of biases and heuristics.
An important element in decision-making is forecasting. Eisenhardt (1992) questions
whether, due to the complexity of decisions, rationality is even an achievable goal. As world
complexity increases, so does forecasting difficulty. Additionally, forecasting is often done in the
manner most comfortable to the leaders involved and even steered toward the predictions that are
most favorable to them as individuals (Waddell & Sohal, 1994). Those leaders with pioneering
and innovative (PI) mindsets tend to choose more innovative choice options (Manimala, 1992).
Busenitz (1997) suggests that in some situations, overconfidence and representativeness are
actually beneficial in entrepreneurial decisions, but may not be in managerial ones. Successful
entrepreneurs select different courses of action than successful managers. Again, outcomes are
dependent on the individuals, not the data set.
Numerical data is often filtered by cognitive heuristics. Research on trend-based
inferences made by experienced marketing managers demonstrates that graphical data displays
greatly impact the conclusions drawn (Hutchinson et al., 2010). Therefore, data and data
representation tools that should improve decision-making can also mislead. Worse, since data is
involved, the decision makers are more likely to overestimate the accuracy of the choice
(Kahneman & Tversky, 1984). What should be an aid can, consequently, be doubly a hindrance.
Very basic erroneous casual relationships are often drawn when reviewing data (Imai &
27
Yamamoto, 2010). The misinterpretations generated in the study by Imai and Yamamoto (2010)
were made by those well-versed in statistical data. Other fields, certainly education, possess a
low percentage of leaders fully cognizant of the rules of statistics.
An extension of the problem caused by misinterpretation of data is the issue of
information gap. When information is missing from a data set, people often infer the value of
missing information based on the value of the known information, sometimes bolstering or
diminishing the value (Henningsen & Henningsen, 2007) -- assuming the groups realize there is
missing information. When groups meet to discuss and make decisions, it is the shared, not the
unique perspectives, that are given the greatest deference (Kahneman et al., 2011). Therefore,
increasing the number of members of a group does not necessarily increase the number of
choices to be considered. Information that is unknown to all is not discussed and information that
is known to just a few is devalued (Henningsen & Henningsen, 2007).
The specific mental shortcuts of biases and heuristics can be viewed through the related
lens of prospect theory which highlights three regularities in decision-making: avoiding loss,
noticing change more than absolutes, and allowing estimations to be biased by anchoring
information (Kahneman & Tversky, 1984). The descriptive model of prospect theory holds that
individual preferences are not consistent and the various outcomes are related to the manner in
which the choices are presented.
Carefully considered decisions utilize what Kahneman calls, System 2, or slow thinking.
This effortful, concentrated thinking corrects the quick assumptions of System 1, which is the
human intuitive, effortless, or fast thinking (2003; Kahneman, 2011). A question from the
Cognitive Reflection Test (CRT) introduced by Frederick (2005), illustrates the working of the
two systems. A bat and a ball cost $1.10. The bat costs one dollar more than the ball. How much
28
does the ball cost? (Frederick, 2005 p. 25) The quick thinking of System 1 selects the first
seemingly acceptable answer, $0.10. Only in engaging System 2 is the quick answer exposed as
incorrect and the correct answer of $0.05 for the ball and $1.05 for the bat realized. For if the bat
costs a full dollar more than the ball, a $.10 ball would require a $1.10 bat and therefore a sum of
more than $1.10. Only a $.05 ball and a $1.05 bat would equal $1.10. Humans have survived
through the millennia because most of life-threatening challenges require quick, intuitive
thinking which System 1 offers. However, complex decisions benefit from the careful
examination of System 2.
Critics of the framework. Bounded rationality grew out of criticism of rational choice
theory (Botterill & Hindmoor, 2012; Mumby & Putnam, 1992; H. A. Simon, 1982). Critics of
bounded rationality suggest that even if bounded rationality alters outcomes, attempts to
overcome bounded rationality may slow decision-making processes thus impeding individual
and organizational reaction time. Alternative critiques included feminist perspectives that
bounded rationality does not remove the patriarchal overtones found in rational choice theory
and specifically theorists such as Mumby & Putnam (1992) call for a theory that connects
rationality and emotionality. Others, such as Mackenzie (1997), offer sympathy toward the
theory of bounded rationality, but critique the applicability of some of the experimental
evidence. This research would allow for expanded understanding of the applicability of bounded
rationality in a study.
Summary. To improve a system, such as education, the sense making of the individuals
within the system must be understood (Jopp, 2005). An important focus of study within systems
theories examines the cultures or systems that support sense making and learning (Schwandt,
2005); however, even the most functional system would be impeded by limitations to individual
29
members’ rational thought caused by biases and heuristics (Kahneman et al., 2011; Sunstein,
Kahneman, Schkade, & Ritov, 2002). The seminal work in the stream of literature focused on
biases and heuristics is that of Sterman (1989) which examined the impact of biases and
heuristics in decision-making of individuals and the implications for organizations.
Sterman’s empirical study (1989) measures business leaders’ heuristics including anchoring,
establishing frames of value that impact subsequent choices, as they manage stock portfolios,
and challenges the central premise of data-driven advocates as it demonstrates that data is
misinterpreted by experts. Although bounded rationality has been a major theory used to
understand decision-making of leaders in other fields, particularly medicine and business, the
alternate lens advocating for data-driven decision-making has dominated educational leadership
research. As the research in bounded rationality demonstrates, the shortcomings of human
thinking are constant across fields and expertise (Botterill & Hindmoor, 2012; Gigerenzer, 2010;
H. Simon, 2000; Wilson, 2010). The very thinking of the leaders involved in educational
decision-making deserves an examination equal to the work offered the thinking of leaders in
other fields.
Decision-making Tools
Kahneman demonstrates that measures can be made to improve the rationality of
decision-making by engaging the System 2 portion of his thinking model. For example, framing
can be used not only to decrease, but also to increase the accessibility of information that might
otherwise be overlooked (2011). Altering the surroundings, cues, and information associated
with decision-making settings can improve the rationality of the decisions made (Adams &
Ericsson, 2000; Bacon, Fulton, & Malott, 1983; Kahneman, 2003; Schlag, 2010; Thaler, 2009).
This is a tremendously important consideration for educational leaders.
30
Decision-making tools used in business. Recent research (Botterill & Hindmoor, 2012;
Halevy & Chou, 2014; Kahneman et al., 2011; Lovallo, D. & Sibony, O., 2010) explores the
manner in which bounded rationality can be discovered and reduced in order to improve the
decision-making process. Business leaders consider behavioral economics when predicting the
actions of other such as investors and consumers, but they are less likely to consider their own
thinking processes (Botterill & Hindmoor, 2012). Senior executives are particularly at risk for
saliency biases, giving too much importance to past events (Lovallo, D. & Sibony, O., 2010).
The decision-making process is more important than the data itself when attempting to “debias”
strategic decisions (Kahneman et al., 2011; Lovallo, D. & Sibony, O., 2010). Selecting and
analyzing data to make predictions of future behavior of customers relates to educational leaders’
attempts to collect data in order to predict a candidate’s future behavior as a teacher.
It has been shown that business executives can reduce the impact of biases such as
anchoring, weighing one piece of information too heavily, and confirmation, ignoring evidence
that contradicts one’s beliefs (Kahneman et al., 2011). “But knowing you have biases is not
enough to help you overcome them. You may accept that you have biases, but you cannot
eliminate them in yourself” (Kahneman et al., 2011, p. 52). However, biases are more readily
noticed in others. Therefore, a properly oriented organization can improve the decision-making
process. This is using System 2, rational thought, to check for erroneous thinking in System 1,
intuition. In decision-making, the focus is on the content of the situation and suggested course of
action. Business executives must receive and comprehend key information from those with
detailed knowledge. Secondly, they must determine if the information is being clouded
intentionally or unintentionally by biases and heuristics. Finally, they must examine the degree to
which their own biases and heuristics are impacting their decision-making. Kahneman and
31
Lovallo (2011) suggest the addition of a systematic review of the process as well. Specifically,
they call for a quality control checklist consisting of 12 driving questions. These are divided into
three categories: the decision makers, the people making the proposal, and the proposal itself.
Additional benefits for decision-making checklists include greater job specification
procedures and self-monitoring interventions (Bacon et al., 1983). In this light, checklists could
be beneficial to individuals as well as groups. In 1983, Bacon et al. asserted checklists to be an
inexpensive and effective means of improving job performance. However, at that time there was
a lack of literature that detailed the checklist components that were most effective. Recent
research (Churchman & Doherty, 2010; Flottorp et al., 2013; Haynes et al., 2009; Kahneman,
2003; Sibbald, De Bruin, & Van Merrienboer, 2013; Singh, 2008; Stubenrauch et al., 2012;
Takala et al., 2011) responds to that gap in knowledge. Checklists work best when these include
task definitions, record responses directly on checklists, and are periodically reviewed by a
supervisor (Flottorp et al., 2013). Research on knowledge based systems also asserts that users
are more likely to adhere to the recommendations of decision-making tools when explanations
are provided along with both feedforward and feedback applications (Arnold, Clark, Collier,
Leech, & Sutton, 2006).
“The sweet spot for quality control is decisions that are both important and recurring, and
so justify a formal process” (2011, p. 59). The use of an agreed upon checklist allows critique
and dialogue to be part of an organization’s climate. If a single individual were asked to fill that
role, social pressure might dissuade her/him. The proponents of this checklist offer
implementation suggestions. The checklist should be used for important, not routine decisions.
When in use, it should be fully used. Kahneman (2011) references the rigor of use of the World
Health Organization (WHO) Surgical Safety Checklist as a model of adherence.
32
Decision-making tools used in medicine. Experts, even highly trained doctors, can
improve their efficacy through the use of checklists and protocols (Fudickar, Hörle, Wiltfang, &
Bein, 2012; Sibbald et al., 2013; Takala et al., 2011). The World Health Organization (WHO)
released a 2008 safety alert and the accompanying WHO Surgical Safety Checklist. The Global
Patient Safety Challenge goal was to reduce the number of worldwide surgical deaths. Consisting
of 19 items arranged in three sections all of which are to be read aloud, the checklist was the
tool. The WHO Surgical Safety Checklist has been found to improve operating teams’ team
communication, the lack of which was cited as (a) leading cause of medical errors (Takala et al.,
2011) and to reduce perioperative mortality (Fudickar et al., 2012). Detractors of the use of
checklists express concerns that experts are be unnecessarily confined by checklists, yet in a
study of electrocardiogram (ECG) interpretation experts, checklist use during diagnostics did not
increase cognitive load; however, it did reduce diagnostic error (Fudickar et al., 2012). Often the
expectation of experts such as doctors is that the formal training they have experienced engrained
a shared knowledge base thus establishing in them a mental discipline that is brought to all
situations. Not only does the safety checklist research indicate that experts are fallible, but that
the inclusion of checklists reduces mistakes of omission while the use increases team
communication (Adams & Ericsson, 2000; Arnold et al., 2006; Bacon et al., 1983; Bekker, 2010;
Flottorp et al., 2013; Funk & Axelrod, 2011; Grantcharov & Reznick, 2009; Haynes et al., 2009;
Sibbald et al., 2013).
The WHO checklist is a model which could be considered in order to determine specific
items within a study on educational checklists. The following are samples of questions from each
of the three sections of the WHO checklist (World health organization: Patient safety.2009) and
the particular relation to this inquiry.
33
The sign-in section of the WHO checklist includes items such as whether the patient
identity and scheduled procedure has been confirmed with the actual patient. This is similar to an
educational checklist including whether the characteristics needed to successfully perform in the
open position have been established and whether the interview questions have been established
to meet identify the correct candidate. Serious medical errors can occur when the actions
performed are not matched to the correct patient. It can be imagined the selection of staffing
should follow the matching of student needs and characteristics of effective teachers. Often
practitioners seek answers before problems are fully investigated (Short & Shindell, 2009). It is
reasonable that educational leaders’ first priority should be the understanding of the needs of
their students (Marzano et al., 2005).
Before start of surgical intervention all team members are instructed to introduce
themselves by name and role. This simple step has been shown to increase communication and
teamwork (Takala et al., 2011). Perhaps this teambuilding step would help in educational settings
as well by supporting individual voices and therefore reducing groupthink, or the poor decision-
making caused by peer pressure, (Janis, 1982 p. 9). This step of the WHO Checklist also includes
a question that asks if all imaging, such as x-rays, has been reviewed. For educators, this relates
to ensuring that all needed data is available and perspectives heard. This could counter
availability, confirmation, and halo effects. Pausing as individuals or groups to explore sources
for missing information can increase rationality (Henningsen & Henningsen, 2007).
The sign out section of the WHO Checklist requires the team to discuss patient recovery
before leaving the operating room. This is similar to a review of decisions held at the end of
educational leadership meetings. It focuses the group on the next steps. This is of critical
34
importance for educational leaders who often leave one discussion only to be entrenched in the
next.
Referring to the WHO Checklist, Kahneman argues that experts should not pick and
choose from questions, but use these systematically. “Using checklists is a matter of discipline,
not genius. Partial adherence may be a recipe for total failure” (2011, p. 60).
Another area of group dynamics that checklists address is that of power dynamics. An
aspect of working with experts is the interplay of deference to expertise and alternately the need
for checks and balances. Often in the medical field, nurses express trepidation to speak out when
they disagree with a doctor’s choice (Churchman & Doherty, 2010). The protocol of the WHO
checklist places the charge nurse in a position of power that better distributes the responsibility
to utilize best practices and, therefore, the checklist serves to make questioning a procedure
rather than the act of a provocateur (Kahneman et al., 2011).
Decision-making tools used in education. Bartels and Mortenson (2005) examined
decision-making tools, checklists and performance feedback utilized in educational settings,
specifically, the use of these tools in adherence to problem-solving methodology in middle
school Intervention and Referral Service (I&RS) Teams. New Jersey statute (NJ DOE, October
2014) requires every school to maintain an I&RS Team to support the efforts of general
education teachers to support students experiencing learning, behavior, or health difficulties.
I&RS Team members examine data on student progress and related interventions instated. From
the collected data, they are to follow problem solving protocols in order to make decisions as to
the recommendations to offer the involved general education teachers. Bartels and Mortenson
(2005) found performance feedback to team members related to adherence to problem solving
protocols was not to be effective until coupled with a checklist.
35
The use of checklists is a very tangible and overt attempt to counter bounded rationality.
If leaders are willing and able to generate a culture of systematic and transparent decision-
making, biases can be minimized (Courtney et al., 2013). But what of less overt manners to
improve decision-making? Can the irrationality of biases and heuristics be limited without overt
discussions?
Subtle decision-making influence. The Cognitive Reflection Test (CRT) has been used
to identify suboptimal decision-making, but it has also been used to demonstrate means to
improve decision-making (Campitelli & Gerrans, 2014; Frederick, 2005). Using one of the items,
the ball and bat question, some participants were given clear copies, others were asked to furrow
their brows while reading and others were given degraded fonts. The latter two groups had
higher rates of accurate answers (Oppenheimer, 2012). The researchers posit the visual challenge
of the degraded font and the furrowing of the brow engaged the critical thinking power of
System 2. The shift from the quick thinking to more careful consideration can be initiated even
without the awareness of the thinker (Ménard, 2010; Schlag, 2010; Thaler, 2009). The idea of
subtly nudging people toward better choices is examined in the arena of choice architecture or
libertarian paternalism (Schlag, 2010; Thaler & Sunstein, 2003).
Sunstein and Thaler propose an approach that is less obvious than checklists, this is
choice architecture, “policies selected with the goal of influencing the choices of affected parties
a way that will make those parties better off” (2003, p. 175). If bounded rationality’s negative
impacts on reasoning are unnoticed, choice architecture designs opportunities to support rational
thought. With a focus on helping people make choices in financial markets and citing the work
of Kahneman and Tversky (1984) on bounded rationality, Thaler and Sunstein (2009) argue that
people do not always make decisions that are in their best interest and it is the role of
36
organizations to help them do so. They suggest that organizational decisions always have an
element of paternalism. They use an example of a cafeteria to illustrate the point. Should not
cafeteria workers place the fruit before the desserts? This still allows for choice, the libertarian
aspect, while protecting individuals from harm, the paternalistic aspect. Rumble strips are nudges
to control driving speed. A nudge is “any aspect of the choice architecture that alters people’s
behavior in a predictable way without forbidding any options or significantly changing their
economic incentives” (Thaler, 2009, p. 6). For Thaler and Sunstein, it is not a question of
whether organizations should act in paternalistic ways, but rather how to do so. “How should
sensible planners (a category we mean to include anyone who must design plans for others, from
human-resource directors to bureaucrats to kings) choose among possible systems, given that
some choice is necessary” (2003, p. 178)? Much of their work focuses on default choices.
Default options appear on numerous forms. A new employee might see an option of
whether to enroll in a 401K program or a certain insurance program. A driver’s license form
will often include a question about whether to join an organ donor program. When faced
with choices, people are more likely to select the default options (Thaler, 2009). Libertarian
paternalism offers choice, but particularly when expert knowledge is required, the argument
is that the default to be chosen should be the one with the greatest likely benefit (Ménard,
2010). Through cost-benefit analysis, three queries are suggested. First, consider what the
majority would select if the choices were fully understood. Second, consider an approach
that would require people to make an explicit choice. Third, consider an approach that
would have the fewest number of people later changing their decisions (Thaler, 2009).
A balance must be maintained so that the actions in the name of libertarian
paternalism do not reduce the amount of available information and the ability of decision-
37
making on the part of the individuals as that might reduce social learning (Carlin, Gervais,
& Manso, 2013). The other balance point is that as the name suggests, the participants
remain free to choose and as the research on human learning detailed above, human choice
is fallible. Libertarian paternalism is a prescription for support decisions regarding financial
decisions and may not be applicable to understanding fully the motives that lead to the
decisions (Mackenzie, 1997) or to arenas outside of financial decisions. Much of the
available research in the stream of bounded rationality is focused on economics. The stream
of literature supporting the use of data-driven decision-making in education does not
address the issues of bounded rationality.
Summary. Using decision-making tools such as researched interview protocols improves
the rationality of the decisions of experts (Bacon et al., 1983; Bartels & Mortenson, 2005;
Brañas-Garza, García-Muñoz, & González, 2012; Cioffi & Markham, 1997; Fudickar et al.,
2012; Haynes et al., 2009; Kahneman, 2003; Place & Vail, 2013; Sibbald et al., 2013). However,
experts in fields of business, medicine and education often ignore protocols (Campbell et al.,
2004; Engel, 2013; Fudickar et al., 2012; Grantcharov & Reznick, 2009; Guerra-López & Blake,
2011).
Missing from the literature is an examination of the impact of cognitive biases and
heuristics in educational settings and whether the decision-making tools such as those used
in business and medicine might ameliorate the deleterious effects of bounded rationality.
This proposal builds a case for additional research into the area of decision-making
in educational leadership and specifically whether the negative impacts of natural biases and
heuristics on decision-making can be noted in the decision-making of educational leaders,
specifically, in the hiring of teachers. Decision-making tools such as checklists have been
38
used to improve decision-making in the fields including business and medicine without
hampering expert decision-making speed (Bacon et al., 1983; Bekker, 2010; Fudickar et al.,
2012; Haynes et al., 2009; Kahneman et al., 2011; Sibbald et al., 2013). This researcher
examines the thinking process of educational experts in order to categorize the biases and
heuristics that are most commonly involved. Further, this researcher seeks to understand if
rational decision-making could be made more overt and therefore manageable. If a checklist
or flowchart could be effectively applied when educators engage in complex decision-
making, specifically, teacher hiring, improved decision-making could be the result.
Decades of research in bounded rationality offers a number of claims regarding common
decision-making patterns (Botterill & Hindmoor, 2012; Eisenhardt & Zbaracki, 1992; Kahneman
et al., 2011; Langlois, 1990; Mumby & Putnam, 1992; H. A. Simon, 1982; Sunstein et al., 2002).
People tend to make decisions based on what they like, not according to a set of rules. With
substitution, the decision-maker converts the choices from questions about which is better to
which is liked more by the decision maker; thus substituting the easier question of which one is
preferred in place of the harder question of which one is better in some more objective sense.
This substitution can be noticed in the response to value questions, such as which teaching
candidate is most likely to be effective. It may seem in an important decision such as hiring this
would not occur, but experiments indicate that large stake decisions do not activate “rule
thinking” more than low stake decisions (Campbell et al., 2004; Kahneman et al., 2011; Kunc &
Morecroft, 2010).
Finally, it should be noted that although the focus of this stream of literature has been on
the filtering and interpretation of data, bounded rationality equally alters the collection of the
data to be reviewed and the manner in which analyses are shared (M. E. Toplak et al., 2014). The
39
first two streams of literature, bounded rationality and decision-making tools, expose the many
ways in which human thought filters, often erroneously, the logical conclusions of data sets.
While specific biases and heuristics interfere with rational decision-making, these can be
identified and ameliorated.
Hiring Practices
Citing research which includes Gentry (2007) and Stronge and Tucker (2000),
Schumacher, Grigsby, and Vesey (2015) assert the factor which most impacts student learning is
teacher effectiveness. Schools are only as good as the staff that work there (Nonaka, 1994; Place
& Vail, 2013); however, the hiring practices used to employee staff vary greatly and the criteria
for selection are often based on feeling over evidence, further, hiring is often “information poor”
(Liu & Johnson, 2006 p. 331). Even beyond the multitude of decisions such as those involved
with building management, curricular support, course scheduling, and budgeting, teacher hiring
may be the most important decision made by school administrators (Cranston, 2012; Pillsbury,
2005). However, the weight of hiring decisions do not receive the attention equivalent to the
import. As Whitworth (2016) states, “Given the time constraints and responsibilities of school
administrators, the teacher selection and employment process occurs quickly and involves a
relatively restricted amount of information and data” (p. 20). In order to better understand this
often ignored, yet vital process, this study examines the two main steps in teacher hiring:
screening and interviewing.
Screening. Compounding the challenge of limited research into teacher hiring has been
an overlapping definition of screening and interviewing process in the research that has been
conducted (I. P. Young & Delli, 2002). Additionally, research available on teacher hiring is far
more likely to focus on the paper screening process as that can be accomplished through surveys
40
and controlled experiments (I. P. Young & Delli, 2002). The first step in the teacher hiring
process is the paper review of applicants. Documentation may include proof of certifications,
resumes, collegiate records, application responses, and letters of reference. Timmons (1998)
argues the specific needs of the position should be generated prior to the start of the screening
process. This would focus the screener(s) on the most important first round criteria. After the
initial paper review is conducted to sort out candidates without proper certification and
qualifications, the remaining candidates should be screened in relation to the criteria determined
by the administrator or in cases of a selection committee, by the selection committee (Heneman
& Milanowski, 2004; Timmons, 1998). This research employs a qualitative study design to
examine the degree to which working administrators enter the hiring process with a list of
criteria and if so the manner in which these characteristics are weighed and applied.
Research (Haskins & Loeb, 2007; Rutledge et al., 2008) has shown the weight items are
held as evidence of teacher potential differ and therefore the pool of candidates gathered for the
next round, interviewing, differ. What is valued as predictors of effective teaching varies.
Although undergraduate GPAs and portfolios are ranked as important by most teacher
preparation program providers, administrators rated written quality of the resume and cover
letter, experience, and known references as key elements in the screening process (Cranston,
2012). As in all decision-making, biases and heuristics are present in the thinking of the
individuals gleaning candidates from paper documents. An example of this could be a hyper
focus on transcripts. Criteria such as advanced degrees have not been shown to be good
predictors of teacher effectiveness (Engel, 2013), yet the data point may be used in confirmation
bias to as evidence of a candidate preferred for another reason. Bias and heuristics are involved
41
in the filtering of the candidate pool of teachers as these are in the hiring process of other fields
(Cranston, 2012; Place & Vail, 2013).
Interviewing. Young and Delli (2002) found no more than two studies specifically
addressing teacher selection. They cite Bolton (1969), which used information masking about
teacher candidates in the selection interview and I. P. Young (1983) who altered the format of
teacher hiring practices to include panel interviews. A small amount of additional research on
teacher hiring has been produced since 2002.
Of all the steps in the teacher hiring process, principals place the highest importance on
interviews (Rutledge et al., 2008), above that of resumes and reference checks, second and third
most important respectively (Cranston, 2012; Rutledge et al., 2008). However, “Interviews are
wrought with inherent biases that interviewers hold relative to the candidate’s appearance,
gender, age, and non-verbal cues” (Cranston, 2012 p. 362). To control for interviewer bias,
researchers (Engel & Finch, 2015; Haskins & Loeb, 2007; Heneman & Milanowski, 2004;
Lange, Range, & Welsh, 2012; Schumacher et al., 2015) have recommended job specific criteria
and interview protocols that focus administrators on specific characteristics be utilized.
Teacher interviews focused on specific job-related questions have been shown to be more
valid (Schumacher et al., 2015). Stronge (2003) suggests specific questions delve into the
characteristics noted within his Domains of Teacher Effectiveness model such as subject
knowledge, classroom management, assessment, and motivation (p. 48). This research examines
the literature supporting the assertion that such a focus is widely utilized and even if so would be
effective since candidates are more often judged by interviewing skills than pedagogical skills
(Engel, 2013; Peterson, 2002). Often likability or the similarity of the new hire to the displaced
teacher is given the greatest consideration (Rutledge et al., 2008).
42
According to Haberman, “School districts need to use validated interviewing instruments
to determine the likelihood that the young adults they hire will be effective and remain in
teaching for 5 or more years” (2012 p. 939). Experimental studies of teacher hiring have
examined the independent variables of hypothetical candidate age, gender, and experience (Place
& Vail, 2013; I. Young & Joseph, 1989). These variables were compared to the administrators’
demographics and their ratings of the hypothetical candidates. One interesting finding was that
although experienced candidates were preferred over those with three or fewer years, generally
younger teacher candidates were preferred by administrators in urban districts (I. Young &
Joseph, 1989). This outcome of this experimental study suggests administrators bring
preferences to the characteristics of the candidates they desire to contract. This is significant as
experience has been shown to improve teacher effectiveness; teacher preparation or youthful
energy have not (Harris & Sass, 2011; Rutledge et al., 2008).
Commercially produced interview protocols have not been shown to improve the focus
on teacher practice as these tend to focus on personal attributes (Schumacher et al., 2015).
Research on teacher hiring that mirrored occupational research has demonstrated structured
interview processes specifically aligned to the domains of teaching to be effective, but these are
often not used (Clement, 2000; Rutledge et al., 2008). Clement (2013) suggests ongoing,
systematic hiring processing which includes behavior-based interviews, as past behavior is the
best predictor of future performance. Interview questions of this approach begin with phrases
such as “how have you…,” or “tell about a time when…” This roots the responses to
experiences rather than hypotheticals. The questions focused on specific areas of teaching:
designing instruction, classroom management, implementing instruction, and monitoring student
progress should be scored individually on a scale from no experience or preparation to excellent
43
experience or preparation (Clement, 2013; Cranston, 2012; Rutledge et al., 2008; Schumacher et
al., 2015). Additionally, to ensure consistency, the interviewer should balance the control of the
interview so that all questions receive a thorough response, but not so much as to give away the
expected answers to the questions (Timmons, 1998).
As principals have been shown to focus on personal qualities such as enthusiasm and
communication (Rutledge et al., 2008), and likeability (Schumacher et al., 2015), and as such
characteristics are more likely noted in interviews, other data collection e.g. demonstration
lessons are noted widely adopted techniques (Hoffman, 1995).
Comparing the hiring records of new principals to those of his/her predecessors,
individual principals found to be more likely to hire effective teachers did so consistently (Papa
& Baxter, 2008). The study also found principals shown to be less able to hire effective teachers
demonstrated poor selections consistently. Further, effectiveness in hiring was not found to be
tied to the school’s urbanicity or the characteristics of the principals such as age, gender, or
experience (Papa & Baxter, 2008). By comparing principal hiring practices to those of
predecessors in the same schools, the importance of individual principal competency is
highlighted as is the need to understand individual principals’ experiences in the hiring process.
Bounded rationality research (Kahneman, 2011; Langlois, 1990; H. A. Simon, 1982; H.
A. Simon, 1991; Sunstein et al., 2002) highlights the detritus effect normal personalities have on
decision-making. One example of this is that leaders tend to hire those who they believe to be
like them or those who are willing to agree with them (Singh, 2008). Over time, this hiring
pattern can negatively impact the decision-making process of the institution as more “yes men”
are assembled. What may appear to be a majority opinion garnered through a logical decision-
making process might just be the predictable result of polling likeminded individuals. Studies of
44
narcissism and its impact on decision-making habits show overconfidence, or the inflated
subjective probability of certain outcomes, leads to a willingness to continue to make losing bets
regardless of the diminishing returns (Campbell et al., 2004). In one of the multiple studies
conducted by Campbell (2004), narcissists are more likely to make risky bets and have lower
performance on betting tasks. Additionally, although no better answering trivia questions, they
rated their confidence in their accuracy to be higher than their peers. The “don’t confuse me with
data, I know what I am doing” attitude, is not surprisingly attributed to narcissists, but can be
noted in decisions that are based on “gut” rather than careful examination (Oppenheimer, 2012 p.
239). Kendrick and Olson (2012) suggest anyone who possess feelings of expertise in their field,
school administrators would fit this category, and will likely view their intuitions as valid
sources of knowledge. The fast thinking illustrated by Kahneman (2011) is reinforced by a
feeling of correctness. It is the ease with which people are drawn to the erroneous answer in the
ball and bat test question (Frederick, 2005) that simultaneously generates a feeling of gut
confidence and lowering of engagement of the slow thinking rational mind (Kahneman, 2003).
Summary. Researchers such as Engel & Cannata (2015) indicate some achievement gaps
between high achieving and low achieving schools are tied to hiring practices. Perhaps due to the
higher turnover associated with lower achieving schools, there is a tendency to use shorter
interviews, hire staff from within the school and near or after the start of the school year, rather
than conducting full rounds of the interview process (Engel & Finch, 2015; Papa & Baxter,
2008). In the conclusion of their study, Rutledge, et al (2008) suggest an area of study needed is
evaluation of the process and tools used in teacher hiring. Protocols such as the use of interview
questions structured around domains of effective teaching have been shown to offer superior
hiring decisions, yet great variety in teacher hiring still persists.
45
In addition to hiring based on likeability, politics and nepotism is a far-reaching context
that impacts hiring and retention decisions (Major, 2013). Some hiring decisions are not based on
demonstrated teacher effectiveness, but instead teacher candidate’s ability to coach athletics or
connections within the community (Murnane & Steele, 2007). Organizations are political
systems in that these are a collection of people with differing goals and ideals (Eisenhardt &
Zbaracki, 1992) and therefore decision-making processes within organizations are influenced by
more than just the available data. Cranston (2012) suggests “administrators at all levels re-
consider their current hiring practices because although intuitive in appeal, they may lack an
evidentiary and/or theoretical basis that is consistent with reliable and valid metrics that help
identify potentially effective teachers” ( p. 360).
Conclusion
It is known that teacher hiring is one of the most important tasks for principals (Clement,
2013; Mason & Schroeder, 2010; Peterson, 2002; Rutledge et al., 2008). However, it is also
known, that rational thinking is not the human default position (Kahneman & Tversky, 1984; H.
A. Simon, 1955; M. Toplak et al., 2011). Finally, it is known that protocols and checklists can
improve expert decision-making (Flottorp et al., 2013; Haynes et al., 2009; Kahneman, 2003;
Stubenrauch et al., 2012; Takala et al., 2011). What is not known is whether the specific biases
and heuristics noted in the decision-making of experts in other fields can be identified in the
hiring practices of educational experts and the degree to which principals express the need or
interest in decision-making tools for the hiring process.
“Organizations need to realize that a disciplined decision-making process, not individual
genius, is the key to a sound strategy. And they will have to create a culture of open debate in
46
which such processes can flourish” (Kahneman et al., 2011 p. 60). This research is needed to
understand Kahneman’s assertion in the context of educational organizations.
The impact of bounded rationality on decision-making and the use of decision-making
tools in fields such as business and medicine have been deeply studied. What is missing from the
literature is a deep understanding of the impact of bounded rationality on teacher hiring.
Research central to the theoretical framework of bounded suggests that the use of structured
protocols that address pitfalls in human thinking would improve the rationality in hiring
decisions (Bingham & Haleblian, 2012; Guerra-López & Blake, 2011; M. Toplak et al., 2011).
The impact of bounded rationality on even the best case scenario for teacher hiring practices
requires the attention of educational leaders. Suboptimal decision-making caused by missing
data, the misinterpretation of data, or overconfidence generated by the presence of some data
could undermine the crucial work of the hiring process and, therefore, the quality of instruction
provided to students.
In keeping with the recommendation that qualitative research be contextualized and
focused on the interactions within that context (Creswell, 2012), the proposed setting for this
study is within K- 8 school districts. Chapter three contains the methods for conducting this
research of the process of hiring new teachers among comparable school districts in the mid-
Atlantic region.
47
Chapter III: Research Design
Methodology
The purpose of this study of educational leaders in mid-Atlantic school districts is to
understand the process of teacher hiring and to answer the following research question: How
does bounded rationality affect administrators’ decision-making processes during the process of
hiring new teachers? This researcher explored the challenges involved in identifying effective
teachers during the hiring process. The researcher studied this question using a qualitative design.
In this study, the researcher sought to understand administrators’ beliefs and actions related to
hiring and examined what they choose to do and what they choose not to do as well as their
explanations as to the reasons for those choices. Careful design is essential in order to gain
control in social science research (Shadish & Cook, 1999). This chapter details the methodology
of the case study designed for this purpose.
Research Design
Whereas quantitative research offers measures of variables, qualitative research examines
the expressed thoughts and beliefs of the participants (Creswell, 2012). In this study, the
participants were school administrators, in particular, elementary school principals. Qualitative
methodology offers understanding of the perceptions and experiences of the participants and the
understanding of a phenomenon for which variables are unknown or cannot be manipulated
(Creswell, 2012) and is therefore the appropriate methodology for this research study into the
decision-making process of individual administrators.
Creswell (2012, p.19) lists three factors for consideration when deciding if a research
design should be quantitative or qualitative. The first consideration is the type of research
question. Quantitative research is used to identify trends or explanations, whereas qualitative
48
research is used to generate deep understanding of problems (Creswell, 2012). This research
question addressed the decision-making of individual administrators and was therefore suitable
for a qualitative design. The second consideration, according to Creswell (2012), is the selection
of a research design which will be more readily consumed by the intended audience. This
research was intended to inform the practice of administrators; as a qualitative study that gives
voice to a group of administrators, it may be more readily accepted by other administrators.
Finally, the selected research design should draw on a strength developed through the experience
and training of the researcher (Creswell, 2012).
In addition, decades of quantitative research have already been conducted in the area of
the applied theoretical framework, bounded rationality (Botterill & Hindmoor, 2012; Eisenhardt
& Zbaracki, 1992; Halevy & Chou, 2014; H. Simon, 2000; H. A. Simon, 1982), yet there are few
qualitative investigations. Jenlink (2005) argued that “The scholar-practitioner understands that
theory has a practical intent” (p. 7). This researcher seeks to utilize the theoretical framework of
bounded rationality to serve the practical and crucial work of teacher hiring and extend it by
investigating the biases and heuristics expressed by administrators as they reflect on their
choices.
Furthermore, bounded rationality, the theoretical framework utilized as the lens for this
study, is replete with decades of experimental studies (Creswell, 2012, p. 21), which are
quantitative procedures in which the investigator determines whether changes in variables make
a difference in results for participants. These experimental studies have identified biases and
heuristics in bounded rationality and have offered means to ameliorate these in various fields
(Kahneman, 2011; Sunstein & Thaler, 2003). This researcher elected to utilize a different
approach, a case study model, in order to understand the expressed experiences of decision-
49
makers, particularly of administrators in the teacher hiring process. This type of qualitative case
study approach of the hiring practices of administrators through the lens of bounded rationality is
missing in the literature. Case study accommodates many factors through the review of multiple
sources of evidence, and “benefits from the prior development of theoretical propositions to
guide data collection and analysis” (Yin, 2014, p. 17).
Research Tradition
Originally confused with quasi-experimental design (Shadish & Cook, 1999), case study
is now recognized as a separate method with its own research design and, as such, it offers a
structure with which to select the questions to study, the relevant data to collect, and the way to
collect and analyze data (Yin, 2014). Case study is not confined to specific epistemological
perspectives; either interpretivist or positivist orientations can be applied (Yin, 2014).
Bounded rationality, as a framework, is in opposition to a positivist view that there is a
singular reality and that humans use rational thought in response to the singular reality.
However, bounded rationality is not an interpretivist philosophy either; one in which every
reality is accurate. Rather, when making decisions, humans are likely to misunderstand the
reality that surrounds these decisions. Bounded rationality, therefore, offers a positivist
understanding of the world, but with expectation that humans will not behave in a predictably
rational manner and that individual interpretations of reality will lead to individual decisions
that are not fully rational. According to Schramm (1971), “the central tendency among all types
of case study, is that it tries to illuminate decisions or sets of decision, why they were taken,
how they were implemented, and with what result” (citing Schramm, 1971, Yin, 2014, p. 15).
In this study, the researcher examined the how and why of decisions made in the hiring process.
50
Case studies are used to understand real-world experiences in their context (Yin, 2014, p.
16). As an empirical inquiry that offers a means to examine the boundaries between
phenomenon and context (Yin & Davis, 2007), case study allows this researcher to study
differences and consistencies in individual decision-making across settings. This researcher
investigated individual decision-makers engaged in the process of hiring teachers, an event that
occurs across various schools and districts, each in its own context.
According to Yin (2014), “Case studies address how and why research questions which
focus on contemporary events without behavioral or experimental control of those events” (p.
9). Similar to a history, case study examines past events, but with the addition of observation
and interviews of those involved in the studied events (Creswell, 2012, p. 10). This case study
allowed the researcher to examine the experience of teacher hiring without altering the behavior
of the participants, and therefore, the employment outcomes of actual teachers.
The research tradition of case study, a common case study (Yin, 2014, p. 52), in
particular, provides the preferred design for the research direction of this study of recent
experiences with hiring teachers in sample schools. A common case study examines an
everyday situation, not outliers as in an unusual case study (Yin, 2014, p. 52), rather, this
approach studies the manner in which experiences transpire most often. This approach was
applicable “because of the lessons it might provide about the social processes related to some
theoretical interest” (Yin, 2014, p. 52).
This case study of multiple school administrators offers deep insight into the thinking of
individuals across contexts. This allows for understanding of the decision-making process of
educational leaders as the inclusion of multiple administrators reduces the limitation of single
case studies: the influence of localized thinking and action.
51
Participants
Whereas quantitative research often includes random sampling to identify participants
and sites, purposeful sampling is used in qualitative research to identify the participants and
sites determined to be most helpful to understand the central phenomenon (Creswell, 2012, p.
205). This researcher utilized an initial survey to identify interested participants who were
representative, or typical, of the larger study group, elementary school administrators. The
sites and individuals selected represent varied approaches to teacher hiring, from informal
decision-making to the use of formal protocols. Such typical sampling is one of the
subcategories of qualitative purposeful sampling recognized by (Creswell, 2012, p. 207).
Typical sampling is used in order to understand what is typical or normal; in this case, within
the hiring practices of elementary schools, thus, it is not random. The design of this study did
not require limiting the participant pool by demographics or experience, as bounded
rationality impacts all decision makers (Kahneman, 1984). However, the preferred
participants were administrators who annually screen paper applications, conduct interviews,
and make hiring recommendations to the superintendent. Five principals were interviewed.
All had more than five years of experience hiring teachers. The inclusion of administrators
from multiple districts and schools allows for examination of any contextual impact and
deepen understanding of the decision-making of elementary school administrators in the
process of hiring teachers.
In keeping with the recommendation that qualitative research was contextualized and
focused on the interactions within that context (Creswell, 2012), the setting for this case
study was a sampling of schools in a county in a mid-Atlantic state. The researcher
purposefully selected administrators whose responses to the initial survey represent the range
52
of protocols used in the hiring process, ranging from those without formal procedures to
those with consistently utilized question scripts and scoring systems. This researcher
examined the phenomenon of teacher hiring through a conceptual framework of bounded
rationality, for as Yin (2014) asserts, “The case study is an opportunity to shed empirical light
about some theoretical concepts or principles” (Yin, 2014, p. 40). This understanding was
represented in a sample of a minimum of three sites and five people in keeping with the
typical range of sample sizes in qualitative research suggested by Creswell (2012, p. 209).
Several considerations help establish transferability (Yin, 2014, p. 48). The first
consideration is the degree to which the participants are representative of the larger group and
the range of available documentation. This is essential in order to transfer findings to
individuals in the larger group. The participants in this study were selected utilizing
purposeful, typical selection, which gathers information on representatives of a larger group
(Creswell, 2012, p. 208).
Another important consideration in qualitative studies is the degree to which the
observer’s presence affects what people say or do (Creswell, 2012, p. 436). This researcher
does not hold a position of authority over the participants, so that form of researcher
influence is eliminated.
Recruitment and Access
The gatekeepers (Creswell, 2012), those with the ability to grant or deny access to the
school, in this study of administrators are the participants themselves. As a member of the
larger group of administrators loosely organized under the county education office, after
approval from the district superintendents, the researcher emailed the principals a detailed
invitation to participate in this study. The included a survey (Appendix A) used to gather
53
information on the interested participants, most importantly, whether and to what extent they
are involved in the hiring of teachers.
The recruitment process followed standard practice for qualitative research (Agee,
2009; Butin, 2010; Ponterotto, 2005; Ryan, Coughlan, & Cronin, 2007). This standard practice
included written consent for a survey as well as to participate in the study as guided by the
University of Northeastern IRB. After IRB and superintendent approval, the survey was
distributed via email to elementary school principals in a particular county in a mid-Atlantic
state. Submitted surveys were analyzed so that the research sampled a variety of location and
hiring processes. The researcher selected principals with ranging years of experience, as well as
protocols ranging from established district-wide hiring procedures to individual autonomy.
Selected participants were interviewed at a time and place convenient to them. Protection of
human subjects included voluntary participation, anonymity of the participants and schools,
ability to opt out at any time, and the surveyed and interviewed participants unrelated to the
school district in which the researcher is employed.
54
Data Collection
Figure 1. Data collection process
This study included data gathered through a survey of administrators from a particular
county in a mid-Atlantic state, as well as interviews of the five members selected from that pool of
respondents, and a focus group of members within that smaller group held after the individual
interviews. Yin cites four principles for data collection in a case study: use of multiple sources of
evidence, creation of a case study database, maintenance of a chain of evidence, and care of
electronic data (2014, p.118). This section details the data collection and maintenance of this
study.
Refinement of instrumentation and coding scheme
Data Set 2: Interview analysis, peer review, and member checking
Synthesis of themes
Findings Conclusions Recommendations
Survey review: identify qualifying responses and selection of representative participants
Data Collection Phase 1: Survey
Data Set 1: Survey results analysis, identify typical and range of systems
Data Collection Phase 2: Interviews with selected participants
Data Collection Phase 3: Focus Group with selected participants
55
Data set 1 – Survey of principals (Appendix D). Data was collected through a nine
question Likert scale survey emailed to the pool of participants. Data collected include the
principals’ years of experience as well as the protocols, if any, used in the process of reviewing
applications and interviewing teacher candidates. Further, this data includes the administrators’
expressed beliefs as to the degree to which evidence impacts their final candidate selection. The
survey informed the researcher as to the range of processes and procedures elementary school
principals utilize to interview teacher candidates. This information first served to establish the
pool of candidates and the application of purposeful selection. This information also included the
range of procedural routines utilized and, therefore, established what is typical among these
administrators. Understanding what is typical is needed for the transferability of the findings of a
study (Yin, 2014, p. 48).
The survey, collected through Survey Monkey (Appendix D), permission request
(Appendix A), and participant recruitment letter (Appendix B) were all be distributed via email
directly to principals of a particular county in a mid-Atlantic state. Participants were notified
from the first contact requesting participation the right to refuse to answer any question and to
withdraw from the study at any point. The survey was used to collect information regarding
hiring practices, data set 1, as well as generate the purposeful range of representatives from the
pool of participants for interviews, data set 2. Principals were selected from the range of novice
to experienced as well as the range of use of formal protocols.
Data set 2– Principal interviews (Appendix E). The second data set, the semi-
structured interviews, informed the researcher of the administrators’ decision-making in the
process of hiring of teacher candidates. Specifically, the interviews allowed understanding of
the administrators’ rationale for selection of data points in the processes of screening and
56
interviewing. This data set is the primary set as it is ideal for understanding the individuals’
experiences within a context of a group or system (Creswell, 2012). Whereas the first data set
offers understanding as to the typical established procedure for teacher hiring, these interviews
allowed for understanding of the actual interpretation and use of the procedures as experienced
by administrators.
The individual semi-structured interviews with the principals focused the research on
each principal’s decision-making process during the review of applications and teacher
interviews as well as his/her evaluation of the hiring protocols utilized. Semi-structured
interviews (Creswell, 2012, p.17) were conducted and audio recorded by the researcher
following the researcher’s designed semi-structured interview protocol (Appendix E). Two sets
of principals from two different school district in addition to a principal from a third district
were included in the study in order to collect different perspectives on a process and to allow
for understanding of any contextual influence on decision-making. Each interview length was
approximately one hour and conducted at the location and time preferable to the interviewee.
Paper review. Application, or paper, review is the first step in the candidate selection
process. To understand the principals’ experience reviewing applications, specific questions
and probes were included in the initial survey, q. 3, and in the semi-structured, open-ended
questions interview questions, specifically q. 1, 3 and 9. These focus on the principals’ criteria
used to identify candidates as well as the principals’ reflection of the protocols they used. As
personnel documentations are protected, the researcher asked about the principals’ experience
of the application review, not the specific application materials.
Candidate interviews. This section of the interview process collected the case studies’
principals’ explanations of the criteria they apply when conducting teacher interviews. This
57
included the principals’ description of characteristics and experiences they seek as well as the
manner in which they conduct interviews including style and question content and the degree to
which regulated protocols are used by each principal and across the district.
General processes. This section of the interview process collected the case studies’
principals’ perceptions of their general procedures. This line of inquiry allowed for participants
to evaluate their overall use of hiring protocols and procedures.
After each interview, the researcher transcribed data through a service, Rev.com. The
interview audio files were converted into Microsoft Word files and delivered electronically and
password protected to the researcher who then uploaded the transcribed data into Nvivo, a
qualitative data analysis software application. This software access was also password protected,
as was all reflexive memos created by the researcher.
Data set 3 – Focus Group (Appendix F). As a third data set, a focus group of members
of the eight candidates was held. The themes identified by the researcher as a result of the
analyses of the survey and interviews were presented to the focus group for their consideration.
Focus groups can be an advantageous form of interviewing in a qualitative study particularly
when participants are similar to each other (Creswell, 2012). The participants in this study are all
elementary school principals in a particular county in a Mid-Atlantic state. As is true for
individual interviews, focus group researchers must be mindful of reflexivity (Yin, 2014, p. 112).
This influence dynamic between the researcher’s perspective and the interviewee – and vice
versa – in which the beliefs of one individual can impact the conversation and therefore the
expressed beliefs of others cannot be completely eliminated; however, Yin (2014), in keeping
with the Kahneman (1984) and the theoretical framework of bounded rationality, states
mindfulness to the potential interaction impact allows for improved interview accuracy.
58
All researchers must ensure participants are aware their interviews remain confidential; as
a member of the educational community studied, this researcher is all the more charged with
conveying that confidentiality be upheld. The Informed Consent form was revisited and initialed
at the start of each interview. The researcher collected interview data via written notation and
audio recording and the transcribing service, Rev.com. Probing sub-questions were utilized as
needed to generate elaboration or clarity. Draft transcriptions of interviews were presented to
participants for member checking.
Confidentiality was be protected in data collection, storage and publication. The name of
the state, county, and school districts was not published. Neither were the names of any
participants published. Pseudonyms were used for the schools and all participants.
Data Storage
The data sources for this research include survey results and interviews. All electronic
data was password protected. The password was only known to the researcher. Hard copy
documentation included informed consent agreements for survey data and interview data. Hard
copy documentation was stored in a locked office within a locked cabinet. Only the researcher
has the key to the cabinet. Electronic data included research notes from interviews, researcher
reflexive memos, communications between the researcher and participants. Only the researcher
viewed all data. Participants only had access to the data from their own survey and interview.
Participant confidentiality was retained through the use of naming conventions. Data will be
stored for five years beyond the publication of the dissertation and destroyed upon the conclusion
of that five-year period.
59
Data Analysis
High quality analysis must include all evidence, examine rival interpretations, focus on
the most significant aspect of the research, and utilize the prior knowledge of the researcher (Yin,
2014, p. 168). Therefore, this researcher applied his knowledge of teacher hiring while also
examining any biases his background may generate toward a specific explanation or away from
plausible rival interpretations.
As a qualitative study, this study included “breaking down qualitative data into discrete
parts, closely examining them, and comparing them for similarities and differences” (Strauss &
Corbin, 1998, p. 12; as cited by Saldana, 2009, p. 81). This section describes this researcher’s the
data analysis process.
This researcher engaged in coding as a method to inform analysis, as coding is “not just
labeling, it is linking” (Saldana, 2009 p. 8). Abbott (2004) compares coding to the decoration
of a room, including stepping back and reexamining for patterns and flow (as cited by Yin
2009, p. 9). First and second cycles of coding was utilized until refined categorization was
achieved. Recoding is to be expected in case study research (Yin, 2014). A theme is the
outcome of the analyses that occur during the process of coding, categorizing, and reflecting.
Aligned to the literature review for this study, themes related to the manner in which
decision makers were impacted by the mental shortcuts of bounded rationality and the manner
in which decision makers ameliorate the irrationality of fast thinking human minds. Table 1
lists the selected biases and heuristics identified within the theoretical framework and the
possible impact on hiring decisions.
60
Table 1
Biases and Heuristics and Possible Impact on Hiring Decisions
Biases and
heuristics Effects Possible impact on hiring decisions
Affect bias causes decisions to be based
on emotions or on
likes/dislikes
Is there evidence of teacher hiring based on
adult to adult likeability rather than teacher
efficacy?
Anchoring causes the weighing of one
piece of information too
heavily, often the first
information encountered
Is there evidence of committees selecting
candidates with shortcomings related to the
abilities as teachers that should have
prevented their selection?
Availability causes overestimation of the
likelihood of events when
similar past events come to
mind
Is there evidence of committees selecting
candidates based on the administrator’s
specific knowledge base rather than from a
full range of needed information?
Confirmation
Bias
causes the searching for
evidence that supports belief
and ignoring evidence that
contradicts it
Is there evidence of individuals selecting
candidates based on the one teaching
domain his/her preferred candidate most
demonstrated and ignore the evidence from
the other teaching domains?
Missing Data
Bias
is the tendency to ignore the
lack of available data
needed for rational decisions
Is there evidence of committees selecting
candidates about whom little is known over
those better known?
Prediction
Overconfidence
is the subjective confidence
level that is not supported
by objectivity
Is there evidence of committees selecting
candidates based on unsubstantiated
predictive tools such as experience as a
coach will cause the candidate to be a
better teacher than those who have not
coached?
Sunk Cost
Effect / Loss
Aversion
is the phenomenon in which
people justify increased
investment of time or
resources into a decision,
based on the cumulative
prior effort
Is there evidence of committees selecting
candidates who at the end of a lengthy
screening process may not be strong
candidates, but the administrators resist
reopening the hiring process due to the
time already invested in finding the first
candidate?
Table 2 overviews the analysis of each of the three data sets. Data set 1, the survey, were
considered through a quantitative analysis of the Likert scale and multiple choice responses. Data
61
sets 2 and 3 were coded in two cycles each beginning through Pattern matching and through In
vivo coding.
Table 2
Summary of Data Collection and Analysis
Research Question Data Source Analysis
How does bounded
rationality affect
administrators’ decision-
making processes in the
process of hiring new
teachers among comparable
school districts in the mid-
Atlantic region?
1. Data set 1 -
Administrator survey
1. Cycle 1: quantitative
analysis of Likert
responses
2. Data set 2– Case study
interviews
3. Data set 3 – Focus group
interview
2. Cycle 1: Pattern Code
Cycle 2: In vivo Code
3. Cycle 1: Pattern Code
Cycle 2: In vivo Code
Data set 1: Survey
The data set of responses gathered through the survey questions (Table 3) inform the
research question by determining the hiring processes of principals across many districts in a
particular county of a mid-Atlantic state. In addition to framing the procedures in practice,
access to the responses expand the study’s information beyond that of the districts selected for
deeper examination. The data collected during the interviews was compared to the individual’s
responses in the survey. The survey gathered information on the participants’ years of
experience and employment setting. During the interviews, each participant was asked if s/he
believed specific bias and heuristics might impact his/her decision-making in the process of
teacher hiring. The survey, therefore gathers information on the quantifiable frames of reference
from which the participants drew upon in the interviews.
62
Table 3
Survey Questions
1. How many years have you been a principal?
a. 1-5
b. 6-9
c. 10+
2. Approximately, how many teachers do you hire in a year?
a. 1-5
b. 6-9
c. 10+
3. How important are the following sources of information when reviewing teacher
applications? 1= somewhat, 2 = important, 3 = very important, N/A
a. Certifications
b. In-district application forms
c. Resumes
d. Transcripts
e. Recommendations, formal or informal
f. Prior knowledge of candidate
4. To what degree are teachers hired through screening committees?
a. Never
b. Sometimes
c. Usually
d. Almost always
e. Always
5. If so, who is included in these committees?
a. Administrators
b. Teachers
c. Parents/community members
d. Other _________________________
6. To what degree are interview protocols or processes (the list of questions, time frames and
manner of interviews, etc.) used for all teacher candidates?
a. Never
b. Sometimes
c. Usually
d. Almost always
e. Always
7. If so, to what degree are those protocols and processes used by all administrators in your
district?
a. Never
b. Sometimes
c. Usually
d. Almost always
e. Always
63
8. Would you be willing to be a participant of this study engaging in an hour long interview on
this topic as described in the accompanying Request for Participation? You may rescind
your agreement to participate at any time.
a. Yes (Please provide your email here__________)
b. No
c. Contact me first as I have questions regarding participation
9. If so, would you also be willing to be a member of a focus group of up to 8 principals who
were also members of the study to discuss the preliminary themes identified in the study.
This group would meet for 1 hour in a location central to the county participants.
a. Yes
b. No
c. Contact me first as I have questions regarding participation
Data set 1, the survey, was utilized to identify participants for the study as questions 8
and 9 specifically asked if the responder were willing to participate in the interviews and focus
group, respectively. The survey questions were designed to illuminate the most relevant aspects
of the theoretical framework. Table 4 lists the relationship between the survey questions, the
research question and the theoretical framework.
Table 4
Survey Questions to Components of TF
The research question: How does bounded rationality affect administrators’ decision-making
processes in the process of hiring new teachers among comparable school districts in the mid-
Atlantic region?
Survey Questions
Relation to research
question
Relation to components of
theoretical framework
1. How many years have you
been a principal?
Allows for comparison of
years of experience as
principals to valuation and
processes
TF holds biases and
heuristics are not reduced
by experience
2. Approximately, how many
teachers do you hire in a
year?
Allows for comparison of
experience to valuation and
processes
TF holds biases and
heuristics are not reduced
by experience
3. How important are the
following sources of
information when reviewing
teacher applications?
Allows understanding of
data deemed important by
principals
TF – confirmation biases
causes individuals to
concentrate on data that
supports preference
64
4. To what degree are teachers
hired through screening
committees?
Allows for understanding of
the range and specifics of
hiring processes
TF –Monitoring of formal
protocols reduces biases and
heuristics
5. If so, who is included in
these committees? Allows for understanding of
the range and specifics of
hiring processes
TF –Monitoring of formal
protocols reduces biases and
heuristics
6. To what degree are interview
protocols or processes used
for all teacher candidates?
Allows for understanding of
the range and specifics of
hiring processes
TF – Formal protocols
reduce biases and heuristics
7. If so, to what degree are
those protocols and processes
used by all administrators in
your district?
Allows for understanding of
the range and specifics of
hiring processes
TF – Formal protocols
reduce biases and heuristics
The collected data allowed the researcher to select a representative group that ranges in
response to the questions as to the hiring processes utilized. Further, this data set was
quantified to serve as a reference point as the range of hiring approaches utilized. The
responses collected through Survey Monkey were quantitatively analyzed. Central tendency
was identified through calculation of the mean.
Data Set 2: Interviews
The second data set was gathered in the in-depth interviews. This data, clarified through
analysis, the degree to which the decision-making as expressed by the five principals was
impacted by bounded rationality. Specifically, the analyses focused on the degree to which
intuition or fast thinking and rational or slow thinking (Kahneman, 2011) was noted in the
administrators’ expressed valuation of certain data in the hiring process as they consider the
impact of biases and heuristics, e.g. anchoring, confirmation, and halo effect, on the hiring
process. This data set included the principals’ paper review and interviewing processes.
65
The researcher designed semi-structured interview protocol explored the administrators’
hiring procedures and protocols so that the research question, how does bounded rationality
affect administrators’ decision-making processes during the process of hiring new teachers,
may be analyzed. Each of the questions offer opportunities to code for themes related to the
specific biases and heuristics identified by bounded rationality as well as the general
proposition of the theory that humans are not as rational as they believe themselves to be. The
analysis of the administrators’ responses may offer understanding of the degree to which the
goal of rational decision-making is undercut by biases and heuristics. One such instance
considered was when administrators or committee members ignore data that counters their initial
choice as this might demonstrate anchoring.
The purpose of the interviews was to understand the principals’ expressed experiences of
the teacher higher process so the framework of bounded rationality may be applied for analysis
and interpretation. This may improve understanding of the hiring process in order to identify the
most appropriate candidate for the particular teaching position. Through the purposeful selection
of principals who use formal protocols and those who do not, the impact of protocols, which are
suggested in research, was studied in practice. The overlay of the theoretical framework
examines the degree to which bounded rationality can be ameliorated through such decision-
making tools.
Therefore, the researcher’s questions targeted the two main components of the teacher
hiring process: paper screening, and interviewing. Each section of questions was designed to
elicit the administrators’ understanding of his/her protocol and, applying the lens of bounded
rationality, the degree to which the administrator’s expressed goal of each step of the process
matches his/her expressed basis for the final hiring selection. For example, it may be expected
66
that transcripts would have comparative weight during the paper screening phase of the hiring
process as it is a measure of four years of academic effort on the part of the candidate. The
interview question 1, “What is most important to you in a teacher’s submitted application?”
was included so that administrators would indicate their valuation of the readily accessible
transcript criteria. This was compared to question 3, “What do you look for during teacher
interviews?” This comparison gave insight into the degree which administrators value
educational background, against other factors such as interviewing skills. The researcher
examined if university transcripts and other factors used to value one candidate over another
remained as decisive criteria once principals meet the candidates.
Table 5 details the relationships among the research question, the interview questions,
the theoretical framework, coding, and possible themes anticipated. The expected themes and
codes derived from the questions were in keeping with the tenets of bounded rationality. Each
question and prompts explores an aspect of teacher hiring likely to be impacted by a specific
bias or heuristic. Question 1a asks: “What is most important to you in a teacher’s submitted
application?” Responses to this question were expected to elicit responses in the areas of
expertise. Principals may reference their past experiences in locating good teachers and
corollary evidence located in applications. The principals stated that certain elements in
transcripts, work experience, or references accurately and quickly identify good candidates.
First cycle coding organized such statements as “expertise.” Second cycle coding furthered such
statements through in In vivo coding as “I was impressed by the level of teaching experience.”
The data gathered from the semi-structured interviews was coded first through pattern
coding and generate codes such as (Ex) Expertise, (I) Intuition, (T) Time constraint, and (Qu)
Quantify. Second cycle coding identified language related to judgments based on intuition or
67
reasoning and the time available for the decision-making process. Codes included “… got a
feeling,” “We discussed our scores,” “…I can tell by now,” and “Hope for the best.”
The two cycles of coding informed the degree to which the theoretical framework of
bounded rationality is applicable to the experience of principals in the process of hiring teachers.
Further, in keeping with the theoretical framework, questions 7 – 9 explored the degree to which
principals express a need for decision-making tools in the hiring process.
Table 5
Data Set 2 Interview Questions to Codes and Themes
Interview questions
and TF Biases and Heuristics Themes
Cycle 1
Pattern
Cycle 2
In vivo
1. (Anchoring/Halo Effect)
a. What is most important to you in a
teacher’s submitted application?
b. How does that help you find the most
effective teacher?
c. Have you ever had a hiring committee
member who held onto a first impression
(good or bad)? How did you manage
that?
Fast
Thinking
(Intuition)
Slow
Thinking
(Reason)
(Ex) Expertise
(I) Intuition
(T) Time
constraint
(Qu) Quantify
“Impressed by
level of…”
“… got a
feeling.”
“…had to get an
applicant before
the BOE
agenda…”
“We revisited
our matrix.”
2. (Missing Data Bias)
a. Do you find you have all of the
information you need to choose the best
candidate?
b. How do you manage the task of
considering all of the information?
Fast
Thinking
(Intuition)
(Ex) Expertise
(T) Time
constraint
“…I can tell by
now.”
“Have other
tasks to get to”
3. (Affect Bias)
a. What do you look for during teacher
interviews?
b. How does that help you find the most
effective teacher?
Fast
Thinking
(Intuition)
(L) Likeability
(Ex) Expertise
“A good fit
here.”
“You can tell
right away.”
68
4. (Confirmation Bias)
a. Have you ever had a committee member
select data as evidence of the preference
he/she had before the interviews began?
b. If so, how did you manage that?
Slow
Thinking
(Reason)
(Qu) Quantify
“We discussed
all of the
responses.”
5. (Availability Bias)
a. Have you ever had a committee member
express a dislike for a candidate because
the candidate reminded the committee
member of a previous poor hire
(graduated from the same university,
lived in the same town, etc.)?
Fast
Thinking
(Intuition)
Slow
Thinking
(Reason)
(L) Likeability
(L) Likeability
“We had a
teacher before
from that
program…”
“But so was
Jason.”
6. (Sunk Cost Effect / Loss Aversion)
a. What experiences have you had in which
after a long interview process, you did
not have a candidate that you were
confident in hiring?
b. Did you restart the hiring process? Why
or Why not?
Fast
Thinking
(Intuition)
Fast
Thinking
(Intuition)
(T) Time
constraint
(C) Confidence
“We already
spent hours.”
“This is the best
we will get
now.”
7. (Decision-making Tools)
a. Does your school/district utilize protocols
or processes for teacher hiring?
b. Do your processes improve your teacher
hiring? If so, how?
c. Do you wish it were different? If so,
how?
d. Expert decision makers such as surgeons
and pilots have been shown to improve
their decisions through the use of
checklists. Do you believe that might also
be true for the expert decision-making of
principals in the hiring process? Why or
Why not?
Slow
Thinking
(Reason)
Fast
Thinking
(Intuition)
Fast
Thinking
(Intuition)
Slow
Thinking
(Reason)
(Qu)Quantify
(C) Confidence
(T) Time
constraint
(I) Intuition
(Ex) Expertise
“We calculate
each question
on a 4 point
scale.”
“It is
cumbersome.”
“I can just tell.”
“You know
right away.”
“Hiring is
different.”
8. (Decision-making Tools)
69
a. Do you always use committees in the
hiring process?
b. If so, how do they operate?
c. How are candidates ranked and selected?
d. Who makes the final decision, the
principal, or the committee?
e. Is someone charged with ensuring
protocols are followed in the process?
Fast
Thinking
(Intuition)
(T) Time
constraint
(Ex) Expertise
(C) Confidence
(Qu) Quantify
(Ex) Expertise
(Ex) Expertise
“…everyone’s
time”
“I just do it.”
“Perfectly.”
“…rank order”
“I do.”
“We are used to
it.”
9. (Decision-making Tools)
Consider your teacher hiring process, what is
working/what do you wish to change?
Slow
Thinking
(Reason)
Fast
Thinking
(Intuition)
(Qu) Quantify
(C) Confidence
(Ex) Expertise
(I) Intuition
“Wish we could
see them all
teach for more
time.”
“Wish it were
faster. You
know right
away.”
Data set 2 - Interviews first cycle coding
This first cycle coding of all interviews in which each principal was asked to describe
his/her processes of paper-review and interviews were conducted using Pattern coding.
Pattern matching (Yin, 2014, p. 143) was used for first cycle coding in order to compare
the identified patterns of administrators in the process of hiring teachers to the components of
bounded rationality. Pattern matching compares the finding from a case study with that of a
predicted pattern. In this case, it is predicted that suboptimal decision-making caused by missing
data, the misinterpretation of data, or overconfidence generated by the presence of some data as
70
established by the theoretical framework of bounded rationality could be found in the decision-
making process of hiring teacher candidates but go unnoticed by the decision makers.
In the first cycle, coding allowed for the emergence of expected themes. In the second
cycle, the researcher narrowed down or expand the themes, or codes, based on what emerged
from the first cycle. After revising the codes, patterns were identified. Evidence of and related
amelioration of seven biases and heuristics were sought: affect, anchoring, availability,
confirmation bias, missing data, prediction overconfidence, and sunk cost. However, the patterns
that emerge from the research differed. Additionally, rival explanations (Yin, 2014, p. 146) were
also sought as matter of practice of good research.
Data set 2 - Interviews second cycle coding
Second cycle coding refines the emerging categorical, thematic, and theoretical analyses
developed in the first cycle coding (Saldana, 2009). Second cycle coding reduced and expanded
the themes identified through the pattern codes produced during first cycle coding. Second cycle
coding condensed the number of codes and deeper analysis of the original plan of study
(Saldana, 2009). The CAQDAS program Nvivo assisted the researcher in this process by
allowing highlighting and coding of specific words or phrases. The researcher evaluated all of
the collected data as CAQDAS located exact phrasing; terms or expressions may be missed if
every interview is not carefully reviewed by the researcher. “In vivo coding is appropriate for
virtually all qualitative studies, but particularly for beginning qualitative researchers learning
how to code data, and studies that prioritize and honor the participant's voice” (Saldana, 2009, p.
75). In vivo coding allows for coding patterns of actual words and statements made by
participants, in this case study, those of principals. The theoretical framework used as lens for
this study involves the mental shortcuts which allow for rapid decision-making. In vivo coding in
71
this study analyzes the collected statements of five principals as they detailed their decision-
making processes in the hiring process. When describing candidates, common expressions such
as “good fit” or “very professional” were noted. These, in turn, were related to the elements of
the specific biases and heuristics detailed in the theoretical framework of bounded rationality.
Codes and themes evolved from the gathered language.
This method of coding uses the direct language of participants as codes rather than
researcher-generated words and phrases (Saldana 2009, p. 48) and were applicable to this
research. By allowing the focus to be on the actual language of the participants, researcher
intrusion was reduced. Quoting Charmaz (2006, p. 57), Saldana (2009) states “In vivo Codes
‘can provide a crucial check on whether you have grasped what is significant’ to the participant,
and may help crystallize and condense meanings” (p. 79). CAQDAS programs assist in
organizing and quantifying this process; however, it is the researcher who is in control of what is
coded and how the coded data is interpreted. Finally, In Vivo coding particularly assists in this
challenging work for beginning qualitative researchers learning how to code data (Saldana, 2009,
p. 74). Reflection through memo writing and second cycle coding coordinated and refined the
pattern codes created in first cycle coding.
Data Set 3: Focus Group
The third data set was collected by conducting a focus group. This group of volunteer
participants consisted of five principals individually interviewed for data set 2. The focus group
session was an open discussion of the principals’ reactions to the themes identified in data set 2
guided through a semi-structured discussion (Appendix F).
72
Data Set 3 – focus group cycle 1 coding
Cycle 1 coding of the focus group transcript was first through Pattern coding. Expected
themes included the defense of the present processes and the intuitive ability on the part of the
principals to find the best candidates quickly and effectively through the interview process.
Themes identified focused the In vivo coding to be utilized in cycle 2. Table 6 lists the
anticipated pattern codes such as (C) Confidence, (Ex) Experience, (I) Intuition, and (T) Time.
Data Set 3 – focus group cycle 2 coding
Cycle 2 In vivo coding identified in the language of the focus group transcript used to
refine the potential themes noted in cycle 1 theming. Specific language of the participants such
as “I just got a bad vibe from her” or “The candidate pool in this subject is limited” were coded
and organized in relation to the theoretical framework concepts of fast thinking, or intuition, and
slow thinking, or rational thought. Tables 6 lists the expected codes and themes from the focus
group interview.
Table 6
Data set 3 – focus group expected codes and themes
Focus Group Dialogue Starters Themes
Cycle 1
Pattern
Cycle 2
In vivo
1. In our first interview, we discussed the
process of selecting candidates to interview
from the pool of applicants. How do you
feel about success of that process?
Slow
Thinking
(Reason)
Fast
Thinking
(Intuition)
Fast
Thinking
(Intuition)
(C)
Confidence
(Ex)
Experience
(I) Intuition
(T) Time
constraint
(Ex)
“The
candidates
may take
other
positions
due to the
delay.”
“I just got a
bad vibe
from her.”
“The
2. Can you recall an experience when a
candidate seemed very promising on paper,
but not in the interview? What did he/she
do/not do in the interview?
3. When you come to the conclusion of the
73
interview process, do you find your
committee members rank the candidates in
the same order? If not, what do you do?
Fast
Thinking
(Intuition)
Experience
(T) Time
constraint
candidate
pool in this
subject is
limited.”
“We may
consider
group
interviews.
”
4. Are you interested in changing your
interview process? If so, in what ways?
Summary
The three data sets of this study allowed for triangulation (Creswell, 2012). Data
collection and analysis of the principal survey, individual interviews, and focus group combine
to increase the trustworthiness of the study. Coding and analysis of each data set served to
inform and evaluate the interpretation of the other two sets.
The data analysis of this study allows for the consideration of the similarities and
differences in the decision-making of principals during the process of hiring teachers. The
analysis utilizes approved research methods of coding, specifically, Pattern and In vivo, in order
to understand the impact, if any, of bounded rationality on the decision-making process of the
participants in this study.
Trustworthiness
This section details this study’s level of trustworthiness, the degree to which the
information matches what it is meant to represent (Yin, 2014, p. 47). Literal replication is the
prediction of similar results. Theoretical replication predicts contrasting results but for
predictable reasons (Yin, 2014, p. 57). Recording all steps of the study was done to a degree
allowing for replication. The intent of this study was to explore the decision-making of
principals in the teacher hiring process through the lens of bounded rationality (Kahneman,
74
2003). Trustworthiness consists of four components: credibility; transferability; dependability;
and confirmability (Lincoln & Guba, 1985).
Credibility. The first step to establish the credibility of a study is to employ recognized
research methods. This study followed the elements of qualitative design (Yin 2014). This
qualitative study includes sampling strategies, data collection, triangulation, pattern and In vivo
coding and analysis and theming. Credibility was maintained through the researcher’s
engagement with participants for an adequate amount of time to build the trust needed to engage
in deep interviews. Member checking (Creswell, 2012), the opportunity for participants to
examine the narrative as it has been summarized by the researcher, also increased credibility.
The study was conducted following the guidelines and approval of Northeastern University’s
IRB.
Transferability. Transferability, a study’s clarity and depth of description which allows
the study to be applicable to other contexts, was accomplished through the establishment of the
context of the study and the production of a full narrative of the transcription so the coding is
easily followed, apparent, and readily analyzed by the reader. As a study of the experiences of a
sample group of principals, this study is transferable to principals engaged in the process of
hiring teachers in other settings.
Dependability. Dependability is the ability to replicate research and yield similar results.
The researcher reported the research process clearly so that the study could be replicated. Further
as the participants are a representative sampling, similar participant pools are available for
repeated studies if needed.
Confirmability. Bracketing, the means in which the researcher examines his beliefs
toward the phenomenon (Tufford & Newman, 2012, p. 85), offers a means to reduce negative
75
researcher influence on the research process and can be conducted prior to and during data
gathering and analysis. Reflective commentary supports all aspects of trustworthiness including
confirmability. To establish confirmability, this researcher detailed his biases in the positionality
section of chapter one. In order to monitor his biases and actions during the study to prevent
those from impacting data collection and analysis, this researcher utilized journaling to document
interpretations. Being reflective (Creswell, 2012, p. 18; Tufford & Newman, 2012) in this
manner would expose and hopefully reduced the impact of biases through examination of the
researcher’s role or position in the study.
76
Chapter IV: Findings and Analysis
Introduction
The purpose of this qualitative study is to understand the teacher hiring practices of
educational leaders in K-8 public schools in a particular county in a mid-Atlantic state. Teacher
hiring practices include the screening, interviewing, and selecting of teacher applicants. In this
study, the researcher seeks to uncover administrators’ beliefs and actions related to hiring,
examine their hiring decisions, and understand their explanations as to the reasons for those
choices. The gathered data and analysis of the data is presented in this chapter.
Review of Research Problem
Of all the decisions made by school principals, the hiring of teachers may be the most
important task (Mason & Schroeder, 2010; Rowan, 1994a); however, the bounded rationality
involved in decision-making left unchecked by the supports of protocols, such as research-
supported screening criteria or interview procedures (Cranston, 2014), engenders an ongoing
problem of practice: sub-optimal decision-making in the process of teacher hiring.
Tasks such as teacher evaluation, curriculum development, or student discipline are often
conducted with expectations and procedures for consistency and interrater reliability; however,
the actual practice of teacher hiring varies greatly and is often “information poor,” and the
criteria for selection is often based on feeling over evidence (Liu & Johnson, 2006 p. 331).
Given the larger pool of teacher candidates than available openings (Clement, 2013) and the
importance of the selection (Engel & Finch, 2015; Stronge et al., 2011), it would be expected
that hiring practices would be deeply researched, carefully and consistently practiced. This is not
the case (Liu & Johnson, 2006). Little is known about the nuances of the process of teacher
hiring (Mason & Schroeder, 2010).
77
This examination of the hiring practices of K-8 educational leaders is viewed through the
theoretical framework of bounded rationality, which holds that rationality is limited by the
availability of information, time, and mental capacity (H. A. Simon, 1982). According to
bounded rationality, the mental shortcuts that allow humans to survive in times of crisis that
require quick, decisive action do not also generate predictable rational decision-making. As a
result of this natural mindset, the answers that come to mind first are deemed to be the best
answer by the thinker.
Biases and heuristics play a significant and often undetected role in the outcomes of
decisions (Hoppe & Kusterer, 2011; Hutchinson et al., 2010; Kahneman et al., 2011). Biases are
the natural tendency to think in certain ways (Kahneman et al., 2011). These patterns of thinking
impact the rationality of decisions. Heuristics are simple practices that quickly produce
seemingly adequate answers to difficult questions. Table 7 details the potential biases and
heuristics which may impact teacher hiring decisions.
78
Table 7
Biases and Heuristics and Possible Impact on Hiring Decisions
Biases and
heuristics Effects Possible impact on hiring decisions
Affect bias causes decisions to be based
on emotions or on
likes/dislikes
Is there evidence of teacher hiring based on
adult to adult likeability rather than teacher
efficacy?
Anchoring causes the weighing of one
piece of information too
heavily, often the first
information encountered
Is there evidence of committees selecting
candidates with shortcomings related to the
abilities as teachers that should have
prevented their selection?
Availability causes overestimation of the
likelihood of events when
similar past events come to
mind
Is there evidence of committees selecting
candidates based on the administrator’s
specific knowledge base rather than from a
full range of needed information?
Confirmation
Bias
causes the searching for
evidence that supports belief
and ignoring evidence that
contradicts it
Is there evidence of individuals selecting
candidates based on the one teaching
domain his/her preferred candidate most
demonstrated and ignore the evidence from
the other teaching domains?
Missing Data
Bias
is the tendency to ignore the
lack of available data
needed for rational decisions
Is there evidence of committees selecting
candidates about whom little is known over
those better known?
Prediction
Overconfidence
is the subjective confidence
level that is not supported
by objectivity
Is there evidence of committees selecting
candidates based on unsubstantiated
predictive tools such as experience as a
coach will cause the candidate to be a
better teacher than those who have not
coached?
Sunk Cost
Effect / Loss
Aversion
is the phenomenon in which
people justify increased
investment of time or
resources into a decision,
based on the cumulative
prior effort
Is there evidence of committees selecting
candidates who at the end of a lengthy
screening process may not be strong
candidates, but the administrators resist
reopening the hiring process due to the
time already invested in finding the first
candidate?
79
Recent research (Botterill & Hindmoor, 2012; Halevy & Chou, 2014; Kahneman et al.,
2011; Lovallo, D. & Sibony, O., 2010) explores the manner in which bounded rationality can be
discovered and reduced in order to improve the decision-making process. It has been shown that
business executives can reduce the impact of biases such as anchoring, weighing one piece of
information too heavily, and confirmation, ignoring evidence that contradicts one’s beliefs
(Kahneman et al., 2011).
Research on teacher hiring is limited, but researchers such as Clement; Rutledge; and
Schumacher (2013; 2008; 2015), offer possible improvements for the process. One example of a
researched technique to improve rationality and consistency in teacher hiring practices is the use
of structured interviews specifically aligned to the domains of teaching: planning, management,
instruction, and reflection (Clement, 2013). These protocols are helpful in selecting effective
teachers, but structured interviews are rarely used (Clement, 2009; Liu & Johnson, 2006;
Rutledge et al., 2008).
Review of Research Question
Despite the human shortcomings involved in processing data demonstrated through the
decades of research building the theoretical framework of bounded rationality (Botterill &
Hindmoor, 2012; Kahneman, 2003; H. A. Simon, 1982), in the field of education, there remains
a widely held belief in the possibility of rational, data-driven decisions in schools (Cooley et al.,
2006; Marzano et al., 2005; Reeves & Flach, 2011). The best practice era that has dominated the
world of teaching, which assumes a set of techniques can be applied to all cases to garner the
same effect, mirrors the zeal in which data-driven decisions are prescribed for educational
leaders. However, bounded rationality holds that regardless of the clarity of the available data,
intuitive, or fast thinking, undercuts rational thought (Kahneman & Tversky, 1984). When
80
individual perceptions filter the interpretation of data, even carefully organized data-driven
decision-making settings are jeopardized (M. E. Toplak et al., 2014). The research question for
this study, how does bounded rationality affect administrators’ decision-making processes in the
process of hiring new teachers among comparable school districts in the mid-Atlantic region, is
rooted in the theoretical framework.
Each survey and interview question was written to compare the bias and heuristics identified in
the theoretical framework with the responses of the principals surveyed and interviewed.
Data Collection
Data collection for this research began in September of 2018 with an online survey of
principals regarding their experiences in the teacher hiring process. Securing superintendent
approval prior to the distribution of the survey reduced the available pool of principals serving in
K-8 settings in the particular Mid-Atlantic county to twenty-one individuals.
Eleven of the principals (52%) completed the initial survey. Further, seven of the eleven (55%)
respondents indicated a willingness to participate in the follow up interviews.
Five principals, four women and one man, from three different districts were interviewed
in 1:1 settings. Two of the principals lead middle schools. Three lead elementary schools. Each
of these sessions lasted approximately one hour.
Two of the principals interviewed joined the one-hour focus group session. Figure 1, data
collection process, overviews the three data collection phases. In phase 1, the researcher
collected survey responses from principals which served as data points and also as a means to
identify interview participants. In phase 2, in-depth interviews provided the majority of the
collected data. This data was multiphase coded by the researcher using pattern matching and In
vivo coding. In phase 3, data collection included the conducting of a focus group, analysis of that
81
data and the triangulation of all three data sets. Data collection was completed in the manner
approved by the Northeastern IRB. The data collection process figure lists the data points:
survey, interviews, and focus group, along with the intermediary steps of researcher analysis.
Figure 1. Data collection process
Data Analysis
Of the four general analytic strategies (Yin, 2014. p.131), a linear analytic structure
relying on theoretical propositions is most applicable for this study. This study applies the
components of the theoretical framework of bounded rationality. The data collection questions,
coding, and theming were all oriented to specific biases and heuristics identified in the
theoretical framework.
Of the five analytic strategies (Yin, 2014, p.143), one of the most desirable for
trustworthiness in a qualitative study is pattern matching. The initial rounds of coding for this
study were oriented toward the comparison of the data collected and the anticipated patterns
Synthesis of themes
Findings Conclusions Recommendations
Survey review: identify qualifying responses and selection of participants
Data Collection Phase 1: Survey
Data Set 1: Survey results analysis, identify typical and range of systems
Refinement of instrumentation and coding scheme
Data Set 2: Interview analysis, peer review, and member checking
Data Collection Phase 2: Interviews with selected participants
Data Collection Phase 3: Focus Group with selected participants
82
established by bounded rationality. In vivo coding was used in second cycle coding to better
understand the significance of the pattern matching coding. The specific language of the
participants was examined in relationship to bounded rationality and the experimental research
that supports it. Based on bounded rationality, there is anticipated phrasing when individuals are
employing intuitive thinking in the hiring process. Phrases such as “good fit” “you just know,” “I
can just tell” are all examples of phrasing that are more likely produced by what Kahneman
refers to as fast thinking rather than the more rational and reflective slow thinking (2011). Table
8 overviews the analysis of each of the three data sets. Data set 1, the survey, was considered
through a quantitative analysis of the Likert scale and multiple choice responses. Data sets 2 and
3 were coded in two cycles: Pattern Matching and In vivo coding.
Table 8
Summary of Data Collection and Analysis
Research Question Data Source Analysis
How does bounded
rationality affect
administrators’ decision-
making processes in the
process of hiring new
teachers among comparable
school districts in the mid-
Atlantic region?
4. Data set 1 -
Administrator survey
4. Cycle 1: quantitative
analysis of Likert responses
5. Data set 2– 1:1
interviews
6. Data set 3 – Focus
group interview
5. Cycle 1: Pattern Code
Cycle 2: In vivo Code
6. Cycle 1: Pattern Code
Cycle 2: In vivo Code
Analysis of Survey Responses
Through multiple choice and Likert scale questions, principals were asked to self-report
on their experiences in hiring teachers. These questions were organized to illustrate the
participants’ work experience, data valuation, use of committees and protocols, as well as
willingness to participate in 1:1 interviews.
83
Participant experience. All of the survey respondents indicated they hire 1-5 teachers
per year, the minimum number required for inclusion in the study. The respondents’ years of
experience as principals were divided across the year ranges: 1-5 years for five principals
(45.45%); 6-9 years for two principals (18.18%), 10+ years for four principals (36.36%).
Applicant data valuation. Survey participants were asked to evaluate the degree of
importance for a number of data points in the paper screening process, detailed in Table 9. As
proper certification is required by the state regulations and statutes, certification was rated the
most important criteria for considering which teachers to consider for an interview. All of the
interviewed principals mentioned culling the candidate pool by this requirement first. As listed in
Table 9, application forms and transcripts were considered far less valuable than resumes,
recommendations, or prior knowledge of the candidate.
Table 9
Importance of applicant documents
Committee use. As detailed in Table 10, “Almost Always” yielded the highest response
rate (4 principals, 33%) to the question, “To what degree are teachers hired through screening
SOMEWHAT IMPORTANT VERY IMPORTANT N/A
Certifications 0.00%
0
8.33%
1 91.67%
11
0.00%
0
In-district application
forms
58.33%
7
16.67%
2
25.00%
3
0.00%
0
Resumes 8.33%
1
33.33%
4 58.33%
7
0.00%
0
Transcripts 50.00%
6
33.33%
4
16.67%
2
0.00%
0
Recommendations,
formal or informal
16.67%
2
25.00%
3 58.33%
7
0.00%
0
Prior knowledge of
candidate
16.67%
2
33.33%
4 41.67%
5
8.33%
1
84
committees?” Yet, the second highest response rate was “Never” (3 principals, 25%). When
used, committees are made up of administrators, and 50% of the time, are joined by teachers. On
some occasions, parents and school board members are included. All interviewed principals
indicated the superintendent was the final interview step and 2-4 candidates were usually brought
to that round.
Table 10
To what degree are teachers hired through screening committees?
Protocols. When asked, “To what degree are interview protocols or processes used for
all teacher candidates,” 92% of the respondents indicated either “Always” (58.33%) or “Almost
always” (36.36%). The affirmative response rate was also 92% for question 7, “If so, to what
degree are those protocols and processes used by all administrators in your district?”
Survey Summary. The five hours of transcripts of the 1:1 interviews comprise the
primary data set of this study. This data held the greatest volume of data to analyze. Further, as
ANSWER CHOICES RESPONSES
Never 25.00%
3
Sometimes 16.67%
2
Usually 8.33%
1
Almost always 33.33%
4
Always 16.67%
2
TOTAL 12
85
individual interviews, the principals thinking was less likely influenced or interrupted – which
could occur in a focus group.
Data gathered from the survey results demonstrated some similarities and a few
differences across the principals’ years of experience, school settings, applicant data valuation,
use of hiring committees and protocols. The full range of response choices were utilized by the
respondents, with the noted exceptions of certification importance and the number of teachers
hired annually. Certification importance was indicated as important or very important. Although
the option for teachers hired each year included 1-5, 6-10 and more, none of the respondents to
this survey indicated hiring more than one to five teachers annually.
Hiring rates. The hiring rate of one to five teachers per year informed the interview
session as the researcher initially anticipated sunk cost (Kahneman, 2011) reactions when
principals faced deadlines for hiring. Discovered through the survey, the rate of open positions
for those interviewed was relatively low at only one to five annually. Therefore, time factors
were not noted by the principals as a major concern. However, originally designed interview
questions were still utilized in the 1:1 interviews to explore the issue of time demands.
Data valuation. The survey results regarding data sources preferred by administrators
matched the research note in the literature (Cranston, 2012; Rutledge et al., 2008). Resumes were
preferred over transcripts. Originally designed 1:1 interview questions were utilized to examine
the principals’ reasons for this preference.
Committee use. The survey results included affirmative responses to the full range of
options from “Never” to “Always.” Although committee use is suggested in the literature
(Heneman & Milanowski, 2004; Timmons, 1998), respondents varied in their reported use. The
86
1:1 interviews investigated the reasons for this variance. Predicted considerations were principal
preference, superintendent preference, district size and staff availability.
Protocols. A high percentage of respondents express their hiring process as utilizing
protocols. The 1:1 interview questions allowed the researcher to explore how individual
principals define protocol usage in the hiring of teachers and the degree to which there is the
consistency of use recommended in the literature (Engel & Finch, 2015; Haskins & Loeb, 2007;
Heneman & Milanowski, 2004; Lange et al., 2012; Schumacher et al., 2015).
Analysis of Data from 1:1 Interviews
Interview questions were constructed to identify the specific biases and heuristics
involved in the decision making of principals when conducting teacher hiring processes. Table
11 includes the interview questions organized by elements of bounded rationality.
Table 11
Interview Questions Organized by Bounded Rationality Identified Biases and Heuristics
10. (Anchoring/Halo Effect )
d. What is most important to you in a teacher’s submitted application?
e. How does that help you find the most effective teacher?
f. Have you ever had a hiring committee member who held onto a first impression (good or bad)?
How did you manage that?
11. (Missing Data Bias)
c. Do you find you have all of the information you need to choose the best candidate?
d. How do you manage the task of considering all of the information?
12. (Affect Bias)
a. What do you look for during teacher interviews?
b. How does that help you find the most effective teacher?
87
13. (Confirmation Bias)
a. Have you ever had a committee member select data as evidence of the preference he/she had
before the interviews began?
b. If so, how did you manage that?
14. (Availability Bias)
a. Have you ever had a committee member express a dislike for a candidate because the candidate
reminded the committee member of a previous poor hire (graduated from the same university, lived
in the same town, etc.)?
15. (Sunk Cost Effect / Loss Aversion)
a. What experiences have you had in which after a long interview process, you did not have a
candidate that you were confident in hiring?
b. Did you restart the hiring process? Why or Why not?
16. (Decision Making Tools)
a. Does your school/district utilize protocols or processes for teacher hiring?
b. Do your processes improve your teacher hiring? If so, how?
c. Do you wish it were different? If so, how?
d. Expert decision makers such as surgeons and pilots have been shown to improve their decisions
through the use of checklists. Do you believe that might also be true for the expert decision making of
principals in the hiring process? Why or Why not?
17. (Decision Making Tools)
a. Do you always use committees in the hiring process?
b. If so, how do they operate?
c. How are candidates ranked and selected?
d. Who makes the final decision, the principal, or the committee?
e. Is someone charged with ensuring protocols are followed in the process?
18. (Decision Making Tools)
Consider your teacher hiring process, what is working/what do you wish to change?
Five individual interviews were conducted, transcribed, and analyzed. First cycle coding
was conducted with the use of CAQDAS, NVivo Pro in particular, and researcher analysis.
Pattern match coding of principal responses generated a number of themes. These themes were
organized by the researcher into the two main sections of bounded rationality as organized by
Kahneman (2011): fast thinking and slow thinking.
88
Identified Fast Thinking. Identified fast thinking includes expertise/confidence,
intuition/gut/fit, time constraints, likeability/background of candidate similar to administrator.
Expertise/confidence. Throughout the interviews, principals expressed their success in
identifying the best teaching candidate from among the pool of candidates. “I have been doing
this for a long time, so you just know. Also, you can tell because this staff is great.”
Intuition/gut/fit. Principals expressed being able to identify weak candidates almost
instantly, stating “It only takes a few minutes.” Additionally, principals expressed having a sense
of how well the candidate would fit in with the rest of the staff.
Time constraints. According to bounded rationality as expressed by Kahneman (2011),
quick thinking is essential to human survival in time sensitive decision making. In the research
study design period, this researcher speculated if the demands on administrator time along with
hiring timelines, which require board of education approval, transfers, and induction, would
force administrators to make hasty and therefore less rational decisions. The 1:1 interviews
revealed that although all the principals described situations in which they were rushed to make
last minute hires, they all preferred to take the time they felt was necessary to find the right
candidate. To the researcher’s question, “Have you ever come to the end of an interview process
without a candidate you wished to hire?” All shared experiences in which, although they had
already invested time which can yield a sunk cost bias, they re-advertised the position and started
over. The exceptions were emergency leave positions that required a certified teacher be
identified quickly or a position with a very limited pool of certified teachers.
Likeability/background of candidate similar to administrator. Principals expressed the
desire to locate teachers with backgrounds similar to themselves. A principal with an interest in
special education law stated that background in that field is something she is looking for in
89
teacher candidates. The middle school principals stated their desire to locate teachers with a wide
range of skill sets; the middle school principals had held careers in other fields and then were
certified as teachers through alternate route to teaching certification programs, so they expressed
the desire to locate candidates like themselves with work experience outside of teaching.
Identified Slow Thinking. Evidence from the interviews included the administrators’
cross checking of data, the use of protocols and procedures, and self-reflection, all of which are
suggested by bounded rationality as means to improve rationality.
Protocols. Principals shared the use of set questions to be asked of each candidate. This
is in line with the notion of decision making tools such as checklists used in surgery procedures.
Cross checking. Principals mentioned contacting references as a means to cross check
impressions generated in the interviews. Additionally, principals expressed the importance of
post-interview discussions with committee members: “Sometimes someone will point out
something you hadn’t considered.”
Self-reflection. Two of the principals made statements which demonstrated self-
reflection such as, “You hope you are right, but you never really know.” The bounded
rationality holds that the mindset of self-reflection is essential to improving rationality.
Second Cycle
In vivo coding of the transcriptions within pattern matched themes was conducted for the
second cycle of coding of the 1:1 interviews.
In vivo coding of transcripts.
Expertise. Differing views of expertise were identified in statements such as “Checklists
might help doctors, but teacher hiring is different” and the contrary view, “Checklists might help.
I would be interested to see that done.”
90
Confidence. Evidence of confidence in the accuracy of one’s decision making was
identified in statements such as, “The system is working because all of our teachers are good”
and “After doing this for 10 years, you know in the first few minutes.” Reflective thinking
related to confidence is demonstrated in statements such as “You never know. You make the
decision with the information you have and hope that it works out. If not, you have to let them go
at the end of the year,” and “It's always interesting. You roll the dice a lot of times. Somebody
can look good on paper and then be a disaster.”
Intuition/gut. Intuition was identified in phrases such as “You can just tell,” and “In
those letters, I'm always looking for what's not there.”
Time Constraint. Time constraint awareness was identified in “We would be able to be
more selective if we had more time.”
Likeability/Similarity to the administrator. This bias was detected in statements such as,
“You may be working with this person for 20 years” and “I had a lot of coursework in that area,
so I am looking for that. I had that background and I think it really helped me be a better teacher,
so I prefer that.”
Table 12 highlights the identified connections of data set 2, the 1:1 interviews, between
the interview questions and the themes of the bounded rationality. Both Cycle 1 and Cycle 2
analyses identified evidence of the themes of bounded rationality in the principals’ expressed
motivations for hiring decisions. In interview question 1, the researcher posed the sub question,
“What is most important to you in a teacher’s submitted application?” This question was
designed to investigate bounded rationality’s concept of fast thinking, in particular, the
likeability bias. Would principals use evidence of the teachers’ likeability in the interview
process instead of concentrating on evidence of teacher effectiveness? This researcher’s cycles
91
of theming identified a pattern of references to administrator expertise and corresponding In vivo
evidence of what administrators look for in teacher candidates. Table 12 includes this
comparative analysis for all of the data set 2 interview questions and sub questions.
Table 12
Data Set 2 Interview Questions to Codes and Themes
Interview questions
and correlated theoretical framework
biases and heuristics
Anticipated
Themes
Cycle 1
Identified
Pattern
Cycle 2
Identified
In vivo
1. (Anchoring/Halo Effect )
a. What is most important to you in a
teacher’s submitted application?
b. How does that help you find the most
effective teacher?
c. Have you ever had a hiring
committee member who held onto a first
impression (good or bad)? How did you
manage that?
Fast
Thinking
(Intuition)
Slow
Thinking
(Reason)
(Ex) Expertise
(I) Intuition
(T) Time
constraint
(Qu) Quantify
“I look for
candidates that will
get along with
students and
parents.”
“You can tell in the
first few minutes.”
“We readvertised.”
“We average the
matrix scores.”
2. (Missing Data Bias)
e. Do you find you have all of the
information you need to choose the best
candidate?
f. How do you manage the task of
considering all of the information?
Fast
Thinking
(Intuition)
(Ex) Expertise
(T) Time
constraint
“No, I have all the
information I
need.”
“…I can tell by
now.”
3. (Affect Bias)
a. What do you look for during teacher
interviews?
b. How does that help you find the most
effective teacher?
Fast
Thinking
(Intuition)
(L) Likeability
(Ex) Expertise
They would be a
good fit here.”
“They can learn the
content; they have
to like children.”
4. (Confirmation Bias)
a. Have you ever had a committee
Slow
Thinking
(Qu) Quantify
“I had a
superintendent
92
member select data as evidence of the
preference he/she had before the
interviews began?
b. If so, how did you manage that?
(Reason) ignored our
recommendations,
but we all tend to
agree with the new
superintendent.
“We discussed all
of the responses.”
5. (Availability Bias)
a. Have you ever had a committee
member express a dislike for a candidate
because the candidate reminded the
committee member of a previous poor
hire (graduated from the same university,
lived in the same town, etc.)?
Fast
Thinking
(Intuition)
Slow
Thinking
(Reason)
(L) Likeability
“From a particular
school, no.”
6. (Sunk Cost Effect / Loss Aversion)
a. What experiences have you had in
which after a long interview process, you
did not have a candidate that you were
confident in hiring?
b. Did you restart the hiring process?
Why or Why not?
Fast
Thinking
(Intuition)
Fast
Thinking
(Intuition)
(T) Time
constraint
(C) Confidence
“If we have time,
we will start over.”
“This is the best we
will get now.”
7. (Decision Making Tools)
a. Does your school/district utilize
protocols or processes for teacher hiring?
b. Do your processes improve your
teacher hiring? If so, how?
c. Do you wish it were different? If so,
how?
d. Expert decision makers such as
surgeons and pilots have been shown to
improve their decisions through the use
Slow
Thinking
(Reason)
Fast
Thinking
(Intuition)
Fast
Thinking
(Intuition)
Slow
Thinking
(Qu)Quantify
(C) Confidence
(T) Time
constraint
(I) Intuition
(Ex) Expertise
“We calculate each
question on a 4
point scale.”
“We don’t actually
use the scores. We
just rank them.”
“No, the proof is in
our staff.”
“You know right
away.”
“Hiring teachers is
different.”
93
of checklists. Do you believe that might
also be true for the expert decision
making of principals in the hiring
process? Why or Why not?
(Reason)
“That might be
interesting.”
8. (Decision Making Tools)
a. Do you always use committees in the
hiring process?
b. If so, how do they operate?
c. How are candidates ranked and
selected?
d. Who makes the final decision, the
principal, or the committee?
e. Is someone charged with ensuring
protocols are followed in the process?
Fast
Thinking
(Intuition)
(Ex) Expertise
(Ex) Expertise
(C) Confidence
(Qu) Quantify
(Ex) Expertise
(Ex) Expertise
“Just the
administrators in
the district.”
“I just do it.”
“Well.”
“We put them in
rank order”
“We come to a
consensus.”
“No, we just are
used to it now.”
9. (Decision Making Tools)
Consider your teacher hiring process,
what is working/what do you wish to
change?
Slow
Thinking
(Reason)
Fast
Thinking
(Intuition)
(Qu) Quantify
(C) Confidence
(Ex) Expertise
(I) Intuition
“Having the time
for model lessons.”
“We like the way
we do it.”
Summary of interviews. Data collected illustrating the decision making processes of
administrators was in keeping with the patterns predicted by bounded rationality. Pattern
matching and In vivo coding located the use of both fast and slow thinking (Kahneman, 2011) in
the statements of the principals. Biases, heuristics, and the slow thinking efforts to offset those
biases were all evident.
Principals expressed efforts to select the best candidate from the pool of applicants. The
researcher’s analyses of the 1:1 transcripts through coding and reflection identified patterns of
94
efforts to make the best possible decisions, examples of the slow thinking likely to improve those
decisions as well as fast thinking that may render those decisions biased by the biases and
heuristics of the specific principal.
Based upon the literature review, the researcher anticipated principal decisions based on
general candidate likeability. Rather the analysis of the transcripts indicated the pattern to be
more specific. Principals weren’t looking for what would be a generalized likeability, but rather,
what they liked in themselves as teachers. The principals expressed preferences as indicators of
teacher quality were those they saw in themselves or in some cases in the staff already employed
in the district. Principals with experience outside education valued that in teacher candidates.
Principals with backgrounds in curriculum and instruction, former curriculum supervisors,
focused on teacher candidates’ facility within the domains of teaching: planning, environment,
instruction, and professionalism (Danielson, 2013).
Patterns of confidence were found across the transcripts the candidates. Principals
expressed general confidence in their past decisions and present processes. However, bold
statements such as, “You can tell in a few minutes,” were mostly reserved for candidates that
would not be considered for the final round. There were examples shared of candidates selected
that did not turn out to be the teachers hoped for by the principals. The contracts of those
teachers were subsequently non-renewed. The general confidence of the principals were not
shaken by these instances.
As a result of the analysis of the 1:1 interviews, the researcher narrowed the focus group
line of questioning to principals’ conception of ideal candidates, their reflection on their
processes for teacher hiring and their willingness to consider decision-making tools to be added
95
to those processes. The next section of this chapter details the information gathered in the third
data set, the focus group.
Analysis of Focus Group Data
All five of the 1:1 interview participants expressed willingness to be members of the
focus group. Through a survey of their preferences for dates, times, and meeting location – face
to face or via Zoom video conferencing – a date, time, and Zoom meeting link was selected. Four
of the principals were scheduled to join the online conversation. Two joined the meeting as
scheduled. The researcher introduced the two participants, neither of whom knew the other
would be participating; however, the two principals work in nearby districts so knew each other
before this research began. Both principals interviewed in this focus group were experienced
educators who worked in multiple instructional and administrative leadership roles, participated
in numerous professional organizations and offered detailed responses to all questions.
The researcher asked each of the focus group questions. The interview quickly became a
conversation with the participants responding not only to the questions directed to them, but to
the responses offered by the other principal or to the literature or tentative findings shared by the
researcher. At the conclusion of the line of discussion on each question, the researcher proposed
the next question. This hour-long question and discussion process covered all proposed
questions. The transcription of this interview was recorded and transcribed using Rev.com
services.
Based on the analyses of the first two data sets, the originally proposed questions for the
focus group discussion were reconsidered by the researcher. For example, the responses to the
first survey, which included principals other than those who were later involved in the 1:1
interviews, indicated resumes were considered the most important of the choices offered with
96
58% of respondents indicating this as very important, but transcripts only received 16.7%.
Therefore, question 1 presents the administrators with the results to this aspect of the survey and
asks for their reflection to their experiences.
Table 13 below lists highlighted focus group questions and notable responses.
Table 13
Highlighted Focus Group Questions and Notable Responses
Focus Group Question Notable Response
1. Does that match your personal
preference, resumes over transcripts?
“Yes, I think the resume gives a better picture
of not only their experiences, but the picture
they are trying to paint of themselves.”
2. Is there any way to know that the
selected candidates are better than the
runners up that you didn't take?
“We never can really know. We hope for the
best.”
3. Bounded rationality suggests that
we're not as rational as we feel like we
are. Do you find that notion to be true
and if so how do you deal with that
when you're making these big decisions?
“Sometimes teachers in the hiring committee
will definitely be thinking about ‘Who can I
work with? Who's going to be a good colleague
for me?’ So that will sometimes influence them,
but then again, that also is the reality that they
do have to work together as a team, so. It's in
my mind as well and maybe in their minds a
little bit differently.”
4. Having had these discussions with
me, in reflection, is there anything about
your interview process that you would
like to be able to change?
“I don't know what I would change. Our
process right now is pretty good.”
“I think if we had the luxury of doing demo
lessons for every teaching position I think that
would be a good procedural change.”
5. If you had some sort of protocol or for
lack of a better word, a checklist, that
reminded you of the things that you say
or some sort of best practices sort of
sheet, would you be open to that as an
addition to your process?
“I think it would be beneficial. That way you're
kind of using a similar lens for every candidate
and applying that those qualities to each one.”
97
Cycle 1 Pattern matching coding.
Administrator Expertise/Confidence. Prediction overconfidence is a bias recognized
within the bounded rationality. In the context of teacher hiring, it may present as overconfidence
of decisions. The coding process identified responses that could demonstrate confidence without
measurable evidence.
Confirmation Bias/Likeability/Beliefs Regarding Characteristics of Successful
Teachers. The questions for this study rooted in the theoretical framework were designed to
yield responses of likeability. However, in this study, the responses were not based on general
likeability or even how well the candidates managed the pressure of the interview. In fact,
principals expressed intentionally overlooking any perceived nervousness on the part of the
candidates. Instead of the anticipated reactions of general likeability, the bias of likeability was
demonstrated through the principals varied assertions of the characteristics of good teachers.
Although, there were variances such as understanding pedagogy, psychology, relationships,
different learners, etc. Across the data sets, the asserted characteristics to describe the high
quality candidates were aligned to the skill set and personalities purported by the principals of
themselves.
Fit. The focus group members discussed the importance of the new hire’s ability to work
within the teams in which they would be placed and the ability of the candidates to move the
district in the direction the leader intends.
Cycle 2 In vivo coding.
Prediction Overconfidence - Administrator Expertise/Confidence. When asked by the
researcher, “Is there any way to know that they are better than the runners-up that you didn't
take?” one of the principals responded, “I don't think there is. I think it's always a little bit of a
98
gamble.” The second principal added, “That's all you can do is make the best choice at the time
with the information you have.”
Gut/Intuition. During the focus group discussion, the researcher asked the following
purposefully leading question, “Bounded rationality suggests that we're not as rational as we feel
like we are. Do you find that notion to be true, and if so, how do you deal with that when you're
making these big decisions?” One principal responded, “Well, I think that's true. I think if we
were totally rational, we would maybe jot way more on the transcripts and the facts, but I think
that since it is a people business, we often kind of go on a gut instinct…[pick] who the panel
clicks with and hope that plays out well for us.” The other principal followed up with a similar
response, but with a take on rationality. “If you're saying being objective, I think even if you
think of being objective during the interview process, you try to ask questions … looking for
something that's either going to swing you one way or the other. Was that kind of a positive that
[I] really wanted to hear?”
Confirmation Bias/Beliefs Regarding Characteristics of Successful Teachers/Fit. The
researcher pressed the focus group to articulate what they meant by effective teachers. The
principals offered areas related to teaching that candidates should know. “Knowing their stuff,
not just knowing the content. To me it's a given that they know the content. We shouldn't have to
kill them with professional development. They should be coming out of college knowing that.”
Another principal added, “I also want them to know the content. I want them to know what is
expected in the classroom. I want to them to know the curriculum. But above and beyond, how to
communicate with parents. So I guess I'm looking less for fitting in and more for knowing your
stuff and being able to handle themselves. And knowing how to treat my children.”
99
Summary of focus group analysis. As the members of the focus group generated 40%
of the data gathered in the 1:1 interviews, the focus group results correlated with the themes
generated in the individual interviews. The main values added from this data point were the
additional clarifications and opportunities for In vivo coding. The fast thinking of biases and
heuristics along with the slow thinking that might improve rationality were noted in the focus
groups and the 1:1 interviews.
In the development of this research proposal, the many biases and heuristics identified
within the theoretical framework of bounded rationality were considered for the potential impact
on the hiring process. Through the analysis of the focus group as the third data set, in this study,
certain themes related to bounded rationality were more evident than others. Table 14 includes
focus group questions, the themes anticipated by the researcher based upon bounded rationality,
the codes identified in cycle 1 pattern coding and those identified the cycle 2 In vivo coding.
The two most common and prominent themes were administrator confidence and beliefs
regarding the characteristics of effective teachers.
Table 14
Data Set 3 Focus Group Interview to Codes and Themes
Focus Group Question Anticipated
Themes
Cycle 1
Identified
Pattern
Cycle 2
Identified In vivo
1. Does that match your
personal preference,
resumes over
transcripts?
Confirmation
Bias
Confirmation
Bias/Likeability
/Beliefs
Regarding
Characteristics
of Successful
Teachers.
“Sometimes a candidate can be very
book smart but not have those
relationship building characteristics
that you really need in the
classroom.”
2. Is there any way to
know that the selected
candidates are better
Prediction
Overconfiden
ce
Administrator
Expertise/Confi
dence
“We never can really know. We
hope for the best.”
100
than the runners up that
you didn't take?
3. Bounded rationality
suggests that we're not
as rational as we feel
like we are. Do you find
that notion to be true and
if so how do you deal
with that when you're
making these big
decisions?
Prediction
Overconfiden
ce
Intuition
Administrator
Expertise/Confi
dence
Fit
“Sometimes teachers in the hiring
committee will definitely be
thinking about ‘Who can I work
with? Who's going to be a good
colleague for me?’ So that will
sometimes influence them, but then
again, that also is the reality that
they do have to work together as a
team, so. It's in my mind as well and
maybe in their minds a little bit
differently.”
4. Having had these
discussions with me, in
reflection, is there
anything about your
interview process that
you would like to be
able to change?
Time
Restrictions
Administrator
Expertise/Confi
dence
“I don't know what I would change.
Our process right now is pretty
good.”
“I think if we had the luxury of
doing demo lessons for every
teaching position I think that would
be a good procedural change.”
5. If you had some sort
of protocol or for lack of
a better word, a
checklist, that reminded
you of the things that
you say or some sort of
best practices sort of
sheet, would you be
open to that as an
addition to your
process?
Prediction
Overconfiden
ce
Administrator
Expertise/Confi
dence
“I think it would be beneficial. That
way you're kind of using a similar
lens for every candidate and
applying that those qualities to each
one.”
The section below considers the analysis of themes across and among the three data sets.
Emergent Themes from the Three Data Sets
The first section of themes below includes bounded rationality’s fast thinking biases and
heuristics.
101
Identified fast thinking. Regardless of the clarity of the available data, intuitive, or fast
thinking, undercuts rational thought (Kahneman & Tversky, 1984). The biases and heuristics
below were identified during this study.
Affect bias/likeability. The researcher anticipated affect bias to be a prominent bias
based on the literature stream of bounded rationality and hiring practices. Data points for the
affect bias were the principals’ beliefs that an interview was more effective to identify skilled
teachers than observing teachers’ actual instruction. The researcher noted this theme as potential
evidence of the affect bias, as hiring decisions based on interviews without mock lesson demos
may be based on likeability over evidence of efficacy.
However, the expected impact of general candidate likeability was not noted. Instead, the
potential expression of the affect bias was presented in principals seeking those with experiences
and educations similar to their own. The anticipated likeability factor was one of geniality or
even distinguishing characteristics of gender, race, or language. Instead, it was the candidate’s
knowledge and experience or the potential for relationship building that was most observed for
the preference of one candidate over another.
Confirmation bias/beliefs regarding characteristics of successful teachers. This
research exposed an unanticipated area for inquiry: administrators’ individual beliefs as to the
characteristics of successful teachers. Cranston (2014) cites evidence that principals who selected
candidates based on one teaching domain ignored the other teaching domains. Some principals in
this study stated differing beliefs regarding essential skills. Although all principals agreed that
the ability to build relationships with students, staff, and parents was an essential skill, middle
school principals expressed interest in candidates’ content knowledge. “I can help them with the
instruction if they are masters of their content area.” Another principal stated, “At the middle
102
school, I have about 40 teachers and I have 12 sports and 15 clubs and a crazy band and a chorus
with 100 kids in it. That piece always is my bias, my personal bias, and I might be the one who is
more biased towards candidates with non-educational experience because I'm the middle school
principal.” Elementary principals expressed the opposite position, “You can learn the content,
but it is very hard for us, regardless of the amount of professional development, to make
someone a masterful instructor.”
Fit. Fit was mentioned by numerous administrators as an aspect they consider when
hiring teachers. “Do I see this candidate fitting in with our school?” In the focus group
discussion, the principals stated fit was a key interest of teacher committee members as it is the
teachers who will work most closely with the new teachers. In the focus group, the researcher
asked a follow up question as to whether looking for a candidate who is similar to the staff limits
the expansion of diversity when the staff is predominately white. The administrators said that
they were unsure as the pool of candidates brought in for interviews were almost exclusively
white.
Missing data bias. The researcher noted a widely-held belief that the hiring process
gathered all data necessary for making an informed decision. The use of follow up questions of
individual candidates was offered as a solution to any missing data. “I can find out all I need to
know by asking follow up questions.”
Prediction overconfidence - administrator expertise/confidence. Across data sets,
administrators stated confidence, varying across individuals, in their ability to identify the best
teacher candidate through the interview process. Levels of confidence tended to correlate to
length of time in the administrative position.
103
Identified slow thinking.
Data review. Principals articulated decision making processes in line with Kahneman’s
(2011) recommendations for slow thinking that may improve rational thinking. Principals
included cross checking of data through reference checking and post interview committee
member discussions. “I think the bigger districts have the benefit in terms of having more
people, so they have more time. They can include more people. Doing interviews with bigger
committees of people sometimes can be really good because you get more points of view.”
Additionally, principals detailed the use of written interview questions and protocols
intended to generate more objective results.
Self-awareness. Two of the principals demonstrated self-awareness with statements such
as, “You hope you are right, but you never really know.” Bounded rationality holds that the
mindset of self-evaluation is essential in improving rationality.
Summary of Themes
Table 14 lists the biases and heuristics selected by the researcher for the research question
investigated by this study. Column two defines the biases and heuristics identified in the
theoretical framework of bounded rationality. Column three lists the researcher’s predictions as
to the manner that the indicated bias and heuristic might be noted in the process of hiring
teachers. The fourth column reports the researcher’s data analysis for each of the focus biases
and heuristics. Evidence of some of the biases and heuristics was identified. For others, it is
indicated that either no evidence of fast thinking via that particular bias/heuristic was noted or,
alternately, that only evidence of slow thinking contrary to the type of bias/heuristic was
discovered.
104
Table 14
Biases and Heuristics and Possible Impact on Hiring Decisions
Biases and
heuristics Effects
Possible impact on hiring
decisions
Researcher’s
observation of data
analyzed
Affect bias causes decisions to be
based on emotions or
on likes/dislikes
Is there evidence of teacher
hiring based on adult to adult
likeability rather than teacher
efficacy?
Researcher noted
hiring decisions based
on interviews without
mock lesson demos
may be based on
likeability over
evidence of efficacy
Anchoring causes the
weighing of one
piece of information
too heavily, often the
first information
encountered
Is there evidence of
committees selecting
candidates with shortcomings
related to the abilities as
teachers that should have
prevented their selection?
Researcher only noted
Anchoring in
instances in which
superintendents
override committee
selections.
Availability causes overestimation
of the likelihood of
events when similar
past events come to
mind
Is there evidence of
committees selecting
candidates based on the
administrator’s specific
knowledge base rather than
from a full range of needed
information?
Researcher was
unable to identify
Availability as a
consistent bias
Confirmation
Bias
causes the
searching for
evidence that
supports belief and
ignoring evidence
that contradicts it
Is there evidence of
individuals selecting
candidates based on the one
teaching domain his/her
preferred candidate most
demonstrated and ignore the
evidence from the other
teaching domains?
Researched noted
evidence of principals
selecting candidates
based on the one
teaching domain
ignore the evidence
from the other
teaching domains
Missing Data
Bias
is the tendency to
ignore the lack of
available data needed
for rational decisions
Is there evidence of
committees selecting
candidates about whom little
is known over those better
known?
Researcher noted
administrators’
willingness to ask
follow up questions of
candidates, but
researcher noted a
widely held belief that
data was not missing
105
from the process
Prediction
Overconfidence
is the subjective
confidence level that
is not supported by
objectivity
Is there evidence of
committees selecting
candidates based on
unsubstantiated predictive
tools such as experience as a
coach will cause the
candidate to be a better
teacher than those who have
not coached?
Researcher noted
middle school
principals who placed
outside experience
above that of teaching
experience
Sunk Cost
Effect / Loss
Aversion
is the phenomenon in
which people justify
increased investment
of time or resources
into a decision, based
on the cumulative
prior effort
Is there evidence of
committees selecting
candidates who at the end of
a lengthy screening process
may not be strong candidates,
but the administrators resist
reopening the hiring process
due to the time already
invested in finding the first
candidate?
Researcher noted
instances when
principals were, due
to student need,
required to accept a
less than stellar
candidate, but all
principals reopened
searches whenever
possible
Data Triangulation
Data was triangulated across three distinct sources: survey, 1:1 interviews, and a focus
group. Further, the interview transcripts were shared with interviewees for accuracy checks and
the researcher conducted a thorough examination of each participant’s responses across the
multiple months of contact and interviews. Each participant’s responses were found to be
consistent from the survey, interview, and the focus group, where available. The data provided
an “up close” and “in-depth” coverage required for qualitative studies (Yin, p. 192). Participant
responses were gathered through multiple forms of collection. In the survey, “Almost Always”
yielded the highest response rate (4 principals, 33%) to the question, “To what degree are
teachers hired through screening committees?” Yet, the second highest response rate was
“Never” (3 principals, 25%). Discussions in the follow up interviews indicated the cause was the
106
size of the school district. The smaller the district, the fewer administrators and other staff were
available to conduct interviews.
The individual and small focus group dialogues allowed principals to express their
thoughts without the filter of written expression. Careful transcript review and coding
highlighted the themes of the principals’ thoughts on the hiring process. Although research is
available on the practice of hiring teachers (Clement, 2013; Cranston, 2012; Rutledge et al.,
2008), additional qualitative research, such as that offered in this study, is needed. This research
sampled 6% of the principals in one of the geographically largest counties in a particular mid-
Atlantic state. Two of the candidates were involved in all three of the data sets: interviewed
through an hour-long focus group, an hour-long individual interview, and an online survey.
Interview questions were semi-structured in order to gather pertinent information, yet to allow
for open dialogue in order to better understand administrators’ decision making processes.
Trustworthiness
Additional participants would have increased the trustworthiness of this research.
Coordinating the schedules of busy principals across multiple districts proved difficult. Four
administrators were scheduled for the final focus group, but only two attended. This was an
unexpected difficulty that might have impacted collection.
Credibility. Through the researcher’s engagement with participants, credibility was
established. Member checking (Creswell, 2012), the opportunity for participants to examine the
narrative as it has been summarized by the researcher also maintained credibility. The study was
conducted following the guidelines and approval of Northeastern University’s IRB.
Transferability. The study’s clarity and depth of description allows the study to be
applicable to other contexts. This was accomplished through the establishment of the context of
107
the study and the narrative of the transcription and coding. As a study of the experiences of a
sample group of principals, this study is transferable to principals engaged in the process of
hiring teachers in other settings.
Dependability. Dependability is the ability to replicate research and yield similar results.
(Creswell, 2012). The researcher reported the research process clearly so that the study could be
replicated. Further as the participants are a representative sampling, similar participant pools are
available for repeated studies if needed.
Confirmability. Bracketing, the means in which the researcher examines his beliefs
toward the phenomenon (Tufford & Newman, 2012, p. 85), offers a means to reduce negative
researcher influence on the research process and can be conducted prior to and during data
gathering and analysis. To establish confirmability, this researcher detailed his biases in the
positionality section of chapter one. Being reflective (Creswell, 2012, p. 18; Tufford & Newman,
2012) through examination of the researcher’s role or position in the study during the length of
the study reduced, but did not eliminate the impact of biases.
Summary of Findings
The findings from this research were based on the analysis of data gathered through a
survey, semi-structured 1:1 interviews, and a focus group discussion. The participants provided
their understanding of their teacher hiring processes.
Eight key findings were generated through the triangulation of the data collected
throughout this study:
Finding one: Bounded rationality affects the decision-making of K-8 educational
leaders.
108
Identified fast thinking.
Finding two: Principals were individually consistent in the characteristics they sought in
teacher candidates, but the teacher characteristics listed as most important were not consistent
among all principals.
Finding three: The concept of the “fit” of the candidate within the present school climate
was a consideration for principals in the teacher hiring process.
Finding four: Principals did not express concerns of missing data in the teacher hiring
process.
Finding five: Principal expressed confidence in their abilities to select the best
candidates from a pool of candidates.
Identified slow thinking.
Finding six: Principals used protocols and question sets within interviews with the
expressed desire of formalizing and thus improving the selection of teacher candidates.
Finding seven: Principals expressed self-awareness and potential fallibility, keys to
combating fast thinking.
Finding eight: Principals stated the addition of a decision-making tool, such as a
checklist, might improve their processes.
Conclusion
The theoretical framework of bounded rationality offers a view of human thinking as
universally flawed (Kahneman, 2011). The intuitive mind, which is essential for surviving in the
wild, interrupts the careful, slow thinking required for decision making in instances such as
teacher hiring. The data gathered in this study located a number of examples of intuitive thinking
which may allow for quick thinking biases to impact decisions without the administrator’s
109
awareness. However, this researcher also identified instances in which administrators, aware of
this potential, used slow thinking to check for biases.
A variance in administrators’ beliefs as to the experiences and characteristics of effective
teachers was noted. This variance mirrored the administrators’ observations of teachers
employed by them, but also of the administrators’ beliefs about their own experiences that led
themselves to become educators. This is correlated with the theoretical framework of bounded
rationality; humans are not unbiased, rational decision-makers. Rather, thinking is filtered by the
collective experiences and even more so by the quick thinking that experience generates. The
researcher found evidence of fast and slow thinking in all participants, with consistent leanings
of some more than others. Therefore, the suggestions from the field of psychology rooted in
bounded rationality to improve consistent decision-making across experts are applicable to the
hiring practices of teachers.
The final chapter of this thesis contains the researcher’s findings generated through this
study. Discussion will include major insights based on data collected, analysis, the theoretical
framework of bounded rationality, and the literature streams reviewed. Additionally, this
discussion will include recommendations to improve the rationality of the teacher hiring
practices through the tools identified in the fields of psychology and social economics.
Suggestions for additional research and consideration of alternate explanations will be included.
110
Chapter V: Findings
Introduction
Chapter five of this qualitative study includes discussion of the key findings identified in
chapter four. Three data sets: a survey, semi-structured 1:1 interviews, and a focus group, were
analyzed through the lenses of bounded rationality and the cumulative literature review. The
literature review consisted of three streams: bounded rationality, decision-making tools, and
hiring practices.
The themes of the research reviewed drove the creation and organization of this inquiry.
First, it is known that teacher hiring is one of the most important tasks for principals (Clement,
2013; Mason & Schroeder, 2010; Peterson, 2002; Rutledge et al., 2008). However, secondly, it is
also known, that rational thinking is not the human default position (Kahneman & Tversky,
1984; H. A. Simon, 1955; M. Toplak et al., 2011). Thirdly, it is known that protocols and
checklists can improve expert decision-making (Flottorp et al., 2013; Haynes et al., 2009;
Kahneman, 2003; Stubenrauch et al., 2012; Takala et al., 2011). Therefore, the researcher
designed this qualitative research study to explore what is not known, whether the specific biases
and heuristics noted in the decision-making of experts in other fields can be identified in the
hiring practices of educational experts and the degree to which principals express the need or
interest in decision-making tools for the hiring process.
Review of Themes from Research
The multiple rounds of analysis and researcher reflection on the three data sets, once
triangulated, produced seven themes. Five themes under the heading of fast thinking: affect
bias/likeability, confirmation bias/beliefs regarding characteristics of successful teachers, fit,
111
missing data bias, and prediction overconfidence and administrator expertise/confidence. Two
additional themes emerged as components of fast thinking: data review and self-awareness.
These themes were very similar to those from the literature, particularly from that of
bounded rationality. This is to be expected as bounded rationality was used to design the research
question and the research that followed. Additionally, the themes noted also mirrored those from
the other streams of literature reviewed: hiring of teachers and decision-making tools. This
chapter focuses on the implications of the findings for the processes of teacher hiring.
Discussion of Key Findings
The purpose of this study of educational leaders in mid-Atlantic school districts is to
understand the process of teacher hiring and to answer the following research question: How
does bounded rationality affect administrators’ decision-making processes during the process of
hiring new teachers? The research question and inquiry which followed is grounded in bounded
rationality. This research included the analyses of a survey of principals, individual interviews,
and a focus group. The themes identified from this process led the researcher to construct a
number of conclusions. Eight key findings were generated through the triangulation of the data
collected throughout this study. Each finding is an answer to the research question: How does
bounded rationality affect administrators’ decision-making processes during the process of hiring
new teachers?
Finding one: Bounded rationality affects the decision-making of K-8 educational
leaders.
This researcher, in investigating how bounded rationality affects the decision-making of
K-8 educational leaders, presumed that principals, as has been shown for leaders in other fields,
are impacted by bounded rationality. The biases and heuristics deemed as common to all humans
112
and noted in numerous decision-making experiments in the fields of psychology and economics
(Kahneman, 2011) were noted in this study. The predictions for this study based on bounded
rationality appear throughout the transcripts. Additionally, the evidence of bounded rationality in
this research serves to support Kahneman’s (2011) assertion that bounded rationality impacts all
decision-makers, even experts.
Finding two: Principals were individually consistent in the characteristics they sought in
teacher candidates, but the teacher characteristics listed as most important were not consistent
among all principals.
This finding is grounded in bounded rationality as it indicates biases may be present in
the decision-making process of teacher hiring. Experimentally observed bounded rationality
indicates the likeability bias comes into play when evaluating candidates (Selten, 1998). So it
was anticipated that principals would express this common bias. The evidence gathered
surrounding this variance was of the greatest surprise and interest to the researcher.
Perhaps due to the lack of a controlled experiment that viewing live teacher interviews
may have allowed, evidence of general likeability was not an identified characteristic used by
principals. Instead, of note were the many references made to the principals’ backgrounds and
the desire to find teachers with similar backgrounds. If a principal studied special education,
he/she looked for that in candidates. If a principal worked outside of education, that was a
preference expressed for candidates. This researcher anticipated principals’ expressions of a
general likeability bias for candidates, however, another bias not selected as a predicted bias
from bounded rationality, social comparison bias (Kahneman, 2011) the tendency, when making
hiring decisions, to prefer potential candidates who don't compete with one's own particular
strengths, may have been more applicable. Perhaps an area for future study, the tendency to
113
prefer a candidate with a similar background may result from the social comparison bias to avoid
those with conflicting strengths.
Finding three: The concept of the fit of the candidate within the present school climate
was a consideration for principals in the teacher hiring process.
This finding is grounded in bounded rationality as fit is a specific bias listed among the
biases and heuristics identified in bounded rationality.
If the goal of a teacher hiring process is to find the best teacher candidates to work with
the students, the question of fit may replace the characteristics needed for students with those
preferred by colleagues. In this way, the notion of fit may serve as another form of likeability
bias. This would be in keeping with bounded rationality which includes the ideal that when
presented with questions, answers that are derived quickly are more likely to be confidently
perceived as accurate by both the thinker and those around him or her (Kahneman & Tversky,
1984). One can readily, from an intuitive mindset determine if someone is similar to others
already in the group. The answer of fit, is an answer, however, to a different question. It is not an
answer to the question of which teacher will be most effective. It is an answer to which candidate
is most similar by appearance and demeanor to those already present.
Additionally, there is a minority teacher staffing crisis in the United States (Leonardo,
2002). 40% of schools do not have a single teacher of color (Kohli, 2008). If teacher candidates
are considered by the concept of “fit,” continued segregation of staff may occur.
Finding four: Principals did not express concerns of missing data in the teacher hiring
process.
This finding is grounded in bounded rationality as missing data is a specific bias listed
among the biases and heuristics identified in bounded rationality. In this instance, the research
114
question is answered not by what is expressed as a data point, but rather by the principals’ lack
of concern about missing data in their decision-making processes.
Missing data is an oft overlooked situation (Henningsen & Henningsen, 2007; Tversky &
Kahneman, 1974). For decisions to be rational, all pertinent information is required. In the hiring
process of teachers, bits of information may be acknowledged about one candidate, especially
candidates previously known to the hiring committee. If those pieces of background information
are to be used, for rational decision-making, the same information would be needed for all
candidates. This form of missing information was not seen as a concern by the principals
interviewed. (Henningsen & Henningsen, 2007) noted that pausing as individuals or groups to
explore sources for missing information can increase rationality, but the bias limits the mind
from noticing a lack of data and therefore there is no internal alarm to pause. Principals may not
notice that more evidence about one or more of the candidates is even needed in order to fairly
compare the candidates. The use of a committee may create the appearance of the expansion of
perspectives and data gathering; however, additional group members may actually reduce the
shared knowledge, which is given greater value than individual information. When groups meet
to discuss and make decisions, it is the shared, not the unique perspectives, that are given the
greatest deference (Kahneman et al., 2011). Therefore, increasing the number of members of a
group does not necessarily increase the number of choices to be considered. Information that is
unknown to all is not discussed and information that is known to just a few is devalued
(Henningsen & Henningsen, 2007). For decisions to be rational, the data that serves the question
should be given the greatest weight. In group dynamics, information that is well known is given
more value by virtue of it being shared knowledge, not due to the data’s value in the situation.
Therefore, increasing committee size may not improve the rationality of the decision.
115
Finding five: Principals expressed confidence in their abilities to select the best
candidates from a pool of candidates.
This finding is grounded in bounded rationality as confidence in one’s expertise is a
specific bias listed among the biases and heuristics identified in bounded rationality. If principals
are not internally or externally motivated to examine the possible fallibility of their decisions,
this finding provides an answer to the research question. Expert overconfidence (Belsky, 2016)
prevents decision-makers slowing thinking to allow for more rational thought. The principals
who expressed less confidence may actually make better decisions because they access slow
thinking as a means of self-checking. Finding seven addresses this alternate option.
It is the ease with which people are drawn to the simplest answer, as indicated by the ball
and bat test question (Frederick, 2005) that simultaneously generates a feeling of gut confidence
and lowering of engagement of the slow thinking rational mind (Kahneman, 2003). The
Cognitive Reflection Test (CRT) introduced by Frederick (2005), illustrates the working of the
two systems. A bat and a ball cost $1.10. The bat costs one dollar more than the ball. How much
does the ball cost? (Frederick, 2005 p. 25) If the bat costs a full dollar more than the ball, a $.10
ball would require a $1.10 bat and therefore a sum of more than $1.10. Only a $.05 ball and a
$1.05 bat would equal $1.10. The quick thinking of System 1 selects the first seemingly
acceptable answer, $0.10. The faster an answer comes to the human mind, the more accurate it
feels. Only in engaging System 2 is the quick answer exposed as incorrect and the correct answer
of $0.05 for the ball and $1.05 for the bat realized. The faster and easier a decision is made, the
more it feels like the right choice. Similarly, the candidate that is most quickly considered to be
best may feel like the correct choice even if countering evidence is identified further into the
process.
116
Finding six: Principals used protocols and question sets within interviews with the
expressed desire of formalizing and thus improving the selection of teacher candidates.
This finding is grounded in bounded rationality as formalized processes are helpful in
engaging the slow thinking mind that is deemed as rational within the framework of bounded
rationality. For principals to include protocols and questions sets within interview processes, this
finding offers an answer to how principals manage bounded rationality as posed in the research
question.
Protocols have been shown to improve expert decision-making (Flottorp et al., 2013;
Haynes et al., 2009; Kahneman, 2003; Stubenrauch et al., 2012; Takala et al., 2011). The
principals’ use of question sets matches the prescriptions noted in the literature. However, the
paper screening which precedes the live interviews did not have the same level of consistency.
Principals consistently weeded out applicants who did not have the proper certifications, but after
that step, each made individual choices on the remaining resumes based on a wide range of
preferences. No checklists were utilized for this portion of the process. Biases have been shown
to have a profound impact on resume screening in other fields (Krings, Sczesny, & Kluge, 2011).
Finding seven: Principals expressed self-awareness and potential fallibility, keys to
combating fast thinking.
This finding is grounded in bounded rationality and offers an answer to the research
question as self-awareness and consideration of fallibility are helpful in engaging the slow
thinking mind that is deemed as rational within the framework of bounded rationality.
“But knowing you have biases is not enough to help you overcome them. You may
accept that you have biases, but you cannot eliminate them in yourself” (Kahneman et al., 2011,
p. 52). However, biases are more readily noticed in others. Therefore, a properly oriented
117
organization can improve the decision-making process. This is using System 2, rational thought,
to check for erroneous thinking in System 1, intuition. Principals interviewed in the process
expressed that in the end, the selection of a candidate was a best effort selection. “You make a
decision and hope for the best.” Opportunities for the slow thinking mind to critique decisions
could cause principals to reevaluate the considered data. This could improve rationality in the
decision-making process of teacher hiring.
Finding eight: Principals stated the addition of a decision-making tool, such as a
checklist, might improve their processes.
This finding is grounded in bounded rationality as bounded rationality researchers (Singh,
2008; Stubenrauch et al., 2012; Takala et al., 2011) have demonstrated that formalized processes
are helpful in engaging the slow thinking mind that is deemed as rational within the framework
of bounded rationality. Additionally, researchers (Churchman & Doherty, 2010; Flottorp et al.,
2013; Haynes et al., 2009; Kahneman, 2003; Sibbald et al., 2013; Singh, 2008; Stubenrauch et
al., 2012; Takala et al., 2011) argue that experts resist the use of decision-making tools even
when these are empirically proven to improve the decision-making within the expert’s field.
Checklists work well when these include task definitions, recording responses directly on
checklists, and are periodically reviewed by a supervisor (Flottorp et al., 2013). The use of an
agreed upon checklist allows critique and dialogue to be part of an organization’s climate. If a
single individual were asked to fill that role, social pressure might dissuade her/him. The
proponents of this checklist offer implementation suggestions. The checklist should be used for
important, not routine decisions. When in use, it should be fully used. Kahneman (2011)
references the rigor of use of the World Health Organization (WHO) Surgical Safety Checklist as
a model of adherence. When asked about their view of a need and willingness to use a decision-
118
making tool such as a checklist for the hiring process, some principals, as predicted by the
framework expressed their decisions would not be improved by such a tool. Other principals
expressed interest in seeing such a tool developed.
Linking Findings to Theoretical Framework
Bounded rationality, first coined by Simon (1955) offered a response to the dominant
theory of the time, neoclassical rationality, which held that human decisions were calculable,
rational responses with the most utility as the result. Simon suggested humans responded not
with the most logical solutions, but the most readily satisfying (H. A. Simon, 1982). It was this
counter narrative to the notion of rationality that drew this researcher to utilize this lens on the
work of educational leaders. As was the case in the 1950s, today there is a great deal of hope
placed on data-driven decision-making (Cooley et al., 2006; Marzano et al., 2005; Reeves &
Flach, 2011). Specific to this study is the notion of data-driven decisions to influence educational
leader decision-making. This researcher questioned if that premise is correct given the exposed
fallibility of humans, even experts, to accurately interpret data (Botterill & Hindmoor, 2012;
Kahneman, 2003; H. A. Simon, 1982).
The findings of this study indicate the specific biases identified in bounded rationality
indicated as shortcomings to rational thinking can be identified in teacher hiring. This questions
the validity of data-based decision-making as a panacea requiring merely data and the time to
review it. Researchers such as(Campbell et al., 2004; M. Toplak et al., 2011; Tversky &
Kahneman, 1974) have shown the presence of data can actually reduce rational thinking as the
data is accepted as evidence of the assumptions intuitively held. Any advancement in the area of
data-driven decision-making in educational leadership settings requires a clear understanding of
educational leaders’ data filtering processes and the resulting impact on their rationality.
119
Central to this area of research is the specific identification of cognitive biases and
heuristics and the manner in which these impact decisions. Of the many biases and heuristics
identified by the seminal work of Tversky and Kahneman (1974), this research specifically
noted: five themes under the heading of fast thinking: affect bias/likeability, confirmation
bias/beliefs regarding characteristics of successful teachers, fit, missing data bias, and prediction
overconfidence and administrator expertise/confidence. Three additional themes emerged as
components of fast thinking: data review and self-awareness.
Sterman (1989) extended research on the impact of biases and heuristics in individual
decision-making to the implications for organizations. Work since continues to explore the
construct between the benefits of heuristics that allow for decision-making and the potential to
shortcut rational decisions (Kahneman, 2003; Kahneman et al., 2011; Waddell & Sohal, 1994).
Sterman’s (1989) empirical study measured business leaders’ heuristics including anchoring and
adjustment. These subconscious patterns of thinking and preferences filter any incoming
information related to the choices available. This was shown in the responses of participants of
this study. Although there were generalities in what characteristics were desirable in teacher
candidates: e.g. caring, communication, and content knowledge, there were individualized
preferences held by principals that were not shared. Therefore, the decisions individuals would
make when presented with the same data points, interviews and resumes, would not be the same.
The allure of rationality and data driven decision-making is the promise of consistency and
accuracy; bounded rationality and this study suggest those are illusions.
As organizations become increasingly complex, so do the decisions. However, empirical
studies (Eisenhardt & Zbaracki, 1992; Kahneman et al., 2011; Nobre, Tobias, & Walker, 2009;
H. A. Simon, 1982) demonstrate that subjects are insensitive to the feedback from their
120
decisions. This impacts the function of the organizations tremendously. Insensitivity to the
complex surroundings distances the decision makers from reality. Insensitivity to data impedes
the very essence of data-based decision-making. During the interviews of this study, when asked
if the hiring decisions were ever poor or how it could be known if runners up would have been
better, principals responded that on occasion teachers turned out to be less stellar in practice than
the impression from the interview process indicated to them. When that happened, the teacher
was coached or released. The principals, however, did not offer changes to the process to reduce
those errors.
An extension of the problem caused by misinterpretation of data is the issue of
information gap. When information is missing from a data set, people often infer the value of
missing information based on the value of the known information, sometimes bolstering or
diminishing the value (Henningsen & Henningsen, 2007), assuming the groups realize there is
missing information. When groups meet to discuss and make decisions, it is the shared, not the
unique perspectives, that are given the greatest deference (Kahneman et al., 2011). Therefore,
increasing the number of members of a group does not necessarily increase the number of
choices to be considered. Information that is unknown to all is not discussed and information that
is known to just a few is devalued (Henningsen & Henningsen, 2007). Perhaps this is why the
majority of principals expressed the use of a committee did not yield selection of the better or
even different candidates than if a single principal made the choice. However, the addition of the
committee members did increase the acceptance of the new candidate if colleagues were
involved in the selection. This was seen as the best reason to include stakeholders in the
committee process. Some principals mentioned utilizing the post interview discussion with the
other administrators if two or more candidates were acceptable to the principal.
121
Linking Findings to Literature
The literature streams of decision-making as viewed through the multiple lenses of
bounded rationality, techniques utilized in the attempt to increase the rationality of decisions in
the practice of various fields such as business and medicine, and teacher hiring practices
informed the construction of this study and were confirmed as relevant to this study of teacher
hiring.
The findings of this study indicate bounded rationality to be involved in principal
decision-making in the process of teacher hiring. This study demonstrates the similarities
between the decision-making settings in other fields and those of education. There is research,
however, which indicates specific means to ameliorate the deleterious effects of bounded
rationality on decision-making (Bacon et al., 1983; Bartels & Mortenson, 2005; Brañas-Garza et
al., 2012; Cioffi & Markham, 1997; Fudickar et al., 2012; Haynes et al., 2009; Kahneman, 2003;
Place & Vail, 2013; Sibbald et al., 2013).
The decision-making process is more important than the data itself when attempting to
“debias” strategic decisions (Kahneman et al., 2011; Lovallo, D. & Sibony, O., 2010). The hiring
of an individual teacher is not a strategic decision; however, the importance of the impact of the
process over data should be considered in hiring activities. A company’s use of data to make
predictions of the future behaviors of costumers relates to educational leaders’ attempts use data
in order to predict a candidate’s future behavior as a teacher.
It has been shown that business executives can reduce the impact of biases such as
anchoring, weighing one piece of information too heavily, and confirmation, ignoring evidence
that contradicts one’s beliefs (Kahneman et al., 2011). “But knowing you have biases is not
enough to help you overcome them. You may accept that you have biases, but you cannot
eliminate them in yourself” (Kahneman et al., 2011, p. 52). However, biases are more readily
122
noticed in others. Therefore, a properly oriented organization can improve the decision-making
process. The transcripts of this study captured principals responding to the question, “Checklists
and other tools have been shown to offset some of the ways bounded rationality negatively
impacts decision-making. Would you be interested in using such a tool?” As would be
specifically predicted by the researchers Kahneman and Lovallo (2011), some principals
expressed that decision-making tools might help experts in other fields, but not them. They
expressed their expertise was built through experience and no external support would be needed.
Other principals, who expressed more reflective responses to other questions, answered this
question with responses of interest and curiosity as to what a tool for this purpose might look
like. These principals were open to the possibility that a decision-making tool might be helpful.
Research has been conducted previously that might guide decision-making tool
construction for principals to utilize during the hiring process. For example, there are parameters
for use such as those offered by Flottorp et al (2013); checklists work well when these include
task definitions, record responses directly on checklists, and are periodically reviewed by a
supervisor. “The sweet spot for quality control is decisions that are both important and recurring,
and so justify a formal process” (Kahneman, 2011, p. 59). Hiring decisions are possibly the most
important recurring decision principals might make (Cranston, 2012; Pillsbury, 2005).
Surgeons are some of the most formally educated professionals and yet the use of the
World Health Organization’s (WHO) Checklist, even with its most obvious and simple set of
questions, has saved literally thousands of lives from the mistakes of highly skilled experts
(Takala et al., 2011). In this study of teacher hiring, the protocols for processing ranged widely
from the number of individuals involved, the questions asked, the answers considered as correct.
This research would suggest that although also highly educated, the decision-making processes
123
of principals could also become more rational and predictable through the use of research
supported decision-making tools. Referring to the WHO Checklist, Kahneman argues that
experts should not pick and choose from questions, but use these systematically. “Using
checklists is a matter of discipline, not genius. Partial adherence may be a recipe for total failure”
(2011, p. 60). Often in the medical field, nurses express trepidation to speak out when they
disagree with a doctor’s choice (Churchman & Doherty, 2010). The protocol of the WHO
checklist places the charge nurse in a position of power that better distributes the responsibility
to utilize best practices and, therefore, the checklist serves to make questioning a procedure
rather than the act of a provocateur (Kahneman et al., 2011). As this research indicated non-
administrators were not seen as in possession of skills in hiring teachers that was needed by the
principals, like nurses in operating rooms, this may suppress critique and data. A regimented
decision-making procedure could empower all voices and analyses within teacher hiring
committees.
Sunstein and Thaler propose an approach that is less obvious than checklists, this is
choice architecture, “policies selected with the goal of influencing the choices of affected parties
a way that will make those parties better off” (2003, p. 175). Thaler and Sunstein (2009) argue
that people do not always make decisions that are in their best interest and it is the role of
organizations to help them do so. Principals shared that different superintendents have often
brought to the organization preferred procedures for teacher hiring. In that manner, the
superintendents are creating for the organization a means to produce the best outcomes.
However, the principals shared, similar to what this researcher noted in the manner of processes
the principals created when able, the processes differed based on the individual preferences of
each superintendent. So in this study within a county in a particular mid-Atlantic state, not only
124
did each district have differing protocols, these fluctuated depending on which person was in
power at the time.
If education is to hold up data-based decision-making as a means to improve, certainly
consistency in the protocols will be required in order to utilize the most likely system of using
said data. Citing research which includes Gentry (2007) and Stronge and Tucker (2000),
Schumacher, Grigsby, and Vesey (2015) assert the factor which most impacts student learning is
teacher effectiveness. Schools are only as good as the staff that work there (Nonaka, 1994; Place
& Vail, 2013); however, this research adds to the studies that caution that the hiring practices
used to employee staff vary greatly and the criteria for selection are often based on feeling over
evidence, further, hiring is often “information poor” (Liu & Johnson, 2006 p. 331).
Study Limitations
As a qualitative study, this research was strengthened through the analyses of the
transcripts. However, as there were a relatively small number, five participants interviewed, a
fuller range of principal perspectives may have been missed. To mitigate the potential impact
from the limited number of participants, the researcher attended to all evidence through the
analysis of all three data sets as unique data and also as the three data sets constructed
trustworthy information when triangulated.
Research can unfold to generate a significant aspect of the case not predicted by the
researcher during the research design (Creswell, 2012). This researcher remained open to this
possibility during the data gathering, reflection, and analysis. The researcher anticipated teacher
hiring to be significantly impacted by the general likeability of the teacher candidates. However,
there was greater evidence that the selected candidates presented themselves as those having the
characteristics the individual administrators preferred in teachers. The noted biases were not
125
about general likeability, but what the principals liked to see in teachers because they liked to see
those characteristics in themselves. This research originated as a study of the procedures
principals used to increase rationality in decision-making, but following the data, explored the
consideration of what principals consider effective teaching such as work ethic, relationship-
building, content knowledge, pedagogy, background experiences. This study was designed to
focus on the experience of biases that shortcut decisions, not the relative validity of the possible
data points principals consider when selecting teachers. However, the principals had dedicated
years of focus on the characteristics they wanted to see in teacher candidates, not the process
itself. Kahneman (2011) suggests the intuitive mind is fast because it goes unnoticed by the
individual. Therefore, a qualitative study that asks participants to note or express their fast
thinking may not be as successful identifying those biases as the quantitative studies upon which
bounded rationality is built (Kahneman & Tversky, 1984; H. A. Simon, 1955; M. Toplak et al.,
2011). Follow-up research examining the characteristics principals consider as indicative of
effective teachers is warranted.
In a consideration of limitations, it is useful to reflect on the impact of this researcher’s
inherent expertise with the research area of inquiry. With twenty-nine years as a teacher,
administrator, and trainer for novice teachers, this researcher is positioned to have shared
experiences with the participants. This may have benefited the study in the understanding of
vernacular, realities, processes, and legalities involved in the teacher hiring process. This lived
experience in the process being studied may have served the researcher to note anomalies and
patterns. Additionally, as the participants and school districts were known to the researcher prior
to the study, background knowledge was available when analyzing the data points. However, it
should be noted that as a researcher-practitioner working in the county studied, this researcher’s
126
own biases regarding teacher hiring, along with the researcher’s own mental biases of slow and
fast thinking through which the data was analyzed, impacted the results of the study.
Principals expressed varied preferences in the characteristics of effective teachers and the
means to identify these. This researcher spent a career forming opinions on the topic, as well as,
years studying the related literature. This generated the researcher’s mindset related to the topic
prior to the study. The very design of the research as well as the analysis offered may be
impacted by the researcher’s prior knowledge of the topic of teacher preparation and pedagogy.
Alternate explanations
The main critique of the study could be the same fallibility of decision-making examined
in the study, the inability of the human mind to be rational and to reflect accurately on its
rationality. When someone is asked about how he/she does something, the response is filtered by
the very filters the questioner attempts to identify (Kahneman, 2011; Langlois, 1990; H. A.
Simon, 1982; H. A. Simon, 1991; Sunstein et al., 2002). Asking someone if s/he is effective
when teaching or rational when hiring, always yields the interpretations of the actor, not the
observable actions. Additionally, it should again be noted that this research is impacted by
bounded rationality and the conclusions drawn through the data gathering and analysis process
may have been erroneously filtered by biases and heuristics. One of the biases noted in bounded
rationality is missing data bias, failing to notice what additional information is needed to make a
decision. Although triangulation and reflection were employed to maximize accurate
interpretation of the data; the researcher may have missed data that would have offered alternate
explanations.
127
Implications for Educational Practice
This study builds a case for additional research into the area of decision-making in
educational leadership and specifically whether the negative impacts of natural biases and
heuristics on decision-making can be noted in the decision-making of educational leaders in the
process of hiring teachers. Decision-making tools such as checklists have been used to improve
decision-making in the fields including business and medicine without hampering expert
decision-making speed (Bacon et al., 1983; Bekker, 2010; Fudickar et al., 2012; Haynes et al.,
2009; Kahneman et al., 2011; Sibbald et al., 2013). This researcher examined the thinking
process of educational experts in order to categorize the biases and heuristics that are most
commonly involved. Further, this researcher designed this study in order to understand whether
rational decision-making could be made more overt and therefore manageable. If a checklist or
flowchart could be effectively applied when educators engage in complex decision-making,
specifically, teacher hiring, improved decision making could be the result.
In keeping with the research (Rutledge et al., 2008), of all the steps in the teacher hiring
process, principals placed the highest importance on interviews, above that of resumes and
reference checks, second and third most important respectively (Cranston, 2012; Rutledge et al.,
2008). Given the potential for bias in the review of all three of those data points, care should be
given to the processes of educational leaders.
To control for interviewer bias, researchers (Engel & Finch, 2015; Haskins & Loeb,
2007; Heneman & Milanowski, 2004; Lange, Range, & Welsh, 2012; Schumacher et al., 2015)
have recommended job specific criteria and interview protocols that focus administrators on
specific characteristics be utilized. This research identified all principals as using sets of
questions, however, the question sets did not indicate the preferable responses. That was left to
the individual principal or committee member. This may allow for the bias noted in this study to
128
emerge, hidden by the seeming rational formality of a set of questions. “Organizations need to
realize that a disciplined decision-making process, not individual genius, is the key to a sound
strategy. And they will have to create a culture of open debate in which such processes can
flourish” (Kahneman et al., 2011 p. 60). Continued research is needed to understand Kahneman’s
assertion in the context of educational organizations.
Assistance in teacher hiring is increasingly viable given the ongoing improvements in
technology. Scholars have provided evidence of the increased ability to construct effective
decision making tools. Computerized model-driven decision support systems (DSS) exist to
provide decision support for leaders (Power & Sharda, 2007). This could allow for a decision-
making tool for educators beyond a static form like the WHO Surgical Safety Checklist.
Dynamic Decision Support Systems (DSS), computer-based systems that help decision makers
confront ill-structured problems through direct interaction with data and analysis models
(Sprague & Carlson, 1982) which organize the increasingly quantifiable data available to
educational leaders might be an area worthy of study.
Implications for Future Research
This researcher explored the applicability of bounded rationality, which is generally
studied within the domains of psychology and economics, to that of education. Specifically, this
researcher considered whether the limits of decision-making noted in the literature also impact
the decisions of principals when hiring teachers. This study utilized principals’ descriptions of
beliefs and recollections. Further studies could utilize the quantitative methods used within
bounded rationality (Hoppe & Kusterer, 2011; Kahneman, 2011; Lovallo, D. & Sibony, O.,
2010; Singh, 2008; Sunstein & Thaler, 2003) to the specific context of teacher hiring.
129
The paper review process is just one portion of the teacher hiring process that could be
examined. Studies of resume review have indicated demographic biases such as ageism even
when age data is not present (Krings et al. 2011). Job related information in resumes might serve
as implicit cues about applicant characteristics. Controlled studies allow for use of the same
stimuli on multiple participants. By controlling for variances in a quantitative study the specific
biases and heuristics involved in teacher hiring might be identified in ways this research did not.
Identification of the specific fast thinking involved in the paper review of applicants might offer
specific challenges to rational thought in the hiring process. Such research might include answers
to the research question, how does bounded rationality affect administrators’ decision-making
processes during the process of hiring new teachers? From that increased understanding of the
biases and heuristics involved in the screening process, protocols to reduce their impact may be
designed.
The interview process is another portion of the teacher hiring process that could be
examined. This qualitative study offered information on principals’ recollections of interviews of
varied candidates and the accompanying data sets of applications and interviews. Empirical
studies could reduce the variance of events and the filter of single source memories. Quantitative
studies in the field of education, and teacher hiring specifically, might pinpoint the biases and
heuristics most impactful and thus answer the research question.
This study’s results and findings are transferable to other contexts within education or
wider research in the area of decision-making. The study’s clarity and depth of description
allows the study to be applicable to other contexts. This was accomplished through the
establishment of the context of the study and the narrative of the transcription and coding. As a
study of the experiences of a sample group of principals in K-8 public school settings, this study
130
is specifically transferable to principals in similar settings engaged in the process of hiring
teachers.
The next step for this research is the creation and evaluation of the efficacy of decision-
making tools in the practice of teacher hiring. In educational settings, performance feedback to
team members related to adherence to problem solving protocols was not effective until coupled
with a checklist (Bartels & Mortenson, 2005). The use of checklists is a very tangible and overt
attempt to counter bounded rationality. If leaders are willing and able to generate a culture of
systematic and transparent decision-making, biases can be minimized (Courtney et al., 2013). In
addition to overt tools such as checklists, subtler nudges in decision-making could be established
by schools. A nudge is “any aspect of the choice architecture that alters people’s behavior in a
predictable way” (Thaler, 2009, p. 6). Researchers could explore the use of the nudges of choice
architecture to extend the research question. The expanded inquiry could ask whether nudges,
such as the inclusion of administrator reflection questions which might slow thinking in order to
reduce the dependence on fast thinking and intuition, improve the administrators’ decision-
making processes during the process of hiring new teachers.
Additional qualitative studies could be used to investigate this study’s research question.
A number of aspects could be considered: e.g. alternate locations, expanded pools of candidates,
different interview questions, a researcher without experience in K-8 education. These additional
studies could establish the dependability of this study as well as generate findings beyond the
eight noted in this study.
Conclusion
As the demands upon and stakes for the success of educational leaders continue to
increase, it is crucial that principals are highly successful in what some (Cranston, 2012;
131
Pillsbury, 2005) consider the most important function of principals: the hiring of teachers. This
research has shown the process of teacher hiring is not optimized. Although the participants in
this study were found to be passionate and knowledgeable about the process of teacher hiring,
they were left without the supports offered to experts in other fields, such as doctors, pilots, and
business leaders. There was great variance in what each principal stated he/she considered as
indicative of prime candidates. This researcher has concluded the process of teacher hiring,
although considered a crucial task, is not given the level of support and oversight dedicated to so
many other aspects of adminstrators’ decision-making. There are manuals of protocols based on
policies and regulations. Most of these are related to student and staff safety, contracts,
expenditures, State Department of Education requirements. However, the actual practice of
teacher hiring varies greatly, is often “information poor,” and the criteria for selection is often
based on feeling over evidence (Liu & Johnson, 2006 p. 331). Improvement in the hiring process
could produce significant improvement in student achievement (Heneman & Milanowski, 2004).
There are regulations designed to ensure the quality instruction through the ongoing evaluation
and professional development of teachers (Chingos & Peterson, 2011). The findings of this study
indicate systemic efforts must be made to the processes that actually place teachers in schools in
the first place.
The impact of bounded rationality on teacher hiring practices requires the attention of
educational leaders. Suboptimal decision-making caused by missing data, the misinterpretation
of data, or overconfidence generated by the presence of some data could undermine the crucial
work of the hiring process and, therefore, the quality of instruction provided to students.
132
References
Adams, R. J., & Ericsson, A. E. (2000). Introduction to cognitive processes of expert pilots. Human
Performance in Extreme Environments, 5(1), 44-62.
Agee, J. (2009). Developing qualitative research questions: A reflective process. International Journal of
Qualitative Studies in Education (QSE), 22(4), 431-447.
Arnold, V., Clark, N., Collier, P., Leech, S., & Sutton, S. (2006). The differential use and effect of
knowledge-based system explanations in novice and expert judgment decisions. MIS Quarterly,
30(1), 79-97.
Bacon, D. L., Fulton, B. J., & Malott, R. W. (1983). Improving staff performance through the use of task
checklists. Journal of Organizational Behavior Management, 4(3), 17-25.
doi:10.1300/J075v04n03_03
Bartels, S. M., & Mortenson, B. P. (2005). Enhancing adherence to a problem-solving model for middle-
school pre-referral teams: A performance feedback and checklist approach. Journal of Applied
School Psychology, 22(1), 109-123. doi:10.1300/J370v22n01_06
Bekker, H. L. (2010). The loss of reason in patient decision aid research: Do checklists damage the
quality of informed choice interventions? Patient Education and Counseling, 78(3), 357-364.
doi:10.1016/j.pec.2010.01.002
Bernheim, B., & Rangel, A. (2004). Addiction and cue-triggered decision processes. The American
Economic Review, 94(5), 1558-1590.
Beshears, J., & Gino, F. (2015). Leaders as decision architects. Harvard Business Review, 93(5), 52-62.
Bingham, C. B., & Haleblian, J. (. (2012). How firms learn heuristics: Uncovering missing components of
organizational learning. Strategic Entrepreneurship Journal, 6(2), 152-177. doi:10.1002/sej.1132
133
Botterill, L. C., & Hindmoor, A. (2012). Turtles all the way down: Bounded rationality in an evidence-
based age. Policy Studies, 33(5), 367-379. doi:10.1080/01442872.2011.626315
Brañas-Garza, P., García-Muñoz, T., & González, R. H. (2012). Cognitive effort in the beauty contest
game. Journal of Economic Behavior & Organization, 83(2), 254-260.
doi:http://dx.doi.org.ezproxy.neu.edu/10.1016/j.jebo.2012.05.018
Briscoe, F. M. (2005). A question of representation in educational discourse: Multiplicities and
intersections of identities and positionalities. Educational Studies, 38(1), 23-41.
doi:10.1207/s15326993es3801_4
Busenitz, L. W., & Barney, J. B. (1997). Differences between entrepreneurs and managers in large
organizations: Biases and heuristics in strategic decision-making. Journal of Business Venturing,
12(1), 9-30.
Butin, D. W. (2010). The education dissertation: A guide for practitioner scholars. Thousand Oaks, CA
US: Corwin Press. Retrieved from
http://ezproxy.neu.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN
=2009-23444-000&site=ehost-live
Cain-Caston, M. (1999). A survey of opinions of north carolina school administrators regarding factors
considered most important in hiring teachers for their first teaching position.(statistical data
included). Journal of Instructional Psychology, 26(2), 69.
Campbell, W. K., Goodie, A. S., & Foster, J. D. (2004). Narcissism, confidence, and risk attitude. Journal
of Behavioral Decision Making, 17(4), 297-311. doi:10.1002/bdm.475
Campitelli, G., & Gerrans, P. (2014). Does the cognitive reflection test measure cognitive reflection? A
mathematical modeling approach. Memory & Cognition, 42(3), 434-447.
134
Carlin, B. I., Gervais, S., & Manso, G. (2013). Libertarian paternalism, information production, and
financial decision making. Review of Financial Studies, 26(9), 2204-2228.
Chingos, M. M., & Peterson, P. E. (2011). It's easier to pick a good teacher than to train one: Familiar and
new results on the correlates of teacher effectiveness. Economics of Education Review, 30(3), 449-
465. doi:http://dx.doi.org.ezproxy.neu.edu/10.1016/j.econedurev.2010.12.010
Churchman, J. J., & Doherty, C. (2010). Nurses' views on challenging doctors' practice in an acute
hospital. Nursing Standard, 24(40), 42-47.
Cioffi, J., & Markham, R. (1997). Clinical decision-making by midwives: Managing case complexity.
Journal of Advanced Nursing, 25(2), 265-272. doi:10.1046/j.1365-2648.1997.1997025265.x
Clement, M. C. (2000). Essentials for principals: How to interview, hire, and retain high- quality new
teachers
Clement, M. C. (2009). Hiring highly qualified teachers begins with quality interviews: Past behavior is
the best predictor of future performance, so educators would be wise to craft interview questions that
explore past experiences, skills, and behaviors of job candidates. Phi Delta Kappan, 91, 22+.
Clement, M. C. (2013). Hiring good colleagues: What you need to know about interviewing new teachers.
The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 86(3), 99-102.
doi:10.1080/00098655.2013.769930
Cooley, V. E., Shen, J., Miller, D. S., Winograd, P. N., Rainey, J. M., Yuan, W., & Ryan, L. (2006). Data-
based decision-making: Three state-level educational leadership initiatives. Educational Horizons,
85(1), 57-64.
Courtney, H., Lovallo, D., & Clarke, C. (2013). Deciding how to decide. (cover story). Harvard Business
Review, 91(11), 62-70.
135
Cranston, J. A. (2012). Evaluating prospects: The criteria used to hire new teachers. Alberta Journal of
Educational Research, 58(3), 350-367.
Creswell, J. W. (2012). Educational research : Planning, conducting, and evaluating quantitative and
qualitative research. Boston: Pearson.
Darling-Hammond, L. (2010). Teacher education and the american future. Journal of Teacher Education,
61(1), 35.
Eisenhardt, K. M., & Zbaracki, M. J. (1992). Strategic decision making. Strategic Management Journal,
13, 17-37.
Engel, M. (2013). Problematic preferences? A mixed method examination of principals' preferences for
teacher characteristics in chicago. Educational Administration Quarterly, 49(1), 52-91.
doi:10.1177/0013161X12451025
Engel, M., & Cannata, M. (2015). Localism and teacher labor markets: How geography and decision
making may contribute to inequality. Peabody Journal of Education, 90(1), 84-92.
doi:10.1080/0161956X.2015.988533
Engel, M., & Finch, M. A. (2015). Staffing the classroom: How urban principals find teachers and make
hiring decisions. Leadership and Policy in Schools, 14(1), 12-41.
doi:10.1080/15700763.2014.983131
Flottorp, S. A., Oxman, A. D., Krause, J., Musila, N. R., Wensing, M., Godycki-Cwirko, M., . . . Eccles,
M. P. (2013). A checklist for identifying determinants of practice: A systematic review and synthesis
of frameworks and taxonomies of factors that prevent or enable improvements in healthcare
professional practice. Implementation Science, 8(1), 1-11. doi:10.1186/1748-5908-8-35
Frederick, S. (2005). Cognitive reflection and decision making. The Journal of Economic Perspectives,
19(4), 25-42.
136
Fudickar, A., Hörle, K., Wiltfang, J. ö, & Bein, B. (2012). The effect of the WHO surgical safety
checklist on complication rate and communication. Deutsches Ärzteblatt International, 109(42),
695-701.
Funk, K. M., & Axelrod, S. (2011). Review of 'the checklist manifesto—How to get things right'. Child &
Family Behavior Therapy, 33(1), 70-72. doi:10.1080/07317107.2011.545016
Gentry, R. (2012). Implementing promising practices to prepare quality teacher educators. Journal of
College Teaching & Learning, 9(3), 179-188.
Gigerenzer, G. (2010). Moral satisficing: Rethinking moral behavior as bounded rationality. Topics in
Cognitive Science, 2(3), 528-554. doi:10.1111/j.1756-8765.2010.01094.x
Grantcharov, T. P., & Reznick, R. K. (2009). Training tomorrow’s surgeons: What are we looking for and
how can we achieve it? ANZ Journal of Surgery, 79(3), 104-107. doi:10.1111/j.1445-
2197.2008.04823.x
Guerra-López, I., & Blake, A. M. (2011). Leadership decision making and the use of data. Performance
Improvement Quarterly, 24(2), 89-104. doi:10.1002/piq.20113
Haberman, M. (2012). The myth of the "fully qualified" bright young teacher.(author abstract). American
Behavioral Scientist, 56(7), 926-940.
Halevy, N., & Chou, E. Y. (2014). How decisions happen: Focal points and blind spots in interdependent
decision making. Journal of Personality and Social Psychology, 106(3), 398-417.
doi:10.1037/a0035351
Hanushek, E. A. (2011). The economic value of higher teacher quality. Economics of Education Review,
30(3), 466-479. doi:10.1016/j.econedurev.2010.12.006
Hargreaves, A., & Fullen, M. (2009). Change wars. Bloomington: Solution Tree.
137
Harris, D. N., & Sass, T. R. (2011). Teacher training, teacher quality and student achievement. Journal of
Public Economics, 95(7), 798-812. doi:10.1016/j.jpubeco.2010.11.009
Haskins, R., & Loeb, S. (2007). A plan to improve the quality of teaching. The Education Digest, 73(1),
51-56.
Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A. S., Dellinger, E. P., . . . Gawande,
A. A. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. N
Engl J Med, 360(5), 491-499. doi:10.1056/NEJMsa0810119
Heneman, H. G., & Milanowski, A. T. (2004). Alignment of human resource practices and teacher
performance competency. Peabody Journal of Education, 79(4), 108-125.
doi:10.1207/s15327930pje7904_6
Henningsen, D. D., & Henningsen, M. L. M. (2007). Do groups know what they don't know? dealing with
missing information in decision-making groups. Communication Research, 34(5), 507-525.
doi:10.1177/0093650207305594
Hoffman, A. R. (1995). Landing the job: A survey of new teachers. Education, 116(2), 279.
Hoppe, E. I., & Kusterer, D. J. (2011). Behavioral biases and cognitive reflection. Economics Letters,
110(2), 97-100. doi:http://dx.doi.org.ezproxy.neu.edu/10.1016/j.econlet.2010.11.015
Hutchinson, J. W., Alba, J. W., & Eisenstein, E. M. (2010). Heuristics and biases in data-based decision
making: Effects of experience, training, and graphical data displays. Journal of Marketing Research,
47(4), 627-642. doi:10.1509/jmkr.47.4.627
Imai, K., & Yamamoto, T. (2010). Causal inference with differential measurement error: Nonparametric
identification and sensitivity analysis. American Journal of Political Science, 54(2), 543-560.
doi:10.1111/j.1540-5907.2010.00446.x
138
Janis, I. L. (. L. (1982). Groupthink : Psychological studies of policy decisions and fiascoes (2nd ed. ed.).
Boston: Boston : Houghton Mifflin.
Jenlink, P. M. (2005). Editorial: On bricolage and the intellectual work of the scholar- practitioner.
Scholar-Practitioner Quarterly, 3(1), 3.
Jevons, W. S. (1888). The theory of political economy London: Macmillan and Co. ; New York:
Macmillan and Co.
Jopp, D. A. (2005). Formal and informal collective learning systems: Understanding the emergent flow of
knowledge (Ed.D.). Available from ProQuest Dissertations & Theses Full Text. (304998750).
Retrieved from http://search.proquest.com.ezproxy.neu.edu/docview/304998750?accountid=12826
Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American
Psychologist, 58(9), 697-720.
Kahneman, D. (2011). Thinking, fast and slow (1st ed. ed.). New York: New York : Farrar, Straus and
Giroux.
Kahneman, D., Lovallo, D., & Sibony, O. (2011). Before you make that big decision. Harvard Business
Review, 89(6), 50-60.
Kahneman, D., & Tversky, A. (1984). Choices, values, and frames. American Psychologist, 39(4), 341-
350. doi:10.1037/0003-066X.39.4.341
Kayes, P. E. (2006). New paradigms for diversifying faculty and staff in higher education: Uncovering
cultural biases in the search and hiring process. Multicultural Education, 14(2), 65-69.
Kersten, T. (2010). Using a group process in teacher hiring. School Administrator, 67(8), 36-37.
Kunc, M. H., & Morecroft, J. D. W. (2010). Managerial decision making and firm performance under a
resource-based paradigm. Strategic Management Journal, 31(11), 1164-1182.
139
Lange, C., Range, B., & Welsh, K. (2012). Conditions for effective data use to improve schools:
Recommendations for school leaders. International Journal of Educational Leadership Preparation,
7(3)
Langlois, R. N. (1990). Bounded rationality and behavioralism: A clarification and critique. Journal of
Institutional and Theoretical Economics (JITE) / Zeitschrift Für Die Gesamte Staatswissenschaft,
146(4), 691-695.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park: Sage Publications.
Liu, E., & Johnson, S. M. (2006). New teachers' experiences of hiring: Late, rushed, and information-
poor. Educational Administration Quarterly, 42(3), 324-360. doi:10.1177/0013161X05282610
Lovallo, D. & Sibony, O. (2010). The case for behavioral strategy. McKinsey Quarterly, 2(3), 30-43.
Lovallo, D., Clarke, C., & Camerer, C. (2012). Robust analogizing and the outside view: Two empirical
tests of case-based decision making. Strategic Management Journal, 33(5), 496-512.
doi:10.1002/smj.962
Machi, L. A., & McEvoy, B. T. (2012). The literature review: Six steps to success. second edition
Corwin.
Mackenzie, C. (1997). Where are the motives? A problem with evidence in the work of richard thaler.
Journal of Economic Psychology, 18(1), 123-135. doi:10.1016/S0167-4870(96)00050-5
Major, M. L. (2013). How they decide: A case study examining the decision-making process for keeping
or cutting music in a K–12 public school district. Journal of Research in Music Education, 61(1), 5-
25.
Manimala, M. J. (1992). Entrepreneurial heuristics - a comparison between high pi (pioneering-
innovative) and low pi ventures Elsevier Science BV.
140
Marzano, R., Waters, T., & McNulty, B. (2005). School leadership that works: From research to results.
Alexandria, VA: ASCD.
Mason, R. W., & Schroeder, M. P. (2010). Principal hiring practices: Toward a reduction of uncertainty.
The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 83(5), 186-193.
doi:10.1080/00098650903583727
Medin, D., & Bazerman, M. (1999). Broadening behavioral decision research: Multiple levels of
cognitive processing. Psychonomic Bulletin & Review, 6(4), 533-546. doi:10.3758/BF03212961
Ménard, J. (2010). A ‘Nudge’ for public health ethics: Libertarian paternalism as a framework for ethical
analysis of public health interventions? Public Health Ethics, 3(3), 229-238.
doi:10.1093/phe/phq024
Mumby, D. K., & Putnam, L. L. (1992). The politics of emotion: A feminist reading of bounded
rationality. The Academy of Management Review, 17(3), 465-486. doi:10.2307/258719
Murnane, R. J., & Steele, J. L. (2007). What is the problem? the challenge of providing effective teachers
for all children. The Future of Children, 17(1), 15-43.
6A:16-8.2 functions of intervention and referral services, NJ 6A:16-8.2U.S.C. 6A:16-8.2 (October 2014).
Retrieved from http://www.state.nj.us/education/code/current/title6a/chap16.pdf
NJDOE. (2016). Educator preparation provider annual report. Retrieved from
http://www.state.nj.us/education/educators/rpr/preparation/providers/2016/provider.shtml
Nobre, F. S., Tobias, A. M., & Walker, D. S. (2009). The impact of cognitive machines on complex
decisions and organizational change. AI & Society, 24(4), 365-381. doi:10.1007/s00146-009-0207-4
Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1),
14-37.
141
Oppenheimer, D. (2012). Hard-to-read fonts promote better recall. Harvard Business Review, 90(3), 32-
33.
Our future, our teachers. (2011, 10/14; 2016/6). ED Review Retrieved from
http://libraries.state.ma.us.ezproxy.neu.edu/login?gwurl=http://go.galegroup.com.ezproxy.neu.edu/p
s/i.do?id=GALE%7CA271817555&v=2.1&u=mlin_b_northest&it=r&p=ITOF&sw=w&asid=ad0f9
9bee68b95a7038c9a56a94f5a6e
Papa, F., & Baxter, I. (2008). Hiring teachers in new york's public schools: Can the principal make a
difference? Leadership and Policy in Schools, 7(1), 87-117. doi:10.1080/15700760701655524
Peterson, K. D. (2002). In ebrary I. (Ed.), Effective teacher hiring a guide to getting the best. Alexandria,
Va.: Alexandria, Va. : Association for Supervision and Curriculum Development.
Pillsbury, P. (2005). Only the best: Hiring outstanding teachers: A great curriculum in the hands of a
mediocre teacher--even one with a credential--is nothing more than a mediocre curriculum.
Leadership, 35(2), 36.
Place, A. W., & Vail, D. S. (2013). The effects of age, years of experience, and type of experience in the
teacher selection process. AASA Journal of Scholarship & Practice, 10, 8+.
Ponterotto, J. G. (2005). Qualitative research in counseling psychology: A primer on research paradigms
and philosophy of science. Journal of Counseling Psychology, 52(2), 126-136.
Power, D. J., & Sharda, R. (2007). Model-driven decision support systems: Concepts and research
directions. Decision Support Systems, 43(3), 1044-1061. doi:10.1016/j.dss.2005.05.030
Rabin, M., & Schrag, J. L. (1999). First impressions matter: A model of confirmatory bias. The Quarterly
Journal of Economics, 114(1), 37-82.
Reeves, D. B., & Flach, T. (2011). Data: Meaningful analysis can rescue schools from drowning in data.
Journal of Staff Development, 32(4), 34-38,.
142
Rowan, B. (1994a). Comparing teachers' work with work in other occupations: Notes on the professional
status of teaching. Educational Researcher, 23(6), 4-17. doi:10.3102/0013189X023006004
Rowan, B. (1994b). Comparing teachers' work with work in other occupations: Notes on the professional
status of teaching. Educational Researcher, 23(6), 4-17. doi:10.3102/0013189X023006004
Rutledge, S., Harris, D., Thompson, C., & Ingle, W. K. (2008). Certify, blink, hire: An examination of the
process and tools of teacher screening and selection. Leadership and Policy in Schools, 7(3), 237-
263. doi:10.1080/15700760701822132
Ryan, F., Coughlan, M., & Cronin, P. (2007). Step-by-step guide to critiquing research. part 2: Qualitative
research. British Journal of Nursing, 16(12), 738-744.
Saldana, J. (2009). The coding manual for qualitative researchers. London ; Thousand Oaks, Calif.:
London ; Thousand Oaks, Calif. : Sage.
Schlag, P. (2010). Nudge, choice architecture, and libertarian paternalism. Michigan Law Review, 108(6),
913-924.
Schumacher, G., Grigsby, B., & Vesey, W. (2015). Determining effective teaching behaviors through the
hiring process. International Journal of Educational Management, 29(1), 139-155.
doi:10.1108/IJEM-04-2013-0071
Schwandt, D. R. (2005). When managers become philosophers: Integrating learning with sensemaking
Shadish, W. R., & Cook, T. D. (1999). Comment-design rules: More steps toward a complete theory of
quasi- experimentation. Statistical Science, 14(3), 294-300.
Shen, J., Cooley, V. E., Ma, X., Reeves, P. L., Burt, W. L., Rainey, J. M., & Yuan, W. (2012). Data-
informed decision making on high-impact strategies: Developing and validating an instrument for
principals. Journal of Experimental Education, 80(1), 1-25. doi:10.1080/00220973.2010.550338
143
Short, D. C., & Shindell, T. J. (2009). Defining HRD scholar-practitioners. Advances in Developing
Human Resources, 11(4), 472-485. doi:10.1177/1523422309342225
Sibbald, M., De Bruin, A., & Van Merrienboer, J. (2013). Checklists improve experts' diagnostic
decisions
. Medical Education, 47(3), 301-308.
Simon, H. (2000). Bounded rationality in social science: Today and tomorrow. Mind & Society, 1(1), 25-
39. doi:10.1007/BF02512227
Simon, H. A. (1955). A behavioral model of rational choice. The Quarterly Journal of Economics, 69(1),
99-118.
Simon, H. A. (1982). Models of bounded rationality. Cambridge, Mass.: Cambridge, Mass. : MIT Press.
Simon, H. A. (1991). Bounded rationality and organizational learning. Organization Science, 2(1), 125-
134.
Singh, J. (2008). Impostors masquerading as leaders: Can the contagion be contained? Journal of
Business Ethics, 82(3), 733-745. doi:10.1007/s10551-007-9588-9
Sterman, J. D. (1989). Modeling managerial behavior: Misperceptions of feedback in a dynamic decision
making experiment
Stronge, J. H., Ward, T. J., & Grant, L. W. (2011). What makes good teachers good?: A cross-case
analysis of the connection between teacher effectiveness and student achievement. Journal of
Teacher Education, 62(4), 339-355. doi:10.1177/0022487111404241
Stubenrauch, S., Schneid, E., Wünsch, A., Helmes, A., Bertz, H., Fritzsche, K., . . . Gölz, T. (2012).
Development and evaluation of a checklist assessing communication skills of oncologists: The
COM-ON-checklist. Journal of Evaluation in Clinical Practice, 18(2), 225-230. doi:10.1111/j.1365-
2753.2010.01556.x
144
Sunstein, C. R., Kahneman, D., Schkade, D., & Ritov, I. (2002). Predictably incoherent judgments.
Stanford Law Review, 54(6), 1153-1215.
Sunstein, C. R., & Thaler, R. (2003). Libertarian paternalism. American Economic Review,
Takala, R. S. K., Pauniaho, S. L., Kotkansalo, A., Helmio, P., Blomgren, K., Helminen, M., . . . Ikonen,
T. S. (2011). A pilot study of the implementation of WHO surgical checklist in finland:
Improvements in activities and communication. Acta Anaesthesiologica Scandinavica, 55(10), 1206-
1214. doi:10.1111/j.1399-6576.2011.02525.x
Thaler, R. H. (2009). In Sunstein C. R. (Ed.), Nudge : Improving decisions about health, wealth, and
happiness (Rev. and expanded ed. ed.). New York: New York : Penguin Books.
Thaler, R. H., & Sunstein, C. R. (2003). Libertarian paternalism. The American Economic Review, 93(2,
Papers and Proceedings of the One Hundred Fifteenth Annual Meeting of the American Economic
Association, Washington, DC, January 3-5, 2003), 175-179.
Timmons, L. J. (1998, 01; 2016/9). Interviewing techniques for hiring quality employees in the OR.67,
222+. Retrieved from
http://libraries.state.ma.us.ezproxy.neu.edu/login?gwurl=http://go.galegroup.com.ezproxy.neu.edu/p
s/i.do?p=AONE&sw=w&u=mlin_b_northest&v=2.1&it=r&id=GALE%7CA20215366&asid=18c51
145a92b324e11c2e91c25380489
Toplak, M. E., West, R. F., & Stanovich, K. E. (2014). Assessing miserly information processing: An
expansion of the cognitive reflection test. Thinking & Reasoning, 20(2), 147-168.
doi:10.1080/13546783.2013.844729
Toplak, M., West, R., & Stanovich, K. (2011). The cognitive reflection test as a predictor of performance
on heuristics-and-biases tasks. Memory & Cognition, 39(7), 1275-1289. doi:10.3758/s13421-011-
0104-1
145
Tufford, L., & Newman, P. (2012). Bracketing in qualitative research. Qualitative Social Work, 11(1), 80-
96. doi:10.1177/1473325010368316
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science,
185(4157), 1124-1131.
Title II — preparing, training, and recruiting high quality teachers and principals, SEC. 2101 | SEC. 2102
| SEC. 2103 (09/15/2004). Retrieved from http://www2.ed.gov/policy/elsec/leg/esea02/pg20.html
Higher education opportunity act: Teacher quality enhancement, 201 (8/14/2008). Retrieved from
https://www.gpo.gov/fdsys/pkg/PLAW-110publ315/html/PLAW-110publ315.htm
Venkatraman, V., Payne, J. W., & Huettel, S. A. (2014). An overall probability of winning heuristic for
complex risky decisions: Choice and eye fixation evidence. Organizational Behavior and Human
Decision Processes, 125(2), 73-87. doi:10.1016/j.obhdp.2014.06.003
Waddell, D., & Sohal, A. S. (1994). Forecasting: The key to managerial decision making. Management
Decision, 32(1), 41-49. doi:10.1108/00251749410050697
Whitworth, J., Jones, S., Deering, T., & Hardy, S. (2016). Hiring quality teachers: The devil is in the
details. Professional Educator, 40(1), 20-32.
Wilson, S. M. (2010). Decision-making by school psychologists: Use of the representativeness heuristic
and importance of assessment data in determination of special education eligibility (Ph.D.).
Available from ProQuest Dissertations & Theses Global. (746787076). Retrieved from
http://search.proquest.com.ezproxy.neu.edu/docview/746787076?accountid=12826
World health organization: Patient safety. (2009). Retrieved from
http://www.who.int/patientsafety/safesurgery/en/
Yin, R. K. (2014). Case study research : Design and methods (5th ed. ed.). Los Angeles, Calif.: Los
Angeles, Calif. : Sage Publications.
146
Yin, R. K., & Davis, D. (2007). Adding new dimensions to case study evaluations: The case of evaluating
comprehensive reforms. New Directions for Evaluation, 2007(113), 75-93. doi:10.1002/ev.216
Young, I., & Joseph, G. (1989). Effects of chronological age, skill obsolescence, quality of information,
and focal position on evaluations by principals. Journal of Personnel Evaluation in Education, 2(4),
355-365. doi:10.1007/BF00139650
Young, I. P., & Delli, D. A. (2002). The validity of the teacher perceiver interview for predicting
performance of classroom teachers. Educational Administration Quarterly, 38(5), 586-612.
147
Appendix A
Permission Letter to Superintendent of Schools
Dear ________ County Superintendents,
I am writing to you today, not as a fellow ___________ County administrator, but as a doctoral
student. I am in the thesis writing phase of my proposal at Northeastern University. The purpose
of this letter is to request consent to conduct a case study of teacher hiring practices.
This researcher will explore the challenges involved in identifying effective teachers during the
hiring process. The researcher will study the research question, “How does bounded rationality
affect administrators’ decision-making processes in the process of hiring new teachers among
comparable school districts in the mid-Atlantic region?” using a qualitative design. In this study,
the researcher will seek to understand administrators’ beliefs and actions related to hiring and
examine what they choose to do as well as what they choose not to do.
Data sets are to include: a survey of administrators hiring practices and participant interest,
interviews with two administrators per school district across a minimum of four school districts
and a focus group consisting of members of the eight administrators. Prior to beginning data
collection, Northeastern University’s IRB will have approved all letters, surveys, consent forms,
and survey and interview questions.
If you have any questions regarding this study, please contact me directly at (609) 412-0237 or
via email at [email protected], or my advisor, Dr. Margaret Dougherty via email at
Sincerely,
Michael Hinman
148
Appendix B
Participant Recruitment Letter (email)
Dear Colleagues,
I am writing to you today, not as a fellow ___________ County administrator, but as a doctoral student. I
am in the thesis writing phase of my proposal at Northeastern University. The purpose of this letter is to
request consent to conduct a case study of teacher hiring practices.
As you know, a great deal of our professional time is focused on the evaluation and development of
teachers. I believe an equal amount of research should examine the manner in which teachers enter the
profession in the first place. Therefore, I plan to explore the challenges involved in identifying effective
teachers during the hiring process. I will study this question using a qualitative design, specifically, a case
study approach. In this study, I will seek to understand administrators’ beliefs and actions related to
hiring.
I am asking you complete the brief survey below as it serves two important functions. First, it allows me
to collect data as to the number of teaching positions principals hire annually as well as the processes used
to do so. Second, the survey will allow me to recruit interested principals involved in the hiring of
teachers for a follow up interview. The ninth question of the survey asks if you will be willing to
participate in a one hour follow up interview to be scheduled at your convenience. Completing the brief
online survey does not require participation as an interviewee. From the pool of interested participants, I
will select eight principals across a minimum of four school districts with varied experiences in teacher
hiring. The final question of the survey asks if you would also be willing to join with the other selected
principals for a focus group discussion. Again, this would be under an hour and scheduled at the mutual
convenience of the educators who volunteer. Survey: https://www.surveymonkey.com/r/6GYGZN8
I and hope you will consider helping me in this study. Please be aware that agreeing or not agreeing to
participate in this study will in no way impact your employment or our relationship as colleagues. Also,
any participation in the study will be completely confidential; names and other personal information will
not be used; likewise the school, even the county or state, will not be named. You can refuse to answer
any question and you may discontinue your participation in this research program at any time without
penalty or costs of any nature, character, or kind.
Data sets are to include: a survey of administrators hiring practices and participant interest, interviews
with one or two administrators per school district across a minimum of four school districts, a focus group
discussion with a subset of the selected administrators. Prior to beginning data collection, Northeastern
University’s IRB will have approved all letters, surveys, consent forms, and survey and interview
questions.
If you have any questions regarding this study, please contact me directly at (609) 412-0237 or via email
at [email protected], or my advisor, Dr. Margaret Dougherty via email at
Sincerely,
Michael Hinman
149
Appendix C
Signed Informed Consent Document of School Staff Participants
Northeastern University, College of Professional Studies
Investigator Name: Michael Hinman (Margaret Dougherty, Academic Advisor)
Title of Project: Hiring Fast and Slow: A Case Study of Teacher Hiring Practices Viewed
Through the Lens of Bounded Rationality
Why am I being asked to take part in this research study?
You have been asked to participate in this study because you have been actively involved in the
hiring of teachers.
Why is this research study being done?
This researcher will explore the challenges involved in identifying effective teachers during the
hiring process.
What will I be asked to do?
The researcher will be looking for you to participate in the following ways:
1. Complete a brief survey
2. Engage in an individual interview that will be audio taped and transcribed
3. Read over transcribed audiotapes to ensure accuracy
4. Join a follow up focus group discussion
Where will this take place and how much time will it take?
1. The survey will take approximately 10 minutes to complete
2. The interview will take approximately one hour, and will take place at a place and time
convenient for you as the interviewee.
3. The focus group discussion will take approximately one hour, and will take place at a
place and time coordinated to be as convenient as possible for the members.
Will there be any risk or discomfort to me?
There are no significant risks involved in being a participant in this study.
Will I benefit by being in this research?
There are no direct benefits to you for participating in this study. It is hoped that the results of the
study may illuminate teacher hiring practices thus improving the percentage of the best teachers
hired for the limited positions available.
Who will see the information about me?
Your part in the study will be completely confidential. Pseudonyms will be used for all study
participants. Only the researcher will be aware of the participants' identities. No reports or
publications will use information that can identify you, the school or any individual in any way.
All audiotapes will be destroyed following transcription and analysis.
150
If I do not want to take part in the study, what choices do I have?
You are not required to take part in this study. If you do not want to participate, you do not sign
this form.
Can I stop my participation in this study?
Participation in this study is voluntary, and your participation or non-participation will not in any
way affect other relationships (e.g., employer, school, etc.). You can refuse to answer any
question and you may discontinue your participation in this research program at any time without
penalty or costs of any nature, character, or kind.
Who can I contact if I have questions or problems?
Michael Hinman, ABD, Northeastern University (609) 412-0237 or via email at
Dr. Margaret Dougherty, Northeastern University, College of Professional Studies via email at
Who can I contact about my rights as a participant?
If you have any questions about your rights as a participant, you may contact Nan C. Regina,
Director, Human Subject Research Protection, 960 Renaissance Park, Northeastern University
Boston, MA 02115 tel. 617-373-4588, email: [email protected]. You may call anonymously if you
wish.
Will I be paid for my participation?
You will not be paid for your participation in this study.
Will it cost me anything to participate?
There is no cost to participate in this study.
I have read, understood, and had the opportunity to ask questions regarding this consent
form. I agree to participate in this study on a voluntary basis.
____________________________________ ____________________
Research Participant (Signature) Date
____________________________________
Research Participant (Printed Name)
____________________________________ ____________________
Signature of the researcher, Michael Hinman Date
151
Appendix D
Survey
(This survey will be used to identify participants.
All names and locations selected for this study will be confidential.)
Name:
District:
School:
1. How many years have you been a principal?
a. 1-5
b. 6-9
c. 10+
2. Approximately, how many teachers do you hire in a year?
a. 1-5
b. 6-9
c. 10+
3. How important are the following sources of information when reviewing teacher
applications? 1= somewhat, 2 = important, 3 = very important, N/A
a. Certifications
b. In-district application forms
c. Resumes
d. Transcripts
e. Recommendations, formal or informal
f. Prior knowledge of candidate
4. To what degree are teachers hired through screening committees?
a. Never
b. Sometimes
c. Usually
d. Almost always
e. Always
5. If so, who is included in these committees?
a. Administrators
b. Teachers
c. Parents/community members
d. Other _________________________
6. To what degree are interview protocols or processes (the list of questions, time frames and
manner of interviews, etc. ) used for all teacher candidates?
a. Never
b. Sometimes
c. Usually
d. Almost always
152
e. Always
7. If so, to what degree are those protocols and processes used by all administrators in your
district?
a. Never
b. Sometimes
c. Usually
d. Almost always
e. Always
8. Would you be willing to be a participant of this study engaging in an hour long interview
on this topic as described in the accompanying Request for Participation? You may rescind
your agreement to participate at any time.
a. Yes (Please provide your email here__________)
b. No
c. Contact me first as I have questions regarding participation
9. If so, would you also be willing to be a member of a focus group of up to 8 principals who
were also members of the study to discuss the preliminary themes identified in the study.
This group would meet for 1 hour in a location central to the county participants.
a. Yes (Please provide your email here__________)
b. No
c. Contact me first as I have questions regarding participation
153
Appendix E
Individual Semi-Structured Interview Protocol
Participant _______________________
Location_________________________
Date ____________________________
Prior to an interview session, the researcher has confirmed survey data and original informed
consent form. _______________ (Researcher initials) ______________ (Participant initials)
Interview questions and probes
1. (Anchoring/Halo Effect)
a. What is most important to you in a teacher’s submitted application?
b. How does that help you find the most effective teacher?
c. Have you ever had a hiring committee member who held onto a first impression (good or
bad)? How did you manage that?
2. (Missing Data Bias)
a. Do you find you have all of the information you need to choose the best candidate?
b. How do you manage the task of considering all of the information?
3. (Affect Bias)
a. What do you look for during teacher interviews?
b. How does that help you find the most effective teacher?
4. (Confirmation Bias)
a. Have you ever had a committee member select data as evidence of the preference he/she
had before the interviews began?
b. If so, how did you manage that?
5. (Availability Bias)
a. Have you ever had a committee member express a dislike for a candidate because the
candidate reminded the committee member of a previous poor hire (graduated from the
same university, lived in the same town, etc.)?
6. (Sunk Cost Effect / Loss Aversion)
a. What experiences have you had in which after a long interview process, you did not have
a candidate that you were confident in hiring?
b. Did you restart the hiring process? Why or Why not?
7. (Decision-making Tools)
a. Does your school/district utilize protocols or processes for teacher hiring?
154
b. Do your processes improve your teacher hiring? If so, how?
c. Do you wish it were different? If so, how?
d. Expert decision makers such as surgeons and pilots have been shown to improve their
decisions through the use of checklists. Do you believe that might also be true for the
expert decision-making of principals in the hiring process? Why or Why not?
8. (Decision-making Tools)
a. Do you always use committees in the hiring process?
b. If so, how do they operate?
c. How are candidates ranked and selected?
d. Who makes the final decision, the principal, or the committee?
e. Is someone charged with ensuring protocols are followed in the process?
9. (Decision-making Tools)
a. Consider your teacher hiring process, what is working/what do you wish to change?
155
Appendix F
Focus Group Semi-Structured Interview Protocol
Participant _______________________
Location_________________________
Date ____________________________
Prior to this focus group session, the researcher has confirmed survey data and initial informed
consent form.
______________ (Researcher initials)
______________ (Participant initials)
1. In our first interview, we discussed the process of selecting candidates to interview from
the pool of applicants. How do you feel about success of that process?
2. Can you recall an experience when a candidate seemed very promising on paper, but not
in the interview? What did he/she do/not do in the interview?
3. When you come to the conclusion of the interview process, do you find your committee
members rank the candidates in the same order? If not, what do you do?
4. Are you interested in changing your interview process? If so, in what ways?