current issues in individual, group, and organizational level measurement : strategic management...
TRANSCRIPT
![Page 1: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/1.jpg)
Current Issues in Individual, Group, and Organizational Level
Measurement :Strategic Management
Prof. Brian BoydW. P. Carey School
Arizona State Universityhttp://www.briankboyd.com
![Page 2: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/2.jpg)
![Page 3: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/3.jpg)
RoadmapRoadmap
• Background
• State of the art
• Consequences of measurement problems
• Application
![Page 4: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/4.jpg)
BackgroundBackground
• Most of my papers have a strong measurement component, e.g.,– SMJ 90: Environmental characteristics– JMS 91: Implications for planning performance– SMJ 94: Board control– AMJ 98: Discretion and executive pay– SMJ 98: Planning systems– SMJ 01: CEO social capital
• Reviewers often profoundly uninterested in measurement
• What is the ‘state of the art’ for construct measurement in strategy?
• What are the implications of this?
![Page 5: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/5.jpg)
Strategy and Strategy and Paradigm Paradigm
DevelopmentDevelopment• In a Kuhnian sense, management is a
young discipline
• Strategy is one of the youngest subsets of management
• Empirics play a key role in advancing paradigm development
How well does strategy research How well does strategy research contribute to the goal of advancing contribute to the goal of advancing
the paradigm?the paradigm?
![Page 6: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/6.jpg)
Where Were We Two Decades Where Were We Two Decades Ago?Ago?
• Frequent use of nominal and single-item scales
• Inadequate assessment of validity
• Inexplicit treatment of strategy as an organization-level construct
Source: Venkatraman & Grant (1986) AMR
![Page 7: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/7.jpg)
Strategy & Methods:Strategy & Methods:An OverviewAn Overview
• Inattention to methodology and design (Bowen & Wiersma, 1999)
• Few replication studies (Hubbard et al., 1997)
• Insufficient power (Mone et al., 1996)
• Little emphasis on construct measurement (Hitt, Boyd & Li, 2004)
Overall, the rigor of much strategy Overall, the rigor of much strategy research is weak.research is weak.
We’ll focus on these issues
![Page 8: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/8.jpg)
A Power PrimerA Power Primer
Type I errorType I error
Type II errorType II error
Example: Population correlation = .30, a = .05Example: Population correlation = .30, a = .05
N = 30, 50% chance of successful detection
N = 50, 70% chance
N = 100, 90+% chance
![Page 9: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/9.jpg)
Power in Strategy ResearchPower in Strategy Research• Published strategy papers had only 4 in 10 chance
of successful detection – half of recommended level (Mazen, Hemmasi & Lewis, 1987)
• Review of one research stream – only 8% of studies had sufficient power (Ferguson & Ketchen, 1999)
• SMJ has lower power than other journals (Mone, Mueller & Mauland, 1996)
• Few studies report power statistics (Mone et al, 1996)
This is the This is the goodgood news… news…
![Page 10: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/10.jpg)
Blalock’s (1979) Elements of Social Blalock’s (1979) Elements of Social ProcessesProcesses
1. A theoretical language explaining causal relations between constructs
2. An operational language for examining relationships between constructs using indicators
3. An integrative theory describing causal relationships between constructs and indicators
![Page 11: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/11.jpg)
Quality of Strategy Measures?Quality of Strategy Measures?
Lack of consensus on Lack of consensus on measurement of basic measurement of basic strategy constructsstrategy constructs
• Strategic planning• Performance• Diversification• Board oversight• Environment• Discretion
Low emphasis on Low emphasis on measurementmeasurement
• Prevalence of single measures
• Sporadic reporting of reliability and validity
![Page 12: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/12.jpg)
The Role of Measurement The Role of Measurement QualityQuality
ConstructConstructxx ConstructConstructyy Hypotheses Hypotheses framedframed
IndicatorIndicatorxx IndicatorIndicatoryy Hypotheses Hypotheses testedtested
Weak indicators lead to measurement Weak indicators lead to measurement error, and attenuation. Power error, and attenuation. Power calculations are based on fully accurate calculations are based on fully accurate constructs.constructs.
In other words, the statistics presented In other words, the statistics presented earlier earlier overestimateoverestimate the power of the power of strategy researchstrategy research
![Page 13: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/13.jpg)
An ExampleAn Example
ConstructConstructxx ConstructConstructyy N = 150N = 150power = .80power = .80
0.300.30
IndicatorIndicatorxx IndicatorIndicatoryy N = 150N = 150power = .33power = .33
0.100.10
.60.60 .60.60
![Page 14: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/14.jpg)
State of the Art & Consequences of State of the Art & Consequences of Measurement Problems in Strategy Measurement Problems in Strategy
ResearchResearch
Study 1: AssessmentStudy 1: Assessment• Content analysis of
several hundred published articles
• Multiple journals• Multiple years
Study 2: ConsequencesStudy 2: Consequences• Replication of a
prominent research question
• Large sample analysis of US firms
• Manually adjust levels of measurement precision
![Page 15: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/15.jpg)
State State of the Artof the Art
![Page 16: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/16.jpg)
AssessmentAssessment• Identified all empirical strategy papers in key
outlets: AMJ, ASQ, MS, and SMJ• Sampled articles published 1998 -- 2002• Content coded articles for:
– Sample size– Measurement schemes– Reliability
• Statistical test is unit of analysis: 625 tests in 196 articles
• High inter-rater reliability on selection & coding of studies
![Page 17: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/17.jpg)
DescriptivesDescriptives
MeanMean Std. DevnStd. Devn
Sample size 2,559 12,908
IVs 5.66 5.09
Controls 8.62 30.26
Obs/Inds 236.12 1,333.82
Sample size limitations do not Sample size limitations do not appear to constrain measurementappear to constrain measurement
![Page 18: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/18.jpg)
Good News…Bad NewsGood News…Bad News
0.7
0.72
0.74
0.76
0.78
0.8
0.82
0.84
Reliability
IV
DV
Control
50%
55%
60%
65%
70%
75%
80%
85%
90%
95%
No Data
IV
DV
ControlWhile reported reliabilities are While reported reliabilities are strong, this information is typically strong, this information is typically
notnot available in the article. available in the article.
![Page 19: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/19.jpg)
Summary of Content AnalysisSummary of Content Analysis
• Most strategy studies rely on single indicators
• Limited efforts made to report reliability data, when multiple indicators are used
• Many studies report either partial or no correlation matrices, making post hoc asessment impossible
Is this a salient problem,Is this a salient problem, or a non-issue?or a non-issue?
![Page 20: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/20.jpg)
ConsequencesConsequences
![Page 21: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/21.jpg)
Why Do Firms Diversify?Why Do Firms Diversify?
• Amihud & Lev (1981): Diversification is a result of agency conditions.
• Lane, Cannella & Lubatkin (1998): No support for A&L agency model.
• Denis , Denis & Sarin (1999): Both studies contaminated by methodological flaws. Substituting improved measures led to stronger results.
““Though both sets of authors conduct Though both sets of authors conduct similar empirical tests on virtually similar empirical tests on virtually identical data, they arrive at completely identical data, they arrive at completely different conclusions.” Denis et al., 1999different conclusions.” Denis et al., 1999
![Page 22: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/22.jpg)
Methods Issues in the Agency Methods Issues in the Agency ArgumentArgument
MeasurementMeasurement• Both A&L and LCL
studies used broad ownership categories
• Denis et al found better results when using ratio level indicators
• Only single indicator measures used
PowerPower• 3 IV regression, w/ a=.05
power = .80
N = 34 for large effect76 for moderate547 for small
• LCL N=309Denis et al N=933
• Bulk of these studies had insufficient power
![Page 23: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/23.jpg)
Insiders
StockOwnership
BoardControl
Diversification
Firm Size
Duality
Owner Reps
Director Pay
Related
Unrelated
-.16 [2.2]
.11 [2.3]
(Palepu, 1985)
(Boyd, 1994)
Sales
Assets
Equity
Sample:Sample:640 640 Fortune Fortune firms, representing 50 firms, representing 50 2-digit SIC s, and 200 4-digit SICs2-digit SIC s, and 200 4-digit SICs
![Page 24: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/24.jpg)
BoardControl
Diversification
Firm Size
-.15 [2.1]-.12 [1.7]
.11 [2.3]
Related
Unrelated
(Palepu, 1985)
(Boyd, 1994)
Sales
Assets
Equity
Now, Let’s Introduce Now, Let’s Introduce Some Shadows…Some Shadows…
Duality
Insiders
StockOwnership
Owner Reps
Director Pay
![Page 25: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/25.jpg)
BoardControl
Diversification
Firm Size
.09 [2.2] .13 [2.9] -.07 [1.6]
-.05 [1.3] .05 [1.2] .06 [1.5]
.12 [2.7] -.06 [1.3] -.03 [0.7]
-.02 [0.5]
.11 [2.3]
Related
Unrelated
(Palepu, 1985)
(Boyd, 1994)
Sales
Assets
Equity
……And Some More…And Some More…Duality
Insiders
StockOwnership
Owner Reps
Director Pay
![Page 26: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/26.jpg)
Quantifying Shadows…Quantifying Shadows…
(a) Single indicator DV(a) Single indicator DV(2 models)(2 models)
• Failure to detect effect in 1 of 2 models
• R2 drops by a 10X factor
• Path coefficient drops slightly
(b) Single indicator DV (b) Single indicator DV and IV (10 models)and IV (10 models)
• Failure to detect effect in 7 of 10 models
• R2 drops by a 10X factor
• Path coefficient cut in half
Bottom Line:Bottom Line:The A&L and LCL debate is The A&L and LCL debate is
entirely an artifact of measurement entirely an artifact of measurement error and attenuation.error and attenuation.
![Page 27: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/27.jpg)
Do We Know What We Know?Do We Know What We Know?• Strategy research has placed
little emphasis on statistical power
• Construct measurement is not a priority of empirical research
• Many strategy constructs are broad
• Weak consensus regarding measurement of key concepts
How many valid research How many valid research questions have been cast questions have been cast aside due to these method aside due to these method limitations?limitations?
Sales Assets
Employees Other
Measuring Firm Size
![Page 28: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/28.jpg)
What’s Next?What’s Next?
![Page 29: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/29.jpg)
A Comparison: Marketing
Two decades agoTwo decades ago
• The field of marketing was criticized for failing to pay attention to construct validity and associated measurement issues
TodayToday
• Studies routinely provide reliabilities, factor models, and evidence of convergent and discriminant validity
Source: Jarvis, Mackenzie & Podsakoff (2005) JCR
![Page 30: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/30.jpg)
Some Easy StepsSome Easy Steps
• Sneak measurement into substantive topics
• Less attention to knee-jerk criticism (why ROE?) and more critical review of published measures
• Collect multiple measures
• Pay attention to validity
• Hang out with the OB crowd
![Page 31: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/31.jpg)
More Balance in Research FocusMore Balance in Research Focus
ConstructConstructxx ConstructConstructyy SubstantiveSubstantiveResearchResearch
IndicatorIndicatorxx IndicatorIndicatoryyConstructConstruct
MeasurementMeasurementResearchResearch
![Page 32: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/32.jpg)
Factor Models for Better Factor Models for Better MeasurementMeasurement
X X X
A
X X X
C
X X X
B
(a) Common Approach(a) Common Approach
![Page 33: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/33.jpg)
Factor Models for Better Factor Models for Better MeasurementMeasurement
X X X
A
X X X
C
X X X
B
(b) Convergent (b) Convergent andand discriminant validity discriminant validity
![Page 34: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/34.jpg)
Factor Models for Better Factor Models for Better MeasurementMeasurement
X X X X X XX X X
A
(b) CMV test(b) CMV test
![Page 35: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/35.jpg)
BoardControl
StockOwnership
Insiders
Duality
Owner Reps
Director Pay
Constructs: Cause or Effect?Constructs: Cause or Effect?
![Page 36: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/36.jpg)
Insiders
StockOwnership
Duality
Owner Reps
Director Pay
BoardControl
Constructs: Cause or Effect?Constructs: Cause or Effect?
![Page 37: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/37.jpg)
Cause or Effect? Deciding…Cause or Effect? Deciding…Item Reflective Formative
Indicators needed Representative pool Census
Directionality Construct drives indicator
Indicators drives construct
Indicators imperfect representation of construct
Yes Yes
Measurement error Indicator level Construct level
Reliabilities Good Good or bad
Sensitivity to change in indicators
Minimal Substantial
![Page 38: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/38.jpg)
What About Board Control?What About Board Control?
• Substitution, portfolios, and balancing (Rediker & Seth ’85 Finkelstein & D’Aveni ’84) all suggest reflective model
• CED of CFA is 0.97• All indicators load at .001, with
some moderate loading• Supplementary model using a
sixth indicator (CEO tenure)– Main model unaffected– 8 of 12 subsets NS: 67% versus
70% NS in 5-factor tests
BoardControl
StockOwnership
Insiders
Duality
Owner Reps
Director Pay
![Page 39: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/39.jpg)
EnvironmentDimensions
Objective Perceptual
Industry Industry
Firm Outsiders Simulation
Insiders
TMT
Middle ManagersSource: Boyd, Dess & Rasheed (1993) AMR
Slope of SalesMean of Sales
Munificence
Discretion
Levels of Levels of MeasurementMeasurement
![Page 40: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/40.jpg)
Organizational Research MethodsOrganizational Research MethodsSpecial issue on
Strategic Management
Sample topics:Sample topics:• Pitfalls of using archival proxies• Evaluation of construct validity of key metrics• Providing insights into studying ‘unobservables’• Articles/notes and empirical/conceptual papers
welcome• Click here for Call for Papers• Proposals due March 1, 2006
![Page 41: Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e0c5503460f94af43db/html5/thumbnails/41.jpg)
Web Resources
• Boyd, Gove & Hitt (2005). Construct measurement in strategic management research, SMJ
• Boyd, Gove & Hitt (2005). Consequences of measurement problems in strategic management research, SMJ
• Manuscript draft, reviews, and reviewer replies to BGH 2005a/b
• Hitt, Boyd & Li (2004). The state of strategic management research, RMSM