The Application of Cognitive Processes to Organizational Surveys:

How Informants Report About Interorganizational Relationships

Joan M. Phillips

Mendoza College of BusinessUniversity of Notre Dame

The author gratefully acknowledges the support provided by the Institute for the Study of Business Markets at Penn State.

Introduction

Conceptual Issues Trend toward fewer but more purposive strategic

partnerships Substantive Issues

Growing interest among practitioners and scholars in understanding... How these partnerships are formed Reasons for failure Determinants of successful relationships

Introduction

Methodological Issues Variables related to successful relationships are

unobservable Obtaining valid measures of these latent

constructs remains a challenge Empirical work used survey methods

Application of cognitive psychology perspectives to improve survey data

Summary of Literature Satisfaction in Vertical Marketing Systems

Growing interest in understanding interorganizational relations

Reliability and Validity of Organizational Informant Reports Obtaining valid measures of organizational contructs

remains a challenge Cognitive Aspects of Proxy Reporting

Application of cognitive theories has improved the validity of self- and proxy-reports

Research Objectives Understand how informants report about

their organizations and its relationships Cognitive processes used Data sources used

Identify the factors associated with consensus among multiple informants

Recommend strategies to improve measures (i.e., questionnaire designs)

Sample Organizations

35 Industrial Manufacturers ($5 Mil.-$4 Bil.) Automotive/Industrial Parts & Equipment [4] Chemical Products [5] Electrical/Electronic Components [4] Heavy Equipment and Machinery [5] Metal Products [6] Machine Tools, Fasteners, & Abrasives [6] Sheet Metals [5]

Informants 109 Informants Interviewed - minimum of 3 per firm

General Management [27] Sales [31] Marketing [26] Customer Service/Administrative [25]

Questionnaire

6 Objective Items % of Sales Volume from

Distribution % of Sales Volume from

Distributor 1 % of Sales Volume from

Distributor 2 Distributor 2’s Sales Volume

Rank Length of Relationship with

Distributor 1 Length of Relationship with

Distributor 2

10 Subjective Items times 2 Distributors Distributor’s Dependence Manufacturer’s Dependence Distributor’s Influence Manufacturer’s Cooperation Distributor’s Cooperation Conflict Distributor’s Communication Trust Distributor’s Performance Manufacturer’s Satisfaction

Data Collection

Face-to-face interviews Verbal protocol methodology

Cognitive “thinkalouds”

Audio recorded Transcribed Coded

Outcome Measures Cognitive Processes

Anchoring Decomposition Calculation Generalizing

Data Sources Self/Job reference Documents Internal and external

communication Participation in event Organizational cues

Consensus = Agreement among informants. Variance for Subjective Questions Relative Error (variance/mean2) for Objective Questions

Consensus Hypotheses Supported

Similar Data Sources Supported -- Use of similar data sources had a positive

effect on informant consensus. Perceived Question Difficulty

Supported for Objective Questions -- Perceived question difficulty has a negative effect on informant consensus for objective questions.

Importance of Target Firm Supported for Objective Questions -- Importance of

target firm had a positive effect on informant consensus

Consensus Hypotheses Not Supported

Similar Cognitive Processes Not Supported -- Use of similar processes did

not have an effect on informant consensus. Use of Anchoring Processes

Not Supported -- Results for distributor 1 were mixed results for distributor 2 were in the opposite direction.

Data Source Hypotheses Objective Questions

Supported -- Informants used self/job cues slightly more frequently than documents.

Supported -- The odds of using documents were greater for objective questions.

Subjective Questions Supported -- Informants relied primarily on distributor

organizational cues. Not Supported -- The odds of using self/job cues,

communication cues, and documents were greater for objective questions.

Anchoring Process Hypothesis

Use of Anchoring & Adjustment Processes Supported -- When documents were not used,

the odds of using an anchoring procedure for subjective questions was greater than for objective questions.

Contributions

Specify cognitive processes and information sources used by informants to report about interorganizational relationships

Understand how this is impacted by question type and informant characteristics

Identify the determinants of convergence among multiple informant reports

Recommend strategies for improving organizational surveys

Findings and Recommendations FINDING

Informant agreement seems to be related to: use of similar information sources

Informants use: Organizational cues for subjective questions Self/job cues and documents for objective data

RECOMMENDATION Ask specific rather than general questions

e.g., Based on last month’s sales report... e.g., Based on the written communication your firm received from

this distributor last week...

Findings and Recommendations

FINDING Use of an anchoring process is greater for

subjective questions RECOMMENDATION

Encourage use of same reference point e.g., best distributor we have now, best ever, etc.

Define scale values for informants Avoid Likert-type (e.g., SA/SD) scales

Findings and Recommendations

FINDING Informants define organizational constructs

differently (e.g., trust, performance, satisfaction, etc.)

RECOMMENDATION Avoid single-item indicators of multi-

dimensional constructs Use global scales with caution

Generalizations Can people really serve as informants?

But my experience matters most! Do we know who our informants are?

Papa Bear, Momma Bear, & Baby Bear Are we asking the right questions?

Which of your kids do you love the best? What is most important?

Trust, Trust, Trust