the simple elegance of causal enquiry robert o. briggs institute for collaboration science...

The Simple Elegance of Causal Enquiry

Robert O. BriggsInstitute for Collaboration ScienceUniversity of Nebraska at Omaha

[email protected]

Today’s program

Epistemology The Philosophy of Science The Scientific Approach

Three Ways to Think About Academia

The Philosophical The Pragmatic The Publishable

Epistemology:The Philosophical View

A way of knowing A way of creating knowledge

Prevailing Epistomologies

Interpretivism Criticalism Causal enquiry

Interpretivism

Creating knowledge about… The inferences people draw

from and the meanings people ascribe to the words and actions of others.

Key Assumption: No objective reality

Key Discipline: Induction

Criticalism

Creating knowledge about… Social Justice Key Assumption:

Injustice is socially and historically constituted

Systemic injustice can be redressed through action

Causal (Science)

Creating knowledge about… Cause-and-effect Key assumption: Objective

reality Key Discipline: Deduction

Epistemology Myths

Causal and Interpretivist enquiry are mutually exclusive world views

Objective Reality vs. No Objective Reality?

What is Reality?

Epistemology Myths

Causal scholars are (or think they are) objective. Reality is objective Scientists are subjective Validation = Intersubjective concurrence

Interpretivists don’t believe in gravity Interpretivists seek to explain meaning,

not causation.

Epistemology Myths

Interpretivism is qualitative Causal enquiry is

quantitative

Epistemology Myths

An epistemology is something you “are” I’m an interpretivist I’m a positivist

Pragmatic Epistemology

A set of mental disciplines To keep us from drawing (and then

publishing) bone-headed conclusions

No Embarrassment

Epistemology: The Publishing Perspective

Bad PapersAre Forever!

Interpretivism, Criticalism, and Causal epistemology (science) are not mutually exclusive. They are, in fact, interdependent; all three are necessary to a complete understanding of Information Systems

Provocation

The Philosophy of Science

Assumptions of Causal Enquiry

Regular patterns of causation

Independent from human mind

“knowable”

The Boundaries of Science

If it’s not about cause-and-effect…

It’s not science Period.

Goals of Science

Discover and describe phenomena and their correlates

Create causal models for phenomena of interest

Test the usefulness of those models Use those models to increase the

likelihood people will survive and thrive.

Goals of Science Discover and describe phenomena of

interest (Exploratory Science) Create causal models for phenomena of

interest (Theoretical) Test the usefulness of the models

(Experimental Science) Use those models to increase the

likelihood people will survive and thrive. (Applied Science / Engineering)

XKED.com

In which aspect of scientific method is this scientist engaged?

A.ExploratoryB.TheoreticalC.ExperimentalD.Applied Science / Engineering

The three most exciting words in science are, “Gee, that’s funny…”

- Isaac Asimov

•Why did that happen?

•Does it always happen that way?

•Does it have to happen that way?

•Could I make it happen on purpose?

Why Should You care?

Good science will Make it more likely that people will survive and

thrive Make you work smart Get you published in good journals

Bad Science will Harm others Waste effort, time, and money Embarrass you for years…

The Causal Disciplines

Phenomenon-of-Interest Who Cares? Theory Hypotheses Research Methods Analysis

The First Discipline

Explicitly Define The Phenomenon of Interest

The Phenomenon of Interest

In the world of cause-and-effect… The phenomenon-of-interest is the

EFFECT The EFFECT is what you seek to explain The EFFECT is what you seek to

improve The EFFECT is the outcome you

measure

The First DisciplineDefine The Phenomenon of Interest

Explicitly In writing(!) Refine the definition as your

understanding deepens Challenge your definition

continuously

Explicitly Define thePhenomenon of Interest

Satisfaction First definition:

The degree to which needs are fulfilled Measures

I am satisfied My needs are fulfilled I feel satisfied

Better definition A valanced affective arousal with respect to goal

attainment Measures

I feel satisfied with… …gave me a feeling of satisfaction I feel good about…

Phenomenon vs. Domain:The Philosophical View

The phenomenon-of-interest is the OUTCOME you hope to improve measurably Productivity Creativity

The domain is the setting in which the outcome manifests Requirements Negotiation Data Mining

Phenomenon vs. DomainThe Pragmatic View

You study the phenomenon of interest …Don’t ever forget it

You sell the domain To funding agencies To reviewers To readers To yourself

The Second Discipline

Who Cares?!?

Who Cares?!?

Why is this phenomenon-of-interest is worthy of study?

Philosophical “Who cares?”

A goal of scientific enquiry is to increase the likelihood that people will survive and thrive

Society provides the scarce resources for scientific enquiry. You must be able to justify your use of them.

Pragmatic “Who Cares”

Reviewer Perspective: “What I do is important. What you do is trivial.”

Your reviewer just had a “much better” paper rejected by the same journal.

Publishable “Who Cares?”

1. The phenomenon of interest is worth studying1.1 People are more likely to survive

and thrive if we understand the cause of this phenomenon

1.2 The existing literature does not fully explain the causes of this phenomenon

Publishable “Who cares?”

You must define explicitly the phenomenon-of-interest in the who-cares argument

It’s your anchor for all that follows You are the most important target

for the argument

Good “Who Cares?” 1.1 Organizations exist to create value for

stakeholders Organizations operate under risk Mitigate risk, the organization may survive Internal risk assessments can mitigate risk Risk assessments must be run by groups If we can make risk assessment groups more

productive, we may increase that people will survive and thrive!

Productivity is…. This study examines the use of GSS to make

risk assessment groups more productive.

Bad “Who cares” 1.1 Organizations do risk assessments

frequently We studied collaborative risk

assessment workshops

Ugly “Who Cares” 1.1

We collected some data about risk assessment workshops

Good “Who Cares 1.2” Connolly et al (1992) showed that productivity

of brainstorming teams could be improved by making them anonymous.

However, Johnson and Stephens (2003) found higher productivity when brainstorming teams were identified

A causal theory of productivity might be useful for explaining these seemingly disparate results, and might allow the development of even better brainstorming techniques.

This paper offers and tests such a theory

Bad “Who Cares” 1.2 Jones (1983) said nothing has been

done about productivity Smith (1978) called for more

research on productivity Johnson (1981) studied productivity

among factory workers I studied productivity among

brainstorming groups

Ugly “Who Cares” 1.2

I searched 3 on-line databases and browsed 6 web search engines and only found 2 articles on this topic.

Little is known about this topic Nobody has studied this topic yet.

Publishable Causal enquiryThe Opening Argument

Section 1. Who Cares?!?Argument: This phenomenon is worth studying.

1.1 People will be better off if we understand this phenomenon

1.2 Current literature does not yet fully explain it

The Third Discipline

Deriving An Explanatory

Theory

‘theory’ With a Small ‘T’

Exploratory Taxonomies Frameworks Descriptive models Correlational models (grounded theory)

Applied Design Theory (design methodologies)

Explanatory Theory

A causal model to explain variations in the phenomenon-of-interest

Data have no scientific meaning

except with respect to

the Explanatory Theory

from which they spring

Today’s Message:Today’s Message:

Goals of Science Discover and describe phenomena of

interest (Exploratory) Create causal models for phenomena

of interest (Theory) Test the usefulness of the models

(Experiment) Use those models to increase the

likelihood people will survive and thrive. (Application)

Anything Missing?

TruthTruthTruthTruth

Anything Missing?

Positivist Perspective

Science = Useful Science = Useful

Science <> TrueScience <> True

A useful model is better than Truth

Useful Is Better Than True

Useful Is Better Than True

Name the PhenomenonName the Phenomenon

BobeziteBlock

Describe the PhenomenonDescribe the Phenomenon

BobeziteBobeziteBlockBlock

A

B

Explore the PhenomenonExplore the

Phenomenon


A

B


Explore the PhenomenonExplore the

Phenomenon

BobeziteBobeziteBlockBlockBobeziteBobezite

BlockBlockBobeziteBobeziteBlockBlockBobeziteBobezite

BlockBlock

A

B


A

B

Describe the dynamics of the phenomenon

Describe the dynamics of the phenomenon

A Useful ModelA Useful Model

One Gear

TruthTruth

One Thousand Gears

When does the Model Become Useful?

When does the Model Become Useful?

When you want todo something newWhen you want todo something new

Therefore

For matters of cause-and-effectA useful model (Theory)

is better than Truth

An experiment, without a Theory is

meaningless

What is a Theory?

An excuse to not do anything meaningful?

Pie-in-the-sky disconnect from reality?

There is nothingmore useful

than a good theory

Theory All Drives:

Hypothesis Experimental design Measures Treatments Statistics

What is a theory?

Causal Model Internally Consistent Explains and/or predicts Proposes mechanisms of causation Testable

Structure of a Theory

Axioms Propositions

Axioms

Assumptions about the fundamental nature of the universe

The beginning of suppositional logic What if we were to assume the world

works like X…would that explain observed variations in Y?

Axioms Are “Received”

Source is irrelevant Axioms cannot be derived or defended Feynman’s Inspiration

Example Axioms

Axiom 1: Human attention is limited Axiom 2: A subconscious cognitive mechanism ascribes utility to salient goals

Propositions Functional Statements of cause-and-

effect that must be logically true if the axioms are true

Force is a function of Mass and Acceleration (F=MA)

Satisfaction is a function of shifts in yield for the set of salient goals.

n

jj

m

ii YYfS

11

Propositions are...

Causal Composed of constructs Without empirical content Logically derivable from

axioms

Propositions of Direct Causation

Proposition 1: Productivity is a function of effortProposition 2: Effort is a function of goal congruenceProposition 3: Effort is an inverse function of distraction

ProductivityEffort

Distraction

Goal Congruence

+

-

+12

3

Debugging A Theory

EffortDesire for outcome

+1

Proposition 1: Effort is a function of desire-for-outcome.

Propositions of Moderating Causation

EffortPerceived

Effort Required

Desire for Outcome

Mathematical Notation of Propositions

P = (E)Where

P = Productivity

E = Effort

E = -(D)Where

E = EffortD = Distraction

Problematic Propositions

WORK DESIGN & EXECUTION OUTCOMES

IMPLICIT INCENTIVES

EXPLICITINCENTIVES

SOCIAL ENVIRONMENT

TECHNICAL ENVIRONMENT

RESOURCE ENVIRONMENT

ORGANIZATIONAL STRUCTURE

ENVIRONMENT

DISTRIBUTED WORK

ARRANGEMENT

ORGANIZATIONAL LEVEL

GROUP LEVEL

INDIVIDUAL LEVEL

INCENTIVE STRATEGY

(e.g. Reward & Compensation)

WORK DESIGN & EXECUTION OUTCOMES

IMPLICIT INCENTIVES

EXPLICITINCENTIVES

SOCIAL ENVIRONMENT

TECHNICAL ENVIRONMENT

RESOURCE ENVIRONMENT

ORGANIZATIONAL STRUCTURE

ENVIRONMENT

DISTRIBUTED WORK

ARRANGEMENT

ORGANIZATIONAL LEVEL

GROUP LEVEL

INDIVIDUAL LEVEL

INCENTIVE STRATEGY

(e.g. Reward & Compensation)

Qualities of a Good Theory

Parsimony Explanation/Prediction Boundaries

Pragmatic Theory

You usually start with propositions and work backward to axioms

You usually start badly and get better

You use someone else’s theory whenever you can

Your technology is not in your theory

Pragmatic Theory

A good theory will get you to the moon and back safely on the first try

Good theory will do more to save you from drawing bone-headed conclusions than any other discipline of Causal enquiry

Good theory will make you look like a genius

Publishing PerspectiveAlternative Wordings for Propositions

Y is a function of Z Z causes Y Z determine Y The more Z you do, the more Y you

get Z has a positive influence on Y

Publishable Causal enquiry

Section 2. TheoryArgument: I understand what causes Y

If we assume that:Axiom 1: The world is like X

Then it must be that: Proposition 1: Y is a function of Z.

The Fourth Discipline

Deriving Hypotheses

The Fourth DisciplineHypotheses Comparative statements

Contrasts value of a dependent variable across at least two treatments that instantiate differing values of the independent variable

Dependent Variable Measures consequent construct of a proposition

Independent Variable Invokes differing levels of causal

construct

Hypotheses

MUST be logically derived from propositions

Test the proposition Should have empirical content

Example Hypothesis

H1: Brainstorming teams with access to an automated feedback graph will produce more unique ideas than teams with no automated graph

Example Hypothesis

H2: During brainstorming, the more we pound randomly on the walls, the fewer unique ideas a team will produce.

Problematic Hypotheses

H3: Groups using richer media will exhibit higher levels of cohesion initially

Problematic Hypotheses

H4: On negotiation tasks, face-to-face groups will outperform computer mediated groups, will experience less process difficulty, than computer-mediated groups, and will have more favorable reactions to their group task performance, interaction process, and communication medium


Section 4. HypothesesArgument: This theory is testable

If, as Proposition 1 posits, Y is a function of Z, and if I do W with Technology-1 to invoke higher values of Z then it must be that: H1. People doing W with Technology-1 will score higher on the Y-test than people doing NOT-W

The Fifth Discipline

Experimental Research Methods

Experimental Design

Construct Validity Statistical Validity Internal Validity External Validity

An experiment without a theory is meaningless

Today’s Message:Today’s Message:

Experiment

Compare outcomes Different treatments Control other possible causes

Experimental InquiryExperimental Inquiry

TreatmentTreatment11




IdenticalSubjectPools

IdenticalSubjectPools

ResultsResultsResultsResults


}} CompareCompare

Investigative InquiryInvestigative Inquiry

PopulationPopulation11






}} CompareCompareOneTreat-ment

OneTreat-ment

Positive Experimental Results may mean...

Manipulation caused difference Hypothesis has support Theory has support

Negative Results Mean

Experiment Flawed? Hypothesis Flawed? Propositions Flawed? Axioms Broken?

The Only Scientific Truth

The Model is No Good


Section 4. MethodsArgument: I found a reasonable way to test the hypotheses

4.1 My DV instantiates the phenomenon of interest4.2 My IV instantiates a causal construct4.3 My approach would reveal a difference if there were one4.4 There are few alternative explanations for any difference discovered

An Experiment without a theory is

meaningless

Phenomena: Phenomena: Large, Odd-Smelling BoxesLarge, Odd-Smelling Boxes

Scientific Instrument: Scientific Instrument: DrillDrill

Collecting Data without A TheoryCollecting Data without A Theory

Collecting Data Without A TheoryCollecting Data Without A Theory

Collecting Data With a TheoryCollecting Data With a Theory

A Physicist Uses the A Physicist Uses the Elephant TheoryElephant Theory

+

=Fission!

A Farmer Uses the Elephant A Farmer Uses the Elephant TheoryTheory

A Farmer Uses the TheoryA Farmer Uses the Theory

There is nothingmore useful

than A Good Theory

An Experiment without a theory is

meaningless

Data have no meaning except in reference to the

theory from which they spring.

Points to Ponder

You don’t have to measure a cause, you only have to manipulate it.

You have to measure effects For experimental research you must have a

theoretical explanation for every effect you measure

Not required for exploratory work

Scientific Method

Discover Phenomenon Theorize Hypothesize Fastest Falsifications Experiment Conclude Apply

Truth

Powerful theory will outperform powerful statistics every time!

Truth There is No Perfect Study You must pilot your experiments

Truth No Theory is made or broken by a single study

Remember

Experiments without theories are meaningless

Remember

Data Have No Meaning except in reference to the

theory from which they spring

Worth Reading Stebbins, Robert. Exploratory Research in the

Social Sciences. 2001. Popper, Karl -- The Logic of Scientific Discovery

(Any edition from 1934-1981) Shadish, W.R., Cook, T.D., & Campbell, D.T.

(2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton-Mifflin.

Hevner, A. and Chatterjee, S. (2010). Design Science in Information Systems: Theory and Practice.

the simple elegance of causal enquiry robert o. briggs institute for collaboration science...

Documents