SYSTEMS SAVVY: THEORY,

MEASUREMENT, and IMPACT

Terri Griffith, Santa Clara UnivJohn E. Sawyer, Univ of Delaware

M. Scott Poole, Univ of Illinois

Slideshare.net/TerriGriffith

#AOM2014_1836

Systems savvy is the ability to grasp possible functions/affordances of

technology tools and organizational processes and how these might be

meshed to best effect.

Technology & Organizational Dimensions, with a touch of Emergence

#AOM2014_1836AP Photo/Charles Rex Arbogast

#AOM2014_1836

“[a]lthough most studies up to this point have sufficed to simply show

that social and the material are thoroughly intertwined, scholars

are just beginning to consider how such intertwinement occurs”

(Leonardi, 2012, p. 35).

#AOM2014_1836

We provide a model of systems savvy and a field validated

measurement tool

(Situational Judgment Task)

• Personal Innovativeness in IT– Agarwal & Prasad 1998– Thatcher & Perrewe 2002– Wang, Hsieh, Butler, & Hsu 2008

• Mann Gulch: Weick 1993 • Expertise

– Johnson and Mervis 1997– Murphy and Wright 1984– Tanaka & Taylor 1991

• Practical Wisdom: Aristotle

Scenarios

Interviews

Outcome Ratings

Focus Groups

6Scaled

Scenarios

Experts Novices

Field Validated Measure

Sternberg, et al. 2000Weekley,Ployhart, & Holtz 2006

#AOM2014_1836

Field Validated Scenarios

#AOM2014_1836

Technology, Organization, & Emergence

Related to research on the positive relationships between organizational change and tolerance for ambiguity and internal locus of control (e.g., Judge, Thoresen, Pucik, & Welbourne, 1999).

toe to e avgT+avgO0

0.5

1

1.5

2

2.5

3

3.5

4

Social Media

SysSavvyNovice

Avg Ranking of Response by Sample

#AOM2014_1836

Comparison of SJT with Likert Scale

Likert Example: When I adopt a new technology,I always consider other changes in my workflow that might help. E.g., MacDonald & Uncles (2007) consumer savvy scale

#AOM2014_1836

Binary Logistic Regression Results With Expert Status Regressed On

SJT Score

Model Summary-2 Log

likelihoodCox & Snell R Square1

Nagelkerke R Square2

148.523a .229 .378aChi-square=57.449, df=1, p<.001

Classification Tablea

Predicted Percent

Correctnovice savvy

Observednovice 179 3 98.4savvy 20 19 48.7

Overall Percent Correct Classification 89.6a. The cut value is .500

SJT scale alone accurately classifies 89.6% of the respondents, while random

assignment would differentiate only 50%.

#AOM2014_1836

Incremental Classification Accuracy of SJT Score Over Likert Scale Only

Model SummaryStep -2 Log

likelihoodCox & Snell R Square

Nagelkerke R Square

1 153.051a .213 .3512 109.815a .353 .582

Likert Scale Classification Table Predicted Percent

Correct novice savvy

Step 1Observed novice 177 5 97.3

savvy 26 13 33.3Overall Percent 86.0

SJT and Likert scale Classification Table Predicted Percentag

e Correctnovice savvy

Step 2Observed novice 177 5 97.3

savvy 14 25 64.1Overall Percent 91.4

SJT scale adds nothing to the correct classification of novices, but doubles the correct classification of system savvy individuals. Correct Classification of 91.4%

#AOM2014_1836

Future Research• Tool Use in Teams– Thomas & Bostrom 2010– Majchrzak, Rice, Malhotra, King, & Ba, 2000– Maznevski & Chudoba, 2000 – Montoya, Massey, & Lockwood, 2011

• Organizational Memory: Argote 2013• Strategy -- IT Leveraging Competence – Pavlou & El Sawy 2006

• Technology Design – TSA, Healthcare.gov

t@terrigriffith.comsawyerj@udel.edu

mspoole@illinois.eduSlideshare.net/TerriGriffith

#AOM2014_1836

US: HealthCare.gov

Thank You

Many Individuals Throughout

#AOM2014_1836

Preference Ordering of Scenario Responses

Combined technology-organization-emergent (scenario responses offering intertwined human and technical dimensions with the acknowledgement of possible adjustments over time)

Combined technology-organizations

Emergent

Organizations only

Technology only

3

1

2

4.5