critical thinking in dod acquisitions...team, led by wag dodge, tasked to put out a lightning caused...
TRANSCRIPT
Source:Unsplash.com
Dave Riel• Professor of Acquisition Management
since January 2010• Sr. Mgr for Manufacturing and CI• 20-yrs USAF career• Enjoy thinking about stuff in a different
way– Photographer– Traveler
3
DoD Report to Congress (Aug 17)
Comments regarding “Acquisition Workforce”
“…teaching … emphasis needs to be on critical thinking, risk management, flexible decision-making and working with Warfighters to evaluate trades pursuant to delivering capability … requires culture change and the re-education of our workforce”
Restructuring the DoD(AT&L) Organization and Chief Management Officer Organization
ASPECTS OF CRITICAL THINKING:THINKING FAST AND SLOW
“He had an impression, but some of his impressions are illusions”
- Daniel Kahneman
Source:smallbusiness.ng
COPYRIGHT NOTICE: This course includes content from a variety of sources to provide DAU students with a comprehensive representation of relevant industry information. Use of this content is for educational purposes only. Modification, reproduction and distribution are prohibited and may be subject to penalty under the law. More information on terms of use of each of the items can be found at their respective websites. By accessing, you acknowledge and accept these terms.
TWO MODES OF THINKING
System 2 Thinking
System 1 Thinking
Source:Unsplash.com
Source:Docorman.com
Source: Amazon.com
5
CRITICAL THINKING DEFINITION
“Reasonable reflective thinking aimed at deciding what to
believe or what to do”- Robert Ennis
“Analyzing and evaluating our thinking with a view to improving it”
- Linda Elder
6
V 15.1
Adaptive Acquisition Framework
“we didn’t want a one-size fits all approach. Instead, we have developed an AAF that allows PMs to choose the right pathway for work they are undertaking to deliver capability to Warfighter as quickly as possible.”
Ms. Stacy Cummings, DASD, Acq Enablers 2019 “The goal of the AAF is to empower innovation and common-sense decision making through the decision-making process, while also maintaining discipline in our practices and procedures..”
Mr. Kevin Fahey, ASD, Acquisition 2019
“This policy embraces the delegation of decision-making tailoring program oversight to minimize unnecessary bureaucratic processes and actively managing risks based on the unique characteristics of the capability being acquired”
Ms. Ellen Lord, USD (A&S) 2019
7
V 15.1
Adaptive Acquisition Framework
“we didn’t want a one-size fits all approach. Instead, we have developed an AAF that allows PMs to choose the right pathway for work they are undertaking to deliver capability to Warfighter as quickly as possible.”
Ms. Stacy Cummings, DASD, Acq Enablers 2019 “The goal of the AAF is to empower innovation and common-sense decision making through the decision-making process, while also maintaining discipline in our practices and procedures..”
Mr. Kevin Fahey, ASD, Acquisition 2019
“This policy embraces the delegation of decision-making tailoring program oversight to minimize unnecessary bureaucratic processes and actively managing risks based on the unique characteristics of the capability being acquired”
Ms. Ellen Lord, USD (A&S) 2019
System 2System 2
System 1?
8
IntuitiveA
cts
on u
s
• Substitutes easy questions for hard questions
• Believes things that are easy to believe
• Operates using habits
9
10
READ THIS
Source:Thinking Fast and Slow
11
READ THIS
Source:Thinking Fast and Slow
12
READ THIS
ANNAPPROACHED
THE BANK
Source:Thinking Fast and Slow
ANNAPPROACHED
THE BANK
Source:Unsplash.com
Source:Unsplash.com13
SYSTEM 1
Source: partshowpartbusiness.com
14
SYSTEM 1
Source: partshowpartbusiness.com15
As printed on the page, is the figure on the right larger than the figure on the left?
Source: unexpectedpages.com16
As printed on the page, is the figure on the right larger than the figure on the left?
Source: unexpectedpages.com17
WHICH LINE IS LONGER?
Muller-Lyer illusioon18
WHICH LINE IS LONGER?
Muller-Lyer illusioon19
ISN’T SYSTEM 1 AMAZING?
20
7H15 M3554G3 53RV35 7O PROV3 H0W 0UR M1ND5 C4N D0 4M4Z1NG
7H1NG5. 1MPR3551V3 7H1NG5! D0 Y0U UND3R574ND 7H15 M3554G3 W3LL 3N0UGH 70 1D3N715Y 7H3
NUM83R 831NG U53D 70 R3PL4C3 ‘A’?
READ THIS!
21
SYSTEM 1 ROCKS!
7H15 M3554G3 53RV35 7O PROV3 H0W 0UR M1ND5 C4N D0 4M4Z1NG 7H1NG5.
1MPR3551V3 7H1NG5! D0 Y0U UND3R574ND 7H15 M3554G3 W3LL 3N0UGH 70 1D3N715Y 7H3 NUM83R
831NG U53D 70 R3PL4C3 ‘A’?
This Message Serves To Prove How Our Minds Can Do Amazing Things.
Impressive Things! Do You UnderstandThis Message Well Enough To Identify
The Number Being Used To Replace “A?”22
SYSTEM 1 THINKING
“Autopilot” -- no conscious effort Draws on heuristics (a.k.a. mental
shortcuts, rules of thumb, associations) to make decisions
Extremely easy & efficient -- brain can move on
Why not use this thinking all the time?
Source arizonalawgroup.com
Source: amazon.com
23
24
Source: Men In Black
25
SYSTEM 1 VS. SYSTEM 2
12 X 12 = ?
26
SYSTEM 1 VS. SYSTEM 2
13 X 13 = ?
DeliberateW
e ac
t on
it
Thinks through the issue but burns a lot of glucose along the way
Source:Unsplash.com
Source: CreativeCommons.org
Source: poly.google.com
Source:clipartmag.com
Source:Unsplash.com
27
28
TRAGEDY AT MANN GULCH
• August 5, 1949
• Extremely dry
• 97 degrees
• 16 Smokejumpers
• Routine 5:00 fire
1
2
Source: wordpress.com
But…then the routine took a turn for the “worse”
Source: wordpress.com
29
30
TRAGEDY AT MANN GULCH
https://www.youtube.com/watch?v=iHNl3EucGYQ
31
SYSTEM 2
123 x 38 = ?
What acquisition pathway or contracting strategy is best for my program?
What US strategy would be most effective in protecting US people and economy from future pandemics?
SYSTEM 2 THINKING
Challenges your heuristics Seeks to validate your “intuition” or develop new
associations that improve your decision or beliefs Grows your System 1 bench strength
Source:Unsplash.com
Source:advantageperformance.com32
WHEN SYSTEM 1 AND SYSTEM 2 INTERACT
Source: amazon.com
Source: empoweredsoul.com
33
The normal state of your mind is that you have intuitive feelings and opinions about almost everything that comes your way. You dislike people long before you know much about
them. You trust or distrust strangers without knowing why. You feel that an enterprise is bound to succeed with
analyzing it.Whether you state them or not, you often have answers to questions that you do not completely understand, relying on evidence that can neither explain or defend.
-- Thinking Fast and Slow (pg. 97)
34
35
Four MBTI Preference Scales
Extraversion – IntroversionE - I Dichotomy
Where do you prefer to focus your attention – and get your energy?
Sensing – Intuition S - N Dichotomy
How do you prefer to take in information?
Thinking – Feeling T - F Dichotomy
How do you make decisions?
Judging – PerceivingJ - P Dichotomy
How do you deal with the outer world?
Everyone Can Do Both, But One Is Usually Preferred And Better Developed
36
“Extroverts have little or no grasp of introversion. They assume that company, especially their own, is always welcome. They cannot imagine why someone would need to be alone; indeed, they often take umbrage at the suggestion. As often as I have tried to explain the matter to extroverts, I have never sensed that any of them really understood. They listen for a moment and then go back to barking and yipping.”
-- Jonathan Rauch, Caring for Your Introvert
Tension builds on the Lewis & Clark Expedition
There is often a natural tensionthat builds up between ‘s
and ‘s
EI
Source: Unknown
37
Can You Find Differences…
Sensing Intuition
MELVIN VASQUEZ/RD.COM, SHUTTERSTOCK
38
Sensing (S) vs. Intuitive (N)
Source: gocomics.com
39
40
Do you sense being micromanaged?
Source: clipart-library.comSource: clipart-library.com
41
Source: clipart-library.comSource: clipart-library.com
Do you sense being micromanaged?Might it be something else?
42
It’s NOT About the Nail
Source: YouTube
T F
Spontaneously Criticizes
Spontaneously Appreciates
“Ok, that was better…next time we’ll practice landing right-side up!”
T’s May Seem Cold & Condescending To F’s
F’s May Seem Fuzzy-Minded & Emotional To T’s
43
Source: meme.com
Source: Unknown
Characteristics and Tendencies
The Thinking-Feeling Pair is the only MBTI dichotomy that is gender related. 75% of all men have a preference for Thinking
and 75% of all women, have preference for Feeling
Interesting Information
44
Judging Perceiving
46
Do you find yourself to be more spontaneous or more organized?
Source: US News and World ReportPhoto by Getty Images/Vetta
Likes Clear Limits And Categories
Likes Freedom To Explore Without Limits
J’s May Seem Demanding, Rigid & Uptight to P’s
P’s May Seem Disorganized, Messy & Irresponsible To J’s
J PCharacteristics and Tendencies
Source: unsplash.com Source: unsplash.com
47
Source: wkfr.com
48
System 1
Where thinking occurs in your brain
Thinking is hard work so the brain uses mental shortcuts or “rules of thumb” (AKA “heuristics”) to solve a problem or
make a decision quickly and efficiently
Striatum – Passive thinking occurs here!
Hippocampus – Active thinking occurs here!
Source: healthcmi.com Source: ucla.edu
49
Heuristics Can Help You
How do you catch a fly ball?
Note: Wind speed and ball spin is not considered in this equation
Source: Gerd Gigerenzer
50
How do you catch a fly ball?
Note: Wind speed and ball spin is not considered in this equation
The Gaze Heuristic: With ball in the air…1. Fix your gaze on the ball2. Start running, and3. Adjust your running speed so the angle of gaze remains constant
The target question is the assessment you intend to produce. The heuristic question is the simpler question that you answer instead
Source: Gerd Gigerenzer
Heuristics Can Help You
51
SOMETIMES THIS WORKS WELL ANDSOMETIMES LEADS TO SERIOUS ERRORS …
How much would you contribute to save strays
How happy is your life?
How do you feel about the President’s COVID 19 response?
How much emotion do you feel for this dog?
What is your mood right now?
How do you feel about the President?
Target Question Heuristic Question
Source: advantagenew.com
52
THINK OF HEURISTICS LIKE SPEARFISHING
IT WILL GET YOU CLOSE, BUT YOU MAY MISS THE MARK
Source: munnscience.com
53
APPLYING TO ACQUISITION
“He likes the project, so he thinks its cost are low and its benefits are high.”
Source: NARA 6452075
54
• Program began in 1983 as a proposed long-range, LO, high-payload medium attack aircraft to replace carrier-based A-6 by mid-1990s
• On 13 Jan 88, McDonnell-Douglas/General Dynamic awarded RDTE contract– FPIF w/TP of $4.38B and CP of $4.84B– First Flight scheduled for 17 Jun 90
• Canceled 7 Jan 91 by SECDEF Cheney – largest termination in DoD history– At least $1B over budget; 8,000 lbs overweight; 18 months behind schedule– So expensive estimated to take up 70% of Navy budget within 3 yrs– Over 20 years of litigation – 1 Jun 09, ktrs ordered to pay $1.35B plus $1.45B interest
• May 10: Supreme Court set aside Appeals Court decision, sends case back to federal courts circuit
• Jan 14: Case was settled w/Boeing & General Dynamics agreeing to pay $200M each to Navy
• Six months earlier, A-12 enjoyed wide support from Congress & SECDEF
13 Jan 88Contract awarded
7Jan 91Contract
Terminated
17Jun 90First FlightScheduled
Source: NARA 6452075
55
• Clear program in trouble – largely ignored by McAir; GD; Navy and DoD– EVMS identified “significant and increasing negative cost and schedule variances”– Ktr team “made best case projections based upon overly optimistic recovery plans”
• PM’s Production Oversight team’s review expected “firm design in hand” and “assembly tooling … designed, built and on floor” by Sep 89
– Early evidence first flight date of June 1990 unattainable• NAVAIR analyst warned of >$1B over ceiling – PM didn’t believe
– Pentagon analyst reviewed reports, concluding A-12 “was in very deep trouble”– Warnings offset at Nov 89 Pentagon review by PM’s “very spirited defense of program”– OSD budget analyst report (5000# over, 2 yrs behind, $500M >) stirred controversy -- withdrawn
• Great political pressure for A-12 success– If A-12 failed, Navy would cede deep-strike bombing to USAF – jeopardizing future $4B funding for
aircraft carriers and other warships– Numerous examples where unfavorable info either dismissed or suppressed
• Due to fundamental design deficiencies, unlikely program was ever salvageable• On 28 Nov 90, Beach report concludes A-12 PM “erred in judgment”
– CAPT PM and RADM PEO removed and censured; USD(A) resigns on 13 Dec 9056
13 Jan 88Contract awarded
7Jan 91Contract
Terminated
Aug 89Cost
AnalysisIgnored
Nov 89Pentagon Review
Summer 89Production
TeamReview
Source: NARA 6452075
57
Source: clker.com
Source: nosweatpublicspeaking.com
Source: Thinking Fast and Slow
IS THE HEIGHT OF THE TALLESTREDWOOD MORE OR LESS THAN 1,200 FT?
Source: unsplash.com
58Source: unsplash.com
WHAT’S YOUR BEST GUESS ABOUT THEHEIGHT OF THE TALLEST REDWOOD?
59
BEWARE OF SYSTEM 1 ANCHORING YOU!
System 1 tries its best to construct a world in which the anchor is true!
60
BEWARE OF SYSTEM 1 ANCHORING YOU!
System 1 tries its best to construct a world in which the anchor is true!
61
SOME THOUGHTS ABOUT YOUR THINKING
System 1 is amazing: Skilled to deal with vast amounts of information
swiftly and efficientlySource: Thinking Fast and Slow
Source: Unknown
62
SOME THOUGHTS ABOUT YOUR THINKING
System 1 is amazing: Distinguishes between normal and surprise events …
generates idea of what was expected, and automatically searches for cause
Source: imgur.com
Effect of Contract Types• Law mandates MDA authorization/ documentation to use cost-type contracts in
MDAP development and prohibits cost-type contracts in MDAP production – Our view is that contract types must be tailored to the specific situation, with the risk associated
with the work to be performed being the most important factor• Analysis showed no statistical correlation between performance and broad contract type
among 433 MDAP development and 440 early production contracts (2013 Report)• Fixed-price contracts exhibit lower cost growth because they are used primarily in lower-
risk situations—not because they inherently lead to lower cost growth– Price can be higher on fixed price contract because contractor prices in as much risk as possible
to shift risk to DoD• When cost control is predetermined and formulaically incentivized, vendors respond
– Predictable incentives, not fixed pricing, is key -- simply considering fixed-price contracts is too simplistic because:
• Some cost-type contracts also have strong incentives to control cost growth• Both cost-type and fixed-price contracts can be modified, i.e., work content growth• Some fixed-price contracts do not provide information on costs, i.e., govt risks paying unnecessarily
high prices on FFP contracts– FFP contracts do not share savings with the government—even when risks are low– Using FFP contracts in full-rate production may incentivize contractors to withhold cost-reduction ideas and
investments until then, when they keep all the cost-reduction rewards– If no govt insight into these savings, subsequent production lot prices may continue to be higher than necessary
Performance of the DAS 2014 Report
(Pg. 87-92)
64
SOME THOUGHTS ABOUT YOUR THINKING
System 1 Heuristic: Ease with which instances come to mind
Not necessarily simpler or more frugal than original question
Source: MinuteVideos.com
65
SOME THOUGHTS ABOUT YOUR THINKING
System 2 Engagement: Could replace heuristic by focus on content
System 1 generally right, except when it’s not!
SOME THOUGHTS ABOUT YOUR THINKING
System 2 is who we think we are …
… but it often endorses the ideas and feelings generated by System 1
Source: teespring.com
66
CONFIRMATION BIAS
Internet is fertile ground for confirmation bias…tailored content is
causing divergent opinions to disappear altogether
Is digital technology splitting society into discrete, ideologically like-minded tribes that read, watch, or listen only to news that confirms their own beliefs?
How about you? Do you stay inside your own “echo chamber”? (Be Honest!)
Food for thought
There is enormous pressures within a military service to be optimistic about a program, and everyone who’s ever had my job has had to push back against those pressures … We’ve had a lot of disasters in acquisition because the services believed what they wanted to believe.
-- Frank Kendall, USD (AT&L), May 2015
SOME THOUGHTS ABOUT YOUR THINKING
Questioning System 1 must be intentional and consistent
Needs to become part of your System 1
1. See signs of cognitive minefield2. Slow down3. Ask System 2 to step in
Source: marriagearchitects.com
Source: i.pinimg.com
70
SOME THOUGHTS ABOUT YOUR THINKING
Organizations must encourage a culture in which people watch out for one another’s System 1 over-reliance
DoD organizational construct not ideal
Source: ieeeusa.org
71
72
MECHANISTIC ORGANIZATIONSCharacteristics
High levels of hierarchical structure and control Clearly defined roles and responsibilities Written policies and procedures Specialized, standardized tasks Centralized decision-making processes
Advantages Higher efficiencies through lessened variation Improved predictability
Disadvantages Restricted innovation, flexibility and creativity Can be problematic in a complex and changing environment
Organization designed to run like a machine in a “routinized, efficient, reliable, and predictable way”
Source: clipart.com
73
Does that organizational structure enhance or inhibit
critical thinking?
Source: twitter.com Source: twitter.com
74
So, do we throw out the mechanistic organizational
construct?
Source: clipart-library.comSource: clipart-library.com
75
MECHANISTIC ORGANIZATIONS-- REALITIES AND BENEFITS --
Retaining overall mechanistic organization construct Ensure fair treatment for employees Bounding innovators from pushing their inclinations too far …
leading to belligerence, chaos, disastrous experimentation, and unprincipled opportunism
Age and size of Department of Defense Tendency towards mechanistic behavior -- more rigid & less
adaptable to change Rigidity and cumbersome environmental change response Can promote decentralization by allowing lower-level managers
autonomy and flexibility while retaining control over strategic decisions
Use of policies and statutes as tools versus obstacles Can enhance critical thought and negate mavericks that circumvent
strictly in name of expediency Can encourage risk-taking along with its potential for failure, but
resist dispersion of boundary/control systems designed to balance warfighter needs and being good stewards of taxpayers’ money
76
77
LEARNING ORGANIZATIONS
Characteristics of learning organizations Little to no hierarchical social structure Free flow of info -- encourage new ways of doing business Continuous improvement of established procedures Change and risk-taking supported
Advantages of learning organizations Maximum flexibility Dynamically responsive to changes in environment Learns from mistake Adapts to new realities more readily Better decision-making in non-problematic situations, i.e.,
innovationOrganizations where people continually expand their capacity to create results,
where new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people are continually learning how to learn together
– Peter Senge, The Fifth Disciple: The Art and Practice of Learning Organizations78
FRAMING ASSUMPTIONS-- 2015 DODI 5000.02 (AND LATEST DODI 5000.85: MCA) --
• Any supposition (explicit or implicit) central in shaping cost, schedule, or performance expectations of acquisition program (i.e., think about your System 1) Generally a small number (3-5) of FAs with following
attributes: Critical: Significantly affects program expectations No work-arounds: Consequences cannot be easily mitigated Foundational: Not derivative of other assumptions Program specific: Not generically applicable to all programs
Created and “owned” by PM; reviewed and approved by leaders
Deepens learning from single-loop learning to double-loop learning79
80
LAYERED LEARNING ORGANIZATIONS
Tier 1
Tier 2
Tier 3
Tier 4
Tier 5
Tier 6 Tier 6
Tier 5
Tier 6 Tier 6
Tier 4
Tier 5
Tier 6
Tier 7 Tier 7
Tier 6
Tier 7
Tier 8 Tier 8
Tier 7
Tier 8 Tier 8
Tier 5
Tier 6 Tier 6
Tier 3
Tier 4
Tier 5 Tier 5
Tier 4
Tier 5 Tier 5
Tier 2
Tier 3
Tier 4 Tier 4
Tier 3
Tier 4 Tier 4
81
ADOPTING LEARNING ORGANIZATIONS
DoD Cultural Change Impractical to expect DoD as a whole to become learning organization
Differentiation and integration – challenging, but not uncommon Differentiation refers to single company operating different
organizational constructs Allows for two different constructs to co-exist -- mechanistic and learning
Integration refers to collaboration needed to achieve common goals, i.e., allows for successful co-existence
Need to incentivize cultural transition from one that desires hierarchical control to one that values innovation and encourages debate
Requires change in cultural values and artifacts Responsible risk-taking preferred and failure acceptable
82
83
WHEN DOES IT MAKE SENSE TO USE…• System 1 Thinking
• Recognizable pattern• Widely accepted principle• Perceived consequences of
delayed decision too great• Minimal
risks/consequences or opportunities
• System 2 Thinking• No heuristic to draw on• Assumptions are in
question• Source of idea/problem not
well understood• Large risks/consequences
or opportunitiesSource: ibpsychmatters.com
- Get them on the board- Use organizing tools- Create areas to think out loud
• Grow your domain knowledge• Nurture System 1 bench strength• Deepens all standards of thinking
Source: homeword.com
Source: topsoint-screener.com
Source: aliem.com
Source: unknown
Source:Unsplash.com