���������������������� ����� ��

���������������� �

Ron S. Kenett

KPA Ltd., IsraelDepartment of Statistics and Applied Mathematics, University of Torino, Italy

�������

• Education in Statistics is preparing for statistical analysis but not necessarily for statistical consulting.

• The objective of the presentation is to explore the phases that precede and follow statistical analysis.

• Specifically these include: Problem elicitation, data collection and, following statistical data analysis, formulation of findings and presentation of findings and recommendations.

• Some insights derived from a literature review and real life case studies will be provided and areas for joint research by statisticians, engineers, social scientists and cognitive scientists will be outlined.

• Key words: statistical consulting, graphs, problem solving, interpersonal skills

������ ����� �������

Problemelicitation

Datacollection

Dataanalysis

Findings’formulation

Findings’presentation

1

2

3

4

5

Kenett, R. and Thyregod, P. (2006) Aspects of statistical consulting not taught by academia, Statistica Neerlandica, special issue on Industrial Statistics, 30, 3, pp. 396-412.

������� �

Modern IndustrialStatistics: Design and controlof quality and reliability

Software Process Quality: Management and control

ModernStatistics: A computer intensive approach

Multivariate Quality Control: Theory and applications

������� �

������� ����������

Greenfield, T. (1987), Consultant's cameos: A chapter of encounters. pp 11-25 in Hand, D.J. and B.S. Everitt eds, The statistical consultant in action, Cambridge University Press

������������

“You can observe a lot by watching”Cuthbert Daniel

“Statistics requires a different kind of thinking, because data are not just numbers, they are numbers with a context”.

Cobb, G.W. and Moore, D.S. (1997), Mathematics, statistics, and teaching, American Mathematical Monthly

104, 801-823.

������������

“the most important thing for children to prepare them for life is:”

61.5%61.5% - “to think for themselves”� good nutrition� sports activity� musical instruments� reading books� to think for themselves

4.6%4.6% - something equivalent to above in an open question

Schuman and Presser (1981)

������������

“how successful have you been in life”

34%34% - high success (“5”)

13%13% - high success (“10”)

Schwarz, Knauper, Hippler, Noelle-Neumann, and Clark, (1991)

54321

� 1065 7 84321

������������

“how often did you experience physical symptoms”

62%62% - more than twice a month

39%39% - more than twice a month

Schwarz and Scheuring, (1992)

Several times a day

Once a weekOnce a month

Twice a month

Twice a month or less

More than twice a month

Twice a month

Once a monthOnce a month or less

never

��� �����

The population of Oldenburg in Germany and the number of observed storks in 1930-1936*

���

��

����

���

��

����

��

��

���

���������Number of storks

������Population in thousands

����������������year

* Box, Hunter and Hunter (1978) Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, J. Wiley

������

�������

���������������

��

��

��

��

��

��

��

��������������������� ����������

��� �����

�������� ������� �����

*Determine which days are Sundays.*In July it was very hot for a period of 9 consecutive days, determine which days.

�������� ������� �����

*Determine which days are Sundays.*In July it was very hot for a period of 9 consecutive days, determine which days.

�������� ������� �����

*Determine which days are Sundays.*In July it was very hot for a period of 9 consecutive days, determine which days.

�� �� ��� ��� �����

Match findings and recommendations with management’s maturity level

Management Management ApproachApproach

Statistical Statistical MethodsMethods

44

33

22

11

Quality by DesignQuality by Design Designed ExperimentsDesigned Experiments

Process Control & Process Process Control & Process ImprovementImprovement

Control ChartsControl Charts

InspectionInspectionSampling PlansSampling Plans

Fire FightingFire FightingTraining in Training in

Statistical ThinkingStatistical Thinking

The Quality LadderThe Quality Ladder**

* Kenett, R.S. and Zacks (1998), Modern Industrial Statistics: Design and control of quality and reliability, Duxbury Press

�� �� ��� ��� �����

Evaluate the total impact of possible solutions

PSE = PSE = E{R}xT{I}xP{I}xP{S}xV{PS}xV{P}xV{M}xV{DE{R}xT{I}xP{I}xP{S}xV{PS}xV{P}xV{M}xV{D}}

Where:V{D} = value of the data actually collected. V{M} = value of the statistical method employed. V{P} = value of the problem to be solved. V{PS} = value of the problem actually solved.P{S} = probability that the problem actually gets solved. P{I} = probability the solution is actually implemented. T{I} = time the solution stays implemented. E{R} = expected number of replications.

* Kenett, R.S., S. Coleman and D. Stewardson (2003). Statistical Efficiency: The Practical Perspective”, Quality and Reliability Engineering International 19, 265-272.

�� �� ��� ����� ����

The US space shuttle Challenger exploded 73 The US space shuttle Challenger exploded 73 seconds after takeseconds after take--off on January, 28th, 1986, off on January, 28th, 1986, and all seven crew members were lost. and all seven crew members were lost.

Subsequent investigation determined that the Subsequent investigation determined that the cause was failure of the Ocause was failure of the O--ring seals that were ring seals that were used to isolate the fuel supply from burning used to isolate the fuel supply from burning gases. gases.

�� �� ��� ����� ����

For some time, engineers from Morton Thiokol, For some time, engineers from Morton Thiokol, manufacturers of the rocket motors, had been manufacturers of the rocket motors, had been worried about problems with the Oworried about problems with the O--ring seals. ring seals. They feared that low temperatures greatly and They feared that low temperatures greatly and adversely affected the ability of Oadversely affected the ability of O--rings to rings to create a seal on solid rocket booster joints. create a seal on solid rocket booster joints.

On the night before the flight the temperature On the night before the flight the temperature predicted at launch time was 3predicted at launch time was 3°°C, and the C, and the engineers expressed their concerns over the engineers expressed their concerns over the effect of the unseasonable cold weather on the effect of the unseasonable cold weather on the OO--rings and suggested to abort the flight. rings and suggested to abort the flight.

�� �� ��� ����� ����

A telephone conference among NASA A telephone conference among NASA engineers and managers and Thiokol engineers engineers and managers and Thiokol engineers and managers were quickly summoned. and managers were quickly summoned.

With short notice the Thiokol engineers With short notice the Thiokol engineers presented their case via 13 presented their case via 13 telefaxedtelefaxed charts and charts and their commentary and argument, but they failed their commentary and argument, but they failed to convince the managers that temperature was to convince the managers that temperature was a factor in Oa factor in O--ring performance or damage, and ring performance or damage, and it was decided to go ahead with the launching. it was decided to go ahead with the launching.

�� �� ��� ����� ����

Neither the 13 charts circulated to the Neither the 13 charts circulated to the teleconferences nor the participants' teleconferences nor the participants' commentary focused on a relationship between commentary focused on a relationship between the 22 available launch temperatures and the the 22 available launch temperatures and the 22 recordings of O22 recordings of O--ring conditions for previous ring conditions for previous launches. launches.

Nor did anyone at the teleconference ask for or Nor did anyone at the teleconference ask for or make reference to either 1) temperature data on make reference to either 1) temperature data on nonnon--damaged flights, or 2) the full set of pairs of damaged flights, or 2) the full set of pairs of temperaturetemperature--damage data points from all flights damage data points from all flights to date. to date.

3

�� �� ��� ����� ����

�� ����� ��� ���� � ��

Outright FailureExample: consultant sells whatever is in the toolkit (snake oil) without regard for client

Unstable RelationshipExample: consultant reports a correct diagnosis that the boss/client is the real problem

Expectations not met

Clean ContractExample: consultant organises and delivers a seminar, as promised

Meaningful EngagementExample: consultant diagnoses and implements changes in core operations capabilities

Expectations met

Core Needs Not Addressed

Core Needs Addressed

�� ����� ��� ���� � ��

Meaningful EngagementExample: consultant diagnoses and implements changes in core operations capabilities

Expectations met Core Needs Addressed

�� ����� ��� ���� � ��

Clean ContractExample: consultant organises and delivers a seminar, as promised

Expectations met Core Needs not Addressed

�� ����� ��� ���� � ��

Unstable RelationshipExample: consultant reports a correct diagnosis that the boss/client is the real problem

Expectations not met Core Needs Addressed

�� ����� ��� ���� � ��

Outright FailureExample: consultant sells whatever is in the toolkit (snake oil) without regard for client

Expectations not met Core Needs not Addressed

�� ����� ��� ���� � ��

The Statistical consulting challenge:

How do you increase the probability of “Meaningful

Engagements”

The Statistical education challenge:

How do you prepare students for “Meaningful

Engagements”

����������� ���� ��

��� ���� ����!�" #• Have a genuine desire to solve real problems and

help others to solve problems.• Be able to help investigators formulate their problem

in quantifiable terms.• Be able to listen carefully and to ask probing

questions.• Have a broad knowledge and true understanding of

statistical and scientific methods.• Be able to adapt existing statistical procedures to

novel environments.• Be able to locate or develop good statistical

procedures in a timely fashion.• Be able to keep abreast of developments in statistics.• Be willing to meet deadlines, even if it requires

substantial extra effort.• Be able to understand something about the clients'

subject matter and speak a bit of the clients' language.

*Encyclopedia of Statistical Sciences (2005) John Wiley and Sons, New York.

����������� ���� ��

��� ���� �����" $

• Be a good teacher—much success in consulting depends on being able to help others understand statistical tools, and their strengths and weaknesses.

• Be willing to settle for a reasonably correct approximate solution, then go on to the next problem.

• Be able to identify important problems (and thus avoid spending too much time on projects of little significance).

• Have the confidence to use as simple a procedure as will get the job done, be it design or analysis.

• Be able to convince others of the validity of a solid solution and see to it that proper action is taken.

• Be able to use computers effectively and direct others in their use.

����������� ���� ��

��� ���� �����" %

• Be a good problem solver.• Be willing to meet clients regularly on their home

ground, and take the responsibility to meet and communicate with all members of the working team.

• Be diplomatic and know when to bend, when to stand firm, and how to help smooth over conflicts among other team members.

• Be willing to get some experience in the actual collection of the data.

• Be willing to take the time to check and double-check procedures and results.

• Be able to communicate effectively in writing as well as orally (this often includes helping clients write their reports as well).

• Be able to make a good estimate of how much effort will be required to solve the problem without actually having to solve the problem itself.

StatisticianEngineer

Manager

Operator

Customer Educators