tmmi history, benefit and future - image.sten.or.krimage.sten.or.kr/document/2016/tmmi_track.pdf ·...

TMMi history benefit and future

Klaus Olsen CEO of TMMi Foundation

klausolsentmmiorg

Session 3

1 TMMi Foundation History

2 Benefit of using TMMi Model

3 The future of TMMi Foundation

Klaus Olsen

Author of ldquoSoftwaretest ndash how to get startedrdquo in Danish Founding member of TMMi reg Foundation CEO of the TMMi Management Executives Member of ISTQB representing Denmark Co-author of ISTQB Foundation and Advanced Syllabus Chair of ISTQB Foundation Working Group Certified ISTQB Foundation and Test Manager Advanced Certified Scrum Master

Founder and owner of the company Softwaretestdk since 2000

Has used the past 23 years to focus on software testing test process improvements teaching and mentoring

EuroSTARacute98 in Muumlnich Germany Second World Congress on Software Quality 2000 in Yokohama Japan EuroSTARacute2001 in Stockholm Sweden Quality Week 2001 in San Francisco USA EuroSTARacute2003 in Amsterdam Holland ASTA 2007 in Seoul Korea Test 2008 in New Delhi India EuroSTARacute2008 in Haag Holland ANZTB Test2009 conference Sydney Australia JSTQB カンファレンス 2010 in Tokyo Japan ASTA 2010 in Seoul Korea Czechtest 2011 in Prague Czech Republic TAPOST 2011 in Riga Latvia TMMi World Conference 2011 in Seoul Korea Czechtest 2012 in Prag Czech Republic Nordic Testing Days 2012 in Tallinn Estonia ANZTB Test2013 conference in Canberra Australia FiSTB Testing Assembly 2013 in Helsinki Finland Testing Portugal 2013 in Lisbon Portugal DSTB 2014 in Copenhagen Denmark Sixth World Congress on Software Quality 2014 in London UK Teszt amp Tea 2014 in Budapest Hungary Czechtest 2015 in Prag Czech Republic SEETEST 2015 Sofia Bulgaria

Public presentation includes

Experience with TMM - TMMi

First experience with TMM in 1997 and presentation about this at EuroSTARacute98 in Munich Germany

TMM is developed by Illinois Institute of Technology and published in two articles in US Air Force magazine Crosstalk in 1996

Meeting with Ilene Burnstein author of TMM in USA in 2000

Ilene Burnstein has described her original TMM model in detail in her book

Several assessment of companies in Denmark and Norway during the past 10 years

Participating in creating the current TMMi model

Those who cannot remember the past are condemned to

repeat itGeorge Santayana1863 in Spain ndash 1952 in Italy

He was a philosopher essayist poet and novelist

Crosby1979

CMM v10 1991

TMM1996

TPI 1997

CMM v111993

TMMDK version

2000

CMMi2000

TMMi reg

2005

TMMi reg

full version

2010

Maturity History Timeline

TMMi Foundation established2005 in London UK

TMMi Foundation - Board of Directors

Chairman

9

Les Murray Fran OrsquoHaraMac Miller

Vice-chair

Geoff Thompson

Treasure

Secretary CEO

Klaus Olsen

Elected by the member of TMMi Foundation

TMMi Foundation - Board of Advisor

10

Lev Lesokhin Brian FordBill Curtis

Steve Connell Brian Wells

Appointed by the BoD

TMMi Foundation ndash Management Executives

CEO

11

Marc Gebauer

Erik van Veenendaal

Deputy CEO

Meile Posthuma

Accreditation

Klaus Olsen Andrew Goslin

Secretary Marketing

Geoff Thompson

TMMi Professional

Training


TMMi Foundation is

33 Certified companies among them 5 from Korea TMMi Level 2

ITSTAR Co LTD (Development amp Operation Division)

Onycom Inc (QA ICT Solution IT Service Division)

TMMi Level 3 LG CNS (DC Test and Performance Test Teams)

Hanwha Thales Co LTD (RampD Division)

LIG Nex1 (Research amp Development Division)

22 training providers among them STA from Korea

15 supporting organisation

1757 LinkedIn members

1686 members of the TMMi Foundation

12

Test Maturity Model Integration TMMi reg

Level 1 Initial

Level 5 Optimization Defect Prevention

Test Process Optimization

Quality Control

Level 3 Defined Test Organization

Test Training Program

Test Life Cycle and Integration

Non-Functional Testing

Peer Reviews

Level 4 Measured Test Measurement

Software Quality Evaluation

Advanced Peer Reviews

Level 2 Managed Test Policy and Strategy

Test Planning

Test Monitoring and Control

Test Design and Execution

Test Environment

TMMi reg maturity levels and process areas

16 Process Areas

TMMi reg Structure and components

Maturity Levels

Process Areas

Generic andSpecific Goals

Generic andSpecificPractices

Indicates Testing Process Capability

Achieves Goals

contains

contains

contains Describes Implementation


Maturity Level

Process Area 1 Process Area 2 Process Area n

Specific GoalsGeneric Goals

SpecificPractices

GenericPractices


Managed

Test Policy and Strategy

Test Planning Process Area

Develop a test plan

Institutionalize aManaged Process

Establish the test plan

Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22




TMMi Foundation benefit we provide

TMMi model free available from our website in pdf

TMMi Professional CertificationTMMi Professional Syllabus from our website in pdf

TMMi Professional trial exam questions from our website in pdf

TAMAR requirement to build an assessment method

TAM TMMi Foundation provide an assessment method on license

18

How you can utilise TMMi reg

If you donacutet know where you are

a map wonacutet helpWatts Humphrey

If you donacutet know where you want to go then any road will do

TMMi reg provides the framework for anassessment of your companyacutes now situation

TMMi reg provides a map of improvement

possibilities from level 2 to level 5

1 Start by using the TMMi reg modelto locate your GPS positionon the TMMi reg Model

2 Once you know whereyou are use the TMMi reg

model as a reference for where you want to go



Study by measuring your companiestest maturity level through an assessment

Act by identifying the weakest areas in your company when using the TMMi reg Model

Plan by creating a test improvement plan based on the TMMi reg Model

Do by implementing the identified task from Act and Plan above

Plan

Do

Study

Act

Assessment ndash how they are donea story

Engaging with a company who needs

an assessment

Should represent both large and

small size projects

Should represent the different

development methods used

Selecting 3-5 projects in the company

Project Manger

Test Manger

Developer

Tester

Select 3-4 people from each project for

interview

Often also 1 or 2 persons from a

Qualtiy or Method department

X questions for each process areaRun interview with all participants

Test strategy

Test Plan

Test case examples

Test reports

And more

Get access to artefacts discussed in

interviews

Score result for each project for

all process areas

Evaluate all input from interviews and

artefacts

Create overall TMMi assessment

report with result

Create proposal for an Action Plan

based on result of assessment

Highlight good practices they

must repeat

Go through proposal action plan

Highlight areas in need of an

improvementPresent result of assessment for the

company

Let the company decide what to do

next

AssessmentStorytelling

bull 96 DDP reached in 2004

bull Better and more stable quality

bull Moved from 78 DDP to 96 DDP during 3 years

Test effectiveness = Numbe

r of Defects found during t

est total number of defect

s after 90 days

96 defect detection rate reached

Effectiveness of test in company

Time for test Effectiveness of test

96 DDP reached

Better and more stable quality

Moved from 78 to 96 DDP during 3 years

Test effectiveness = Number of Defects found during test total number of defects after 90 days

96 defect detection rate

Year Test Effectiveness in Average Time for Testof total project

200x 78 DDP 23200x+1 94 DDP 28200x+2 96 DDP 25200x+3 95 DDP 28200x+4 92 DDP 19

bull Improved higher level of quality in projects

bull Now some with a 100 DDP others between 75 and 90DDP in average DDP were 92

Metrics from a company using TMMi

Effectiveness in projects





TMMi our future

TMMi and Agile

Alumni Club Free for all TMMi Professional Certified

Able to vote on our AGM for the BoD

Able to participate in our working groups

28

Currently TMMi reg Foundation are working with the following organizations to grow recognition of the TMMi reg Model on a global scale

TMMi Professional Training providers

30

Belgium Bolivia China Denmark France Germany

Hungary India India India India Ireland

Ireland Korea Malaysia Netherlands Netherlands Scotland

Spain UK UK UK USA

Make a Difference and use TMMiClive Bates clivebatesexperimentuscom

Session 4

1 The challenges you face

2 Examples of client benefits

3 How to move forward with TMMi

4 TMMi trends

5 Donrsquot fear failure ndash embrace it

Experimentus services

3

Typical reasons for test improvement

Many Organizations are still facing tough business objectives in the current climate eg

bull Reduced time to market

bull Increased functionality and complexity

bull Increased reliability

bull Increased speed of performance

bull Increased number of users

bull Reduced prices being paid (cheaper productsapplications)

bull Need to invest in and utilizing new technologies and compete on a global stage Need to innovate or be left behind

bull Re-design of the workplace including co-developed or outsourced development

bull Meeting client requirements

Therefore

It can be said that applications are now more challenging to build

4

The result of all this is helliphellip

bull Systems and applications are playing an ever increasing part in every day life (social and business)

bull There is a greater emphasis to focus on quality aspects

bull Customers and users are becoming more and more demanding

bull Software failures are much more visible now and can have major impact on day to day activities ndash economically socially and internationally etc

bull Testing is becoming a key activity for helping to improve quality and the performance of the development process

bull The fast pace of change in the industry ndash consider what has happened over the last 5 or 10 years What will we see next

5

Whose having assessments amp where

bull Service providers

bull Mining company

bull Pharmaceutical

bull Banking

bull Financial Services

bull Retail

bull Insurance

bull Mobile Telephone

bull Consulting System Integrators Outsourcing

bull Utilities

bull Software developers

bull TV and Communications

bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico

Example benefits seen - summary

bull Retain andor gain contracts Customers increasingly want their suppliers to be certified to a specific TMMi level if they want to have the business

bull Releases now meeting scheduled dates

bull Organisation have working processes in place for projects ndashno longer a game of chess consistent approach

bull Management receiving relevant progress information through correct metrics

bull ldquoFerrarirdquo type processes not ldquobicyclerdquo ones

bull Conducted internal assessments over period of time and introduced relevant improvements ndash achieved TMMi levels straight away

7


bull Able to involve product owners and identified risks were handled much better

bull Able to motivate staff

bull Found shortcomings in risk and processes despite having had an industry based audit that did not find problems

bull Improved quality products smoother releases

bull We can help quantify savings through improvement

8

How to improve

bull Start simple ndash do an online checkout first bull httpwwwexperimentuscomcustomer-journeyassess

bull Use the TMMi Framework as a guide

bull Be realistic what target level you want to aim for

bull Have an informal assessment to your target TMMi level ndash use the framework against your organisation and type of business

bull Identify the strengths and any gaps

bull Hold a workshop to discuss the gaps

bull Create a strategy defining what is to be improved and when and how they will be monitored

9

How to improve

bull Get yourself a championowner

bull Communicate what you are doing and why

bull Split improvements to quick wins and longer term goals

bull Have access to a TMMi expert to help you

bull Allocate the right amount of time to do this

bull Gain commitment from individuals and ensure they are open to change

bull Have an improvement plan and target metrics

bull Use an improvement model like IDEAL

COMMUNICATE PROGRESS

10

IDEAL model

11

Definemonitor improvements

12

Priority Item Specific Goals Specific Practices Improvement Opportunity Task to Address Activity Individuals Involved Individual LeaderEstimated Effort

( Days)

Target

Completion DateDependencies

Training Mentoring

RequirementsStatus

Need Additional

ClarificationComments Questions

21 Test Policy amp

Strategy

SG1 Establish a Test

Policy

1SP 11 Define Test Goals Define and maintain test goals based

upon business needs and objectives

2

SP 12 Define Test Policy A test policy aligned with the business

(quality) policy is defined based on the

test goals and agreed upon by the

stakeholders

3

SP 13 Distribute Policy to Stakeholders The test policy and test goals are

presented and explained to stakeholders

inside and outside testing


Strategy

4

SP 21 Perform a generic product risk

assessment

A generic product risk assessment is

performed to identify the typical critical

areas for testing

TMMi Certification Trends

13

IT Service Provider Results

14

0

10

20

30

40

50

60

70

80

90

100

21 Test Policy and Strategy

22 Test Planning

23 Test Monitoring andControl

24 Test Design andSpecification

25 Test Environments

31 Test Organisation

32 Test Training program

33 Test Lifecyle andIntegration

34 Non Functional Testing

35 Peer Reviews

IT Service Provider Sector Results

2015

2014

Results across all industries

15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation

Test Training Programme

Test Lifecycle and Integration

Non Functional Testing

Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2

TMMi Level 2 amp 3 Process Areas

Fully Achieved Partially Achieved Not Achieved

Recognise and embrace the failures

bull The fear of failure and a sense of failure are both emotional responses that need to be managed to bring about success

bull You should think about failure in a different way and create an awareness that failure should be regarded as a positive every time

bull To help you appreciate that not all failures are equal and then explore ways of dealing with and preventing them in future

bull See how failure sits alongside process improvement and how risk fits in all of this

16

Is this right

17

General views on Failure

bull Comparing Aviation to Healthcare

bull Generally a failure to learn from mistakes has been one of the single greatest obstacles to human progress ndash Matthew Syed

bull Studies show we can be so worried about failure that we create vague goals so nobody can point a finger when they are not achieved

bull When we fail we donrsquot get a process improvement because the information about errors problems etc are misinterpreted or ignored This we call a closed loop

bull An open loop does lead to progress because you act on feedback

o If we fear failure we donrsquot even try

o If we donrsquot try we donrsquot take risks

o If we donrsquot take the risk then we donrsquot fail and learn how to improve

18

Amy Edmondson - Harvard Business School

ldquoLearning from failures is anything but straightforward The

attitudes and activities required to effectively detect and

analyse failures are in short supply in most companies and

the need for context-specific learning strategies is

underappreciated Organisations need new and better ways

to go beyond lessons that are superficialrdquo

19

bull Failure has created a bad feeling from a young agebull from childhood we learn that admitting to failure can lead to blame

low marks ridicule etc ndash so weve had 12 years or so of training to fear failure

bull often as we grow up we donrsquot shift to the rewards of learning from failure

bull sometimes we fear failure even before we start - aim low you get low

bull We can fear other peoplersquos view of us and this can lead to a lack of confidence

bull We imagine others care as deeply about our own success and failure as we do personally

bull We fear shame and our responses to this can manifest itself in many ways

So overall failure can and has created negative connotations in many of us

20

Why do we fear failure

How we should really see failure

bull Organisations need to make it a ldquosaferdquo place to admit and report failure

bull with the right mind-set the concern that accepting failures will lead to poor quality is not relevant

bull Google X ndash Astro Teller says

bull ldquoYou must reward people for failing If not they wont take risks and make breakthroughs If you dont reward failure people will hang on to doomed ideas for fear of the consequences That wastes time and can damage an organisations spiritrdquo

bull Harvard Business Review research developed ldquoa Spectrum of Reasons for Failurerdquo

bull Recognise that no-one deliberately wants to fail

bull We learn from failure with the intention that we can then improve and learn

21

Spectrum of failure

22


Opportunity to

bull Learn and grow from the experience

bull Consider and have a healthy approach to risk

bull Forget perfection ndash ldquofail wellrdquo take risks and learn from them Helps you to find the gaps and take actions to fix

bull Be resilient and keep trying ndash eg practise at music to get better ice skating ski jumping etc

bull Recognise that young people donrsquot naturally fear failure so they can learn

bull In a UK school they have a ldquofailure weekrdquo to de-mystify the word Failure happens Then have a ldquoblow your trumpet weekrdquo to recognise and celebrate success and achievements

Be expert learners not just experts

True success is not avoiding failure but learning what to do with it

23

Wimbledon High School

24

FAILURE WEEKA top girls school is planning a failure week to teach pupils to embrace risk build resilience and learn from their mistakes The emphasis will be on the valueof having a go rather than playing it safe and perhaps achieving less Pupils atWimbledon High School will be asked how they feel when they failThe headmistress Heather Hanbury said she wanted to show it is completelyacceptable and completely normal not to succeed at times in lifeldquoMs Hanburys pupils achieve some of the highest exam scores - but from Monday they will be invited to focus on failure There will be workshops assemblies and activitiesfor the girls with parents and tutors joining in with tales of their own failuresThere will be YouTube clips of famous and successful people who have failed along the way and moved onThe emphasis will be discussions on the merits of failure and on the negative side of trying too hard not to failCourage in the classroomMs Hanbury told BBC News that she had placed a great emphasis on developing resilience and robustness among the girls since she arrived at the school fouryears ago The girls need to learn how to fail well - and how to get over it andcope with it she said Fear of failing can be really crippling and stop the girlsdoing things they really want to doThe pupils are hugely successful but can sometimes overreact to failure even though it can sometimes be enormously beneficial to them We want them to be brave - to have courage in the classroom she added Wimbledon High is an independent school part of the Girls Day School TrustGDST chief executive Helen Fraser said Resilience is so important in working life these daysWimbledon High School is showing how making mistakes is not necessarily a bad thing that it is fine to try - and fail - and then pick yourself up and try again -or as Samuel Beckett said fail better

Blow your own trumpet weekrun by Wimbledon High School (motto

Stepping in Striding out) aims to help its over-pressured high-achievingpupils who find it difficult to accept that they have been successful Head Heather Hanbury explains Some teenagers will continue to push themselves never recognising when theyve done enough The school has also run a successful failure week

Key quotes

25

What does success involve

26

Project Retrospectives Can Help

Definition of Retrospective Relating to or thinking about the past

bull Where were the failures ndash any failures

bull Identify what worked well and what needs improvement

bull Led by an independent facilitator

bull Held in a safe place ndash quiet not disturbed

bull Not a witch hunt ndash looking at processes

bull Plan the event ndash issues questions at the start of the project

bull Have the right people involved

bull Record results in open area allocate and monitor actions

bull Use the results in future projects

bull Be honest

27

Retrospectives

Norm Kerthrsquos Prime Directive

Regardless of what we discover we must understandand truly believe that everyone did the best job he orshe could given what was known at the time his orher skills abilities the resources available and thesituation at hand

28

Emotional Histogram

29

Richard Branson ndash His views

Irsquove been failing for as long as I can remember In fact Irsquove

been failing even longer than that ndash I fell over many times as a

baby before learning how to walk The pattern has continued

into adulthood and my life as an entrepreneur and I have

learned and loved every step of the way Wersquore celebrating

failure all month at Virgin and teamed up for a Hangout on

How to Turn Failure into Success

30

Thomas Edison - Quote

31

American Businessman and Inventor Among other things he invented the Electric Light Bulb

Lessons to learn

32

Everyone should aim to

33

TRANSFORM THE FEAR OF FAILURE

TO THE ACTION OF SUCCESS

ldquoI Dare you to Failrdquo

ldquoIt does not mean we should aim at failure rather that creativity isa journey that involves taking wrong turns along the wayrdquo

34

Summary

Looked at

bull The challenges you face

bull Examples of client benefits

bull How to move forward with TMMi

bull TMMi trends

bull Donrsquot fear failure ndash embrace it

35

TMMi 모델을 활용한 Quality Process 개선 사례

HH Cho hhchostacokr

Session 5

Ⅰ The importance of (SW) Quality

Ⅱ Introduction to TMMi

Ⅲ SW Quality Improvement using TMMi

3

Question 1

Are you aware of TMMi using small i

and CMMI using Large I

No Yes

4

Question 2

Do you know why TMMi using small i


5

Question 3

Are you new to SIX-SIGMA(DFSS) LEAN or Lean-Six Sigma

No Yes




7

품질 이슈

Quality can make the difference between excellence and disaster

Toyota Recall(2009)

BPrsquos Oil Spill(2010)

Accident Reason Root Cause Quality Focused

O-ring Flaw

Lack of O-ring Testin a very low temperature

NASA disregarded warning from engineers

Lack of factual based decision making

Sudden unintended acceleration

Lack of quality controlfor the TPS and so on

Growth-oriented policies

Safety of drilling equipment (Blowout Presenver) will not

operate hydraulic leaks and battery failure

Poor test for cement borehole and so on

Lack of systematic approachLack of factual approach to

decision making

ChallengerExplosion(1986)

8

SW 품질과 비즈니스 영향력(1)

lt Forrester의 북미(US Canada) 유럽(the UK France and Germany) IT 종사자 Survey gt

[ Source Software Integrity Risk Report April 2011 Forrester Research ]

다양한

SW 제품

개발

다양한직무

[ Base 336 product development and IT professionals involved with software development ]


There is tight link between SW Quality and Business Impact


ldquoSW 코드 결함으로 가장 큰 영향을 받는 것은 무엇인가rdquo

Customer satisfaction

Product release scheduletime-to-market

Brand image

Company revenues

Product uptime

Donrsquot have an opinion

None of the above

10

비즈니스 목표와 SW Quality 노력

Software Quality(Risks) Have a Direct Impact on Business Success

Software Integrity Risk Report April 2011 Forrester Research

7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5


ldquo품질 개선 활동을 추진하는 주요 이유가 무엇인가 rdquo

분명한비즈니스

목표




왜 테스팅이 개선되어야 하나

Software industry has invested substantial effort to improve the quality of its products but the software industry is still far from zero defects

[ Source The Standish Group Chaos Manifesto 2013 ]

Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012

We can no longer imitatethe three monkeys Hear no failures See no failures Speak no failures

Testing often takes 30 - 40 of project costs

IT Projects are harder than climbing Everest

SW 테스팅을 개발 앞 단계에서부터~~

Early removal of defects contribute to mitigating risks and improving SW quality


8

10

14

10

4

22

27

2

ldquo개발 프로젝트의 성공에 가장 큰 영향을 줄 이슈는 무엇인가rdquo

1

1

프로세스 개선 프로그램

많은 조직들이 품질과 생산성 향상을 위해

전사관점의 프로세스 개선 프로그램을 수립함

(company-wide process improvement program)

이러한 프로그램은 종종 많은 양의 프로세스 문서를 만들고

결국엔 부담이 되어 무시되어 버리는 결과를 초래함

(become a burden or are ignored completely)

In many cases the organization is left

with very little benefit to show for its efforts

린 식스 시그마 Timeline

1900 1930 1950 20001980 1990

Just-in-Time

포드- Assembly line- Flow Concept

Walter ShewhartSPC 개발

도요타 TPS- Flow concept from ford- Pull concept from grocery store

SPC TQM MotorolaIntroducesSix Sigma

Deming SPC 확산14Points 제안

AlliedSignalGE Adapt LSSto BusinessProcesses

CMM(I) SPICE TMM(i) Timeline

1991 1993 20091995 1996

CMM V10 CMM V11 ISO 15504(SPICE)

first draft released

TMM

TMMi V20

(Level 2 amp3) released

2015

ISO 33063 released

개선을 위한 두 가지 Approach

벤치마크 정보 활용 =gt 모델 기반

bull Compare yourself to others who have similar strategies

(but not necessarily similar products)

bull Top-down strategy of borrowing from others to learn something new

bull Decomposes the improvement needs into processes

문제해결을 위한 Flexible way

bull Improve the quality of your processes by eliminating waste and variation

bull Bottom up strategy where improvement begins at the work-team level

bull Used to solve specific problems

TMMi Model ndash Maturity Level

bull Test Policy and strategybull Test Planningbull Test Monitoring and Controlbull Test Design and Executionbull Test Environment

L2 Managed

bull Test Organizationbull Test Training Programbull Test Lifecycle and Integrationbull Non-functional testingbull Peer Reviews

L3 Defined

bull Defect Preventionbull Test Process Optimizationbull Quality Control

L5 Optimization

Level 1 Initial

bull Test Measurementbull Product Quality Evaluationbull Advanced Peer Reviews

L4 Measured

High Maturity

Infrastructure for Process

Improvement

TMMi Maturity Level(45)

Level 4 Measured

Definition

Features

Visibility

Testing is a thoroughly defined well-founded and measurable process

bull Testing is perceived as evaluationbull Measuring Test Performance and costbull Measuring Product Qualitybull Measuring Review Result

Quality and process efficiency are being statisticallymeasured through the whole development lifecycleA test measurement program supports predictionsrelating to test performance and cost

Support fact-based decision

making

IN OUT


Level 5 Optimization

Definition

Features

Visibility

Optimized test process is focused on defect prevention

bull Continuously improvedbull Fine-tuning methods and techniquebull Tool support as much as possiblebull Process reuse

Test Process is optimized and continually improvingits processes

Six Sigma(DFSS) amp Leanis helpful for

CMMI and TMMi Level 5

IN OUT

The Benefits of TMMi

An IT organization that reached TMMi Level 3 as one of the first testing organizations worldwide reported a higher Defect Detection Percentage during the system test

[ Source Erik van Veenendaal and Jan Jaap Cannegieter The Little TMMi UTN Publishers 2011 ]

(DDP)

75

70

65

60

55

50

1st year 2nd tear 3rd year 4th year

DDP(Defect Detection Percentage)


An IT organization at TMMi Level 2

reported better predictability of the

testing process


A Financial organization reached

TMMi Level 2 reported the

improvement of the DDP

()(DDP)

50

40

30

20

10

0

1st year 2nd tear 3rd year

Defect Detection Percentage of the system testDeviation of test time spent versus estimated test time (percentage)




Case Study of Defense Industry

Company L Company H

bull CMMI L5 bull SIX SIGMA(DFSS) LEAN SIX SIGMA

bull 경영진의 품질 및 테스트 인식 수준이 높음

bull Continuous improvement activities

bull Perform a step-by-step risk analysis(from generic risk analysis)

Strong Points


bull 경영진의 품질 및 테스트 인식 수준이높음


bull Well known about process (Recently enhanced dev quality control gate)

bull Some members do not well known about test processes

bull Lack of risk tracking bull Lack of test log managing

Weak Points

bull Lack of dedicated test organizationbull Lack of professional test expertisebull Lack of risk tracking


Similarities and differences of both companies

Company L Company H

CMMI Level 5 Development CMMI Level 5

TMMi Level 3 Testing Level TMMi Level 3

Common risk analysis based on quality attributes

Risk Based Testing Requirements based risk analysis

Define new procedures to derive Test conditions

Test conditionApply to the existing test Design

documents

Newly create new Master Test Plan(MTP)

Test PlanUpdated Original

Master Test Plan(STP)

V-Model-based testing activitiesIntegration with Dev

lifecycleV-Model-based testing activities

CMMI Process Areas that Support a TMMi Implementation

CMMi Level Process Area

Level 5 (Optimizing)

Org Innovation and Deployment

Causal analysis and Resolution

Level 4(Quan Managed)

Organizational Process Performance

Quantitative Project Management

Level 3 (Defined)

Requirements Development

Technical Solution

Product Integration

Verification

Organizational Process Focus

Organizational Process Definition

Organizational Training

Integrated Project Management

Integrated Supplier Management

Risk Management

Decision Analysis and Resolution

Level 2 (Managed)

Requirements Management

Project Planning

Project Control and Monitor

Supplier Agreement Management

Measurement and Analysis

Process and Product Quality Assurance

Configuration Management

Process Area TMMi Level

Defect Prevention

Level 5 (Optimization)


Quality Control

Test Measurement

Level 4 (Measured)

Product Quality Evaluation


Test Organization

Level 3 (Defined)



Non-functional testing

Peer Reviews

Test Policy and strategy

Level 2 (Managed)

Test Planning



Test Environment

L 사 사례 ndash WHY

L 사가 TMMi 기반 테스트 프로세스 개선을 추진한 이유

Everyone else is doing it An executive heard about it or read about it somewhereA customer demands itIf they donrsquot do something they wonrsquot survive

In order to maintain a leader amp

have a good position in bidding situation

(국방산업 분야 최초 인증)

L 사 사례 ndash 프로세스 개선 활동

TMMi based Test Process Improvement March 2015 LIG Nex1

RampDProcess HW based

ProcessSW based Process

HWSW Integration Process

Measurement and Continuous Improvement


Defense embedded SW guideline

SW related contents update

MRA update

MRA Manufacturing Readiness Assessment CMMI Capability Maturity Model Integration TMMi Test Maturity Model integration

RampD 품질경영시스템

국방 정책

SW Process Update

L 사 사례 - SW 테스트 프로세스 구현


Establish Master Test Plan

Establish Test Policy and Strategy Strengthen Test Design Activities

Organizational Test Policy

rarr Product Risk based test strategy

Establish Test Performance Indicators

Perform integrated test planning activities

rarr Integrate Level test plan

Establish product risk analysis activities

and testing strategy

Clarification of the testing exit criteria

Adopt formal test techniques

Make test design specification templates

Identify early test item and design

for the dynamic test

Try to separate test organization

rarr SW Test and Whole System Test

Develop test Job description amp Career Path

Develop test training Program

Develop Test organizationand training program

L 사 사례 - Testing Activities


bull Static Testing tool

- QAC++ CodeSonar


- QAC

bull Adopt SW Incident management tool

- QualityCenter

bull introducing dynamic testing tools

- VectorCAST LDRA

bull SW Resource Management tool

- SharePoint

bull SW Dynamic Test tool

- DT10

bull SW Static test tool

- CodeSonar(USB)

bull SW Test management tool

- Jama

bull SW Build Automation tool

- Jenkins


- QAC++(USB) QAV

SW Coding guideline

Static Testing Quality

Management

SW ReliabilityTest

TMMi based Process

Implementation

TMMi based Process Institutionalization

bull SW Coding guideline

bull SW Testing Pilot project

bull Extend SW Testing Scope

bull SW Reliability Test(Organizational)

bull Build SW testing process

bull TMMi Level 3

bull TMMiinstitutionalization

bull Independent testing

bull Setup Partnerrsquos SW Process

bull Educate Partnerrsquos SW reliability test

bull Perform Partnerrsquos SW reliability test

bull Develop SW Quality Training Program and Train(3 classes)

bull Develop SW Quality Training Program and Train( 6 classes)

Quality Management Team

Development Team Leads the Activities and Quality Team is supportive

Suppliers are also applied

TMMi 적용 효과 - L 사

SW 개발 수명주기 앞부분에서 결함 검출율 향상


- The defect density of the coding and unit testing and SW integration testing phase increased significantly

- The defect density of System integration testing is reduced

Applied TMMi based Processes7 Projects

Not applied TMMi Processes33 Projects

of Defects KLOC


Organizational Pride

ldquoThe successes weve had in process improvement have been a source of great pride for our quality assurance teamrdquo

사내에 품질 경영을 위한 기본적인 품질관리 프로세스가CMMi 등으로 정착이 되어 있는 상태에서

테스트를 위한 프로세스를 보완해품질관리의 성과가 더욱 빛나도록 함

H 사 사례 ndash WHY

Why did Company L apply Process Improvement using TMMi

Everyone else is doing it An executive heard about it or read about it somewhereA customer demands itIf we donrsquot do something we wonrsquot survive

A compelling reason for changeAt the VOC meeting

The CEO announced to initiate TMMi

Applied

H 사 사례 - Process Improvement








TMMi based Process update

DFSS Design for Six Sigma

Defense Industry Policy(Quality Management System)

SW Process Update

H 사 사례 - SW 테스트 프로세스 구현

Establish Master Test PlanDevelop Test organization

and training program




Reused Test Performance Indicators

Update original Master test plan(STP)rarr Integrate Level test plan








SE WPL and SW WPL have Responsible

for Testing and Development work



TMMi 를 효과적으로 적용하기

To effectively apply TMMi implies reflecting on the realities of your business environment and applying the model in a thoughtful appropriate manner

bull Identifying problems as objectively as possible and tieprocess improvement to your business goals

bull Tailoring (interpreting) TMMi to fit your context and needs

bull Not forcing inappropriate solutions simply because a practice is identified

in the TMMi model

Use TMMi as a Guide not as a Rule Book

TMMi 기반 프로세스 개선의 핵심요소

A compelling reason for change

Leadership of the change effort by the top executive

Informed commitment of the top management team

Designation of a primary change agent and an adequate mandate for change

Sound performance measures that drive change

Without these itrsquos very likely that you will experience

a ldquofalse startrdquo of your process improvement program

테스트 프로세스 개선의 Manifesto

Detailed Process

Process oriented

Templates

Quality assurance

Model driven

IP3 Deployment oriented

IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions

1 TMMi can be a good tool for test process improvement

2 TMMi has similar structure with CMMI so a company using CMMi can

easily apply TMMi practices

3 Needs a compelling reason for change

Just trying to get to TMMi Leve3 without understanding the

business context will be fail in the short or long term

4 The improvement team must understand the business problem

5 As TMMi is a relatively recent model the number of publications

about benefits in practice is limited However applying the TMMi maturity

will improve test process and have a positive impact on product quality

TMMi based improvement case study of

software reliability testing process at LIG Nex1

LIG Nex1 김명호 myonghokim91gmailcom

LIG Nex1 신영섭 busigeenavercom

Session 6

1 회사 소개

2 LIG Nex1 테스트 프로세스 개선 현황

3 소프트웨어 신뢰성 시험 개요

4 TMMi 기반의 테스트 Process 구축

5 소프트웨어 신뢰성 시험 프로세스 개선

6 결론 및 향후 계획

1 회사 소개

3

LIG넥스원은 정밀유도무기 등 방위산업

전 분야에 걸친 최첨단 무기체계를

연구개발생산 기업임

1 회사 소개

4

5

2 테스트 프로세스 개선 현황

국방 정책 및 Global 모델(CMMI TMMi 등)에 따라 지속적인 프로세스 개선활동을 수행하고 있음

6


rsquo07년 부터 SW 신뢰성을 향상을 위한 프로세스 인프라 교육 등의 활동을 수행함


7

무기체게 소프트웨어는 ldquo무기체계 소프트웨어 개발 및 관리 매뉴얼rdquo을 기준으로시험을 수행함

bull ldquo소프트웨어 신뢰성 확보 활동rdquo이란 소프트웨어의 잠재적 결함을 최소화시키기 위한

제반 활동(신뢰성 확보계획 수립 검증 및 확인 신뢰성 시험 등) 을 말한다

bull ldquo소프트웨어 신뢰성 시험rdquo 이라 함은 소프트웨어 코드가 일으킬 수 있는 결함을 사전에

식별하여 제거하기 위한 시험을 말하며 정적 시험 및 동적 시험으로 구분한다

1) ldquo소프트웨어 정적(Static) 시험rdquo이란 소프트웨어를 실행하지 않은 상태에서

잠재적인 결함을 검출하는 시험을 말하며 코딩 규칙(Coding Rule) 검증 취약점

점검 및 소스코드 메트릭 점검으로 구성된다

2) ldquo소프트웨어 동적(Dynamic) 시험rdquo이란 소프트웨어의 요구사항 및 구조를

기반으로 소프트웨어를 실행하면서 기능의 정상 동작 여부를 확인하는 시험을

말한다


8

소프트웨어 신뢰성 시험의 시험 기준은 지속적으로 높아지고 있음

구분 rsquo11년 rsquo14 년 rsquo16 년

코딩규칙

bull 방위사업청 코딩규칙 적용- 공통 규칙 45개- C 언어 전용 5개- C++ 언어 전용 10개


bull 국제 표준을 사업별 조정 및 합의- C 언어 MISRA C (143개 규칙)- C ++ 언어 MISRA C++ (228개 규칙)- C 언어 Microsoft (62개 규칙)- JAVA Oracle

bull 방위사업청 코딩규칙은 유지

취약점bull 실행시간 오류검출 목록

CWE-658659 bull 사업별로 선별 적용

bull 실행시간 오류검출 목록 CWE-658659

bull 사업별로 선별 적용

bull 실행시간 오류검출 목록 CWE-658659660bull CWE 목록 중 도구가 지원하지 않는 항목에 대

해서는 대안과 사유를 제출하고 협의하여 결정

소스코드

메트릭bull 주석률 30 이상 bull 주석률 30 이상

bull 주석률은 코딩규칙으로 이동

bull Cyclomatic Complexity 20 이하bull Number of Call Levels 6 이하bull Number of Function Parameters 8 이하bull Number of Calling Functions 8 이하bull Number of Called Functions 10 이하bull Number of Executable Code Lines 200 이

하

bull 정적 시험 분야


9

rsquo11년 rsquo14 년 rsquo16 년

bull 임무중요도 기능안정성 및 통제능력 사용빈도에 따라 목표값 설정 후구조기반 동적 시험수준 결정

bull NA Statement Branch MCDC

bull 국제기준 및 규격(MIL-STD-882E DO-178B IEC 61508 ISE 26262 등)을 참고하여 시험대상 결정 후위험수준별 구조기반동적 시험 수준 결정


bull 결함 발생빈도 영향성 및 제어가능성에 따라 수준별요구사항기반 동적 시험 수행

bull Statement Branch MCDC ( NA 없음)bull 단 국제표준(MIL-STD-882E DO-178 IEC 61508 ISE

26262 등) 을 적용하여 개발한 경우는 해당 표준을 따를 수 있다

bull 동적 시험 분야

3 소프트웨어 신뢰성 시험

10

bull Open Source Software


- -

bull 소프트웨어 신뢰성 시험 대상

bull 상세 설계 단계에서 사용필요성 및 신뢰성 확보 제시하여 승인 시 제외 가능

bull 오픈 소스 사용 시는 라이선스 조항을 반드시 확인하여야 하며 소스코드 공개 의무가 있는 라이선스는 사용이 불가함

4 TMMi 기반의 테스트 프로세스 구축

11

Optimization1 결함 예방2 테스트 프로세스 최적화3 품질제어

Measured1 테스트 측정2 제품 품질 평가3 발전된 동료 검토

Defined1 테스트 조직2 테스트 훈련 프로그램3 테스트 수명주기amp통합4 비기능 테스팅5 동료 검토

Managed1 테스트 정책amp전략2 테스트 계획3 테스트 모니터링amp제어4 테스트 설계amp실행5 테스트 환경

Level 5

Level 4

Level 3

Level 2

Initial

Level 1

TMMi 공식 인증 시작(2010년~)

bull 조직 차원 4개 영역 테스트 정책 및 전략 시험조직 테스트 훈련 프로그램 테스트 수명주기 amp 통합

bull 프로젝트 차원 6개 영역 테스트 계획 테스트 모니터링amp제어 테스트 설계amp실행 테스트 환경 비기능 테스팅 동료

검토

LIG넥스원은 Level 2 Process Area 5개 Level 3 Process Area 5개 총 10개영역을 적용함


12

LIG넥스원에 구축된 SW 테스트 프로세스를 TMMi 영역 별로 도식화하면다음과 같은 연관 관계가 있음

개발과 시험 수명주기 통합 모델

시험 정책 및 전략

시험 조직

시험 훈련 프로그램

시험 계획

시험 환경

시험 설계 및 수행

비기능 시험

시험 모니터링 동료 검토

PA21 Test Policy and Strategy

PA33 Test Lifecycle and Integration

PA23 Test Monitoring and Control PA35 Peer Reviews

PA31 Test Organization PA22 Test Planning

PA32 Test Training Program

PA25 Test Environment

PA24 Test Design amp Execution

PA34 Non-Functional Testing

조직차원 정의

프로젝트 차원 정의


13

TMMi Level 2~3의 10개 Process Area는 사내 표준 프로세스에 포함되어 있음

14

소프트웨어 시험 프로세스는 방위사업청 규정 및 MIL-STD 기반으로 CMMI 레벨5및 TMMi 레벨 3 요건을 충족시키는 총 33종의 표준 프로세스가 정의되어 있음


15

기본 방침은 V 모델 기반으로 개발 초기부터 신뢰성 시험활동을 수행토록 함


16


배경 소프트웨어 신뢰성 시험 기준 변경

추진 방법

Step 1 신뢰성 시험의 주요 변경사항 분석 및 이슈 도출

Step 2 도출된 이슈를 Process People Product 및 Environment 관점에서 개선 과제 도출 및 우선순위를 고려한 개선 계획 수립

Step 3 개선 과제 중 신뢰성 시험에 대한 As-Is 프로세스를 정의 및문제점 도출

Step 4 개정된 규정에 의해 추가되어야 할 활동을 정리

Step 5 3단계와 4단계를 통해 개선 및 추가되어야 할 활동을 통합하여To-Be 프로세스를 구축

17



총 16개 주요 변경 사항 및 이슈

- 공통 분야 8개

- 정적 시험 분야 4개

- 동적 시험 분야 4개

18



개별 사업별 해결 3개

과제 우선 순위

- A 즉시 실행

- B rsquo17년 실행 과제

19



Process Modeling BPMN 20

개발 단계별 Process 정의

20



21


To-Be Process 특징

첫째 신뢰성 시험을 시험 항목별로 연관성을 분석하여 시험 프로세스를

표준화하고 시험 항목별 목표치를 설정하여 재작업을 최소화

둘째 개발 초기부터 전문 부서의 검토 단계를 추가하여 시험 단계에 발생

할 수 있는 문제를 사전에 최소화할 수 있도록 하였다

셋째 신뢰성 시험 전문 조직의 역할을 늘리고 개발 조직과의 협업

프로세스를 구축하여 시험 생산성을 향상하였다

향후 계획

프로세스 개선 효과 측정 테스트 생산성 결함 검출률 등

L4 amp L5 기반 프로세스 개선 동료 검토 생산성 향상 정적 시험 목표치 설정 등

대한민국 SW 품질을 책임집니다

대한민국 SW 가치 증진을 선도하고 산업을 리딩하는SW테스팅 전문기업 네트워크

회원사 명단

회장사

회원 가입 문의

와이즈와이어즈

임원사 MDS테크놀로지 STA테스팅컨설팅 어니컴 와이즈스톤 지티지 파수닷컴

회원사 가온테스팅 두루이디에스 링크제니시스 솔루션링크 슈어소프트테크

씨엔큐소프트 아이오에이솔루션 아이지오 아주큐엠에스

에스이앤티소프트 에이취큐텍 엠스텍 지텍에스 큐머드

한국전력공사 전력연구원 NIPA SW공학센터

컴퓨터월드ITDAILY

한국SW테스팅협의회에서는 관련 전문 기업 및

조직의 참여를 기다리고 있습니다

(자문) 숭실대학교 SW학부 (박진호 교수)

한국SW산업협회 배범진 책임

연락처 bbj99sworkr




Klaus Olsen
















Crosby1979

CMM v10 1991

TMM1996

TPI 1997

CMM v111993

TMMDK version

2000

CMMi2000

TMMi reg

2005

TMMi reg

full version

2010




Chairman

9


Vice-chair

Geoff Thompson

Treasure

Secretary CEO

Klaus Olsen



10





CEO

11

Marc Gebauer

Erik van Veenendaal

Deputy CEO

Meile Posthuma

Accreditation


Secretary Marketing

Geoff Thompson

TMMi Professional

Training


TMMi Foundation is











12


Level 1 Initial



Quality Control





Peer Reviews





Test Planning



Test Environment


16 Process Areas


Maturity Levels

Process Areas




Achieves Goals

contains

contains



Maturity Level



SpecificPractices

GenericPractices


Managed



Develop a test plan



Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22










18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사


























Crosby1979

CMM v10 1991

TMM1996

TPI 1997

CMM v111993

TMMDK version

2000

CMMi2000

TMMi reg

2005

TMMi reg

full version

2010




Chairman

9


Vice-chair

Geoff Thompson

Treasure

Secretary CEO

Klaus Olsen



10





CEO

11

Marc Gebauer

Erik van Veenendaal

Deputy CEO

Meile Posthuma

Accreditation


Secretary Marketing

Geoff Thompson

TMMi Professional

Training


TMMi Foundation is











12


Level 1 Initial



Quality Control





Peer Reviews





Test Planning



Test Environment


16 Process Areas


Maturity Levels

Process Areas




Achieves Goals

contains

contains



Maturity Level



SpecificPractices

GenericPractices


Managed



Develop a test plan



Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22










18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















Chairman

9


Vice-chair

Geoff Thompson

Treasure

Secretary CEO

Klaus Olsen



10





CEO

11

Marc Gebauer

Erik van Veenendaal

Deputy CEO

Meile Posthuma

Accreditation


Secretary Marketing

Geoff Thompson

TMMi Professional

Training


TMMi Foundation is











12


Level 1 Initial



Quality Control





Peer Reviews





Test Planning



Test Environment


16 Process Areas


Maturity Levels

Process Areas




Achieves Goals

contains

contains



Maturity Level



SpecificPractices

GenericPractices


Managed



Develop a test plan



Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22










18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















10





CEO

11

Marc Gebauer

Erik van Veenendaal

Deputy CEO

Meile Posthuma

Accreditation


Secretary Marketing

Geoff Thompson

TMMi Professional

Training


TMMi Foundation is











12


Level 1 Initial



Quality Control





Peer Reviews





Test Planning



Test Environment


16 Process Areas


Maturity Levels

Process Areas




Achieves Goals

contains

contains



Maturity Level



SpecificPractices

GenericPractices


Managed



Develop a test plan



Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22










18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















CEO

11

Marc Gebauer

Erik van Veenendaal

Deputy CEO

Meile Posthuma

Accreditation


Secretary Marketing

Geoff Thompson

TMMi Professional

Training


TMMi Foundation is











12


Level 1 Initial



Quality Control





Peer Reviews





Test Planning



Test Environment


16 Process Areas


Maturity Levels

Process Areas




Achieves Goals

contains

contains



Maturity Level



SpecificPractices

GenericPractices


Managed



Develop a test plan



Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22










18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














TMMi Foundation is











12


Level 1 Initial



Quality Control





Peer Reviews





Test Planning



Test Environment


16 Process Areas


Maturity Levels

Process Areas




Achieves Goals

contains

contains



Maturity Level



SpecificPractices

GenericPractices


Managed



Develop a test plan



Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22










18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















Level 1 Initial



Quality Control





Peer Reviews





Test Planning



Test Environment


16 Process Areas


Maturity Levels

Process Areas




Achieves Goals

contains

contains



Maturity Level



SpecificPractices

GenericPractices


Managed



Develop a test plan



Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22










18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















Maturity Levels

Process Areas




Achieves Goals

contains

contains



Maturity Level



SpecificPractices

GenericPractices


Managed



Develop a test plan



Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22










18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















Maturity Level



SpecificPractices

GenericPractices


Managed



Develop a test plan



Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22










18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















Managed



Develop a test plan



Plan theprocess

TMMi level 2

SpecificGoal 4

SpecificPractice 45

GenericGoal 2

GenericPractice 22










18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사























18

















Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사






























Plan

Do

Study

Act



an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사
















an assessment


small size projects




Project Manger

Test Manger

Developer

Tester


interview




Test strategy

Test Plan

Test case examples

Test reports

And more


interviews


all process areas


artefacts


report with result




must repeat




company


next








s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사




















s after 90 days




96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














96 DDP reached















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사






















TMMi our future

TMMi and Agile




28



30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사
















30




Spain UK UK UK USA


Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















Session 4




4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사

















4 TMMi trends



3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















3












Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사

























Therefore


4








5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사





















5



bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사
















bull Mining company

bull Pharmaceutical

bull Banking


bull Retail

bull Insurance



bull Utilities



bull Etchellip

6

bull UK

bull India

bull USA

bull Malaysia

bull Australia

bull China

bull Ukraine

bull Spain

bull France

bull Mexico








7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사





















7







8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사




















8

How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














How to improve








9

How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














How to improve










10

IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














IDEAL model

11


12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















12


( Days)

Target


Training Mentoring

RequirementsStatus

Need Additional


21 Test Policy amp

Strategy


Policy



2




stakeholders

3





Strategy

4


assessment



areas for testing


13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















13


14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















14

0

10

20

30

40

50

60

70

80

90

100


22 Test Planning








35 Peer Reviews


2015

2014


15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















15

54

62

56

40

45

51

45

49

69

60

32

27

34

45

43

36

43

42

25

33

14

12

10

15

12

13

12

9

6

7

0 10 20 30 40 50 60 70 80 90 100

Test Organisation




Peer Reviews


Test Planning



Test Environment

Leve

l 3Le

vel 2








16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사



















16

Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














Is this right

17










18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사























18








19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사





















19









20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사






















20










21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사






















21

Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














Spectrum of failure

22


Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















Opportunity to









23


24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















24




Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














Key quotes

25


26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















26












bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사

























bull Be honest

27

Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














Retrospectives



28

Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














Emotional Histogram

29









30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사






















30


31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















31


Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














Lessons to learn

32


33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















33





34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사
















34

Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














Summary

Looked at




bull TMMi trends


35


HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















HH Cho hhchostacokr

Session 5




3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사

















3

Question 1



No Yes

4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














4

Question 2



5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














5

Question 3


No Yes




7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사

















7

품질 이슈


Toyota Recall(2009)



O-ring Flaw











decision making


8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














8




다양한

SW 제품

개발

다양한직무








Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사




















Brand image

Company revenues

Product uptime


None of the above

10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














10




7

7

11

13

7

rsquo

rsquo

3

2

3

4

3

15

19

2

5




목표







Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사




















Challenged

Succeed

Failed

60

50

40

30

20

10

0

2004 2006 2008 2010 2012







8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사

















8

10

14

10

4

22

27

2


1

1











1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사
























1900 1930 1950 20001980 1990

Just-in-Time








1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















1991 1993 20091995 1996



TMM

TMMi V20


2015

ISO 33063 released













L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사


























L2 Managed


L3 Defined


L5 Optimization

Level 1 Initial


L4 Measured

High Maturity


Improvement


Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















Level 4 Measured

Definition

Features

Visibility





making

IN OUT



Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사
















Definition

Features

Visibility






IN OUT




(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사

















(DDP)

75

70

65

60

55

50






testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사

















testing process





()(DDP)

50

40

30

20

10

0







Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사


















Company L Company H





Strong Points







Weak Points




Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사
















Company L Company H







documents














Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사






















Level 3 (Defined)


Technical Solution

Product Integration

Verification






Risk Management


Level 2 (Managed)


Project Planning







Defect Prevention



Quality Control

Test Measurement

Level 4 (Measured)



Test Organization

Level 3 (Defined)




Peer Reviews


Level 2 (Managed)

Test Planning



Test Environment
















MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사





























MRA update



국방 정책

SW Process Update

























- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사






































- QAC++ CodeSonar


- QAC


- QualityCenter


- VectorCAST LDRA


- SharePoint


- DT10


- CodeSonar(USB)


- Jama


- Jenkins


- QAC++(USB) QAV

SW Coding guideline


Management

SW ReliabilityTest

TMMi based Process

Implementation







bull TMMi Level 3


















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사





















of Defects KLOC











Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사
























Applied












SW Process Update

























in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사






































in the TMMi model











Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사























Detailed Process

Process oriented

Templates

Quality assurance

Model driven


IP2 Best Practices

IP4 Peer reviews

IP1 Flexibiltiy

IP5 Business driven


Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














Conclusions















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사


















Session 6

1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














1 회사 소개






1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














1 회사 소개

3




1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














1 회사 소개

4

5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














5



6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














6




7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















7











말한다


8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















8



코딩규칙











소스코드




하



9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















9











10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















10



- -





11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















11





Level 5

Level 4

Level 3

Level 2

Initial

Level 1




검토



12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















12




시험 조직


시험 계획

시험 환경


비기능 시험










조직차원 정의



13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사















13


14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














14



15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














15



16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














16



추진 방법






17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














17







18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














18





- A 즉시 실행


19





20



21









향후 계획





회원사 명단

회장사














19





20



21









향후 계획





회원사 명단

회장사














20



21









향후 계획





회원사 명단

회장사














21









향후 계획





회원사 명단

회장사
















회원사 명단

회장사














tmmi history, benefit and future - image.sten.or.krimage.sten.or.kr/document/2016/tmmi_track.pdf ·...

Documents