7/31/2007se 652 - 2007_7_31_cmmi_software_quality.ppt 1 standards & assessments cmmi, iso 9000,...
Post on 21-Dec-2015
219 Views
Preview:
TRANSCRIPT
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
1
Standards & AssessmentsCMMI, ISO 9000, TL9000
Sources: ASQ CSQE Primer
Introduction to CMMI
CMMI Distilled
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
2
August 4 Class
CMMI Introduction & Configuration Management Appraisal
ISO 9000/TL-9000
Due today (31-July): Cycle 2 Design & Code, hand off to System Tester
System Test Plan Inspected & Baselined
Project notebook updates including inspection records, meeting minutes, etc.
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
3
Topics
Audits & Assessments
CMM / CMMI & SCAMPI
ISO 9000: ISO 9001:2000, ISO 9000-3:1997, TickIT
Q9000, TL9000
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
4
Capability Maturity Model (CMM)
Created in 1987 by Software Engineering Institute (SEI)
5 level model based on proficiency in Key Process Areas (KPAs)
Migrating to Capability Maturity Model Integration (CMMI)Three source models:
– CMM for Software
– Systems Engineering Capability model
– Integrated Product Development CMM
CMMI v1.1
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
6
Why Would I want one?
Required– Contractual
– Senior Management Decree (e.g. ROI of 7 to 1)
Sales Tool
Want to improve
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
7
Schedule Example
Organization 1Project Schedule Performance
0
1
2
3
4
1 2 3 4 5 7
# Pro
jects
Drop Page Fields Here
Count of Months Late
Months Late
Drop Series Fields Here
Organization 2Project Schedule Performance
0
1
2
3
4
5
4 5 6
# Pro
jects
Drop Page Fields Here
Count of Months Late
Months Late
Drop Series Fields Here
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
8
Process Capability
Ability of a process to produce planned results
• Predictable
• Measureable
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
9
Process Models
CMMI is model basedModel = structured collection of elements that describes characteristics of
effective processesProcess Area = cluster of related practices that when performed collectively,
satisfy a set of goals considered important for making significant improvement in that area
Processes selected are those proven by experience to be effective (i.e. best practices, practical knowledge from previous endeavors)
Notes: A process area is not a processA model is not a process
models show what to do, not how to do it!
Philosophy“All models are wrong, some are useful” – George Box
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
10
CMMI Models
Model Options:Software Engineering (SW)
Systems Engineering + Software Engineering (SE/SW)
Systems Engineering + Software Engineering + Integrated Process & Product Development (SE/SW/IPPD)
… + Supplier Sourcing (SE/SW/IPPD/SS)
Representation Options:Staged (Maturity Levels)
Migration from CMM to CMMI
Continuous (Capability Levels)Migration from EIA/IS-731 to CMMI
Recommended order for process improvements, but not prescribed …
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
11
Levels
Zero – Ad Hoc
One – Doing it (in Continuous, Ad Hoc in Staged)
Two – Process performed for individual projects
Three – Process focus at organizational level
Four – Projects and processes are quantitatively managed
Five – Projects and processes being optimized based on performance data & results
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
12
Representations Revisited
Continuous Model– 25 Process Areas each assessed at level 0-5
Configuration Mgmt = capability level 3
Risk Mgmt = not done (capability level 0)
Requirements Mgmt = capability level 2
– Result can be presented as a Kiviat chart
Staged Model– 25 Process Areas assigned to each of 4 Maturity Levels (see next slide)
– Result is a grade (1-5)
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
13
Staged RepresentationMaturity Levels (MLx)
1Initial
Process unpredictable, poorly controlled & reactive
2Managed
Process characterized for project & often reactive
3Defined
Process characterized by organization is proactive
4Quantitatively Managed
Process measured & controlled
5Optimizing
Focus on process improvement
Proj
ect
Org
aniz
atio
n
Project
Organization
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
15
Staged RepresentationProcess Area Mapping to Maturity Levels
5. Optimizing Continuous Process Improvement
Organizational Innovation & DeploymentCausal Analysis & Resolution
4. Quantitatively Managed
Quantitative Management
Organizational Process PerformanceQuantitative Project Management
3. Defined Process Standardization
Requirements DevelopmentTechnical SolutionProduct Integration
VerificationValidation
Organizational Process FocusOrganizational Process Definition
Organizational TrainingRisk Management
Decision Analysis & Resolution
2. Managed Basic Project Management
Requirements ManagementProject Planning
Project Monitoring & ControlSupplier Agreement Management
Measurement & AnalysisProcess & Product Quality Assurance
Configuration Management
1. Initial None
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
16
Continuous RepresentationProcess Areas
Process Management– Organizational Process Focus (OPF-3)– Organizational Process Definition (OPD-3)– Organizational Training (OT-3)– Organizational Process Performance (OPP-4)– Organizational Innovation & Deployment
(OID-5)Project Management
– Project Planning (PP-2)– Project Monitoring & Control (PMC-2)– Supplier Agreement Management (SAM-2)– Integrated Project Management (IPM-3)– Risk Management (RSKM-3)– Integrated Teaming (IT-3)– Integrated Supplier Management (ISM-3)– Quantitative Project Management (QPM-4)
Engineering– Requirements Management (REQM-2)– Requirements Development (RD-3)– Technical Solution (TS-3)– Product Integration (PI-3)– Verification (VER-3)– Validation (VAL-3)
Support– Configuration Management (CM-2)– Process & Product Quality Assurance
(PPQA – 2)– Measurement and Analysis (MA-2)– Decision Analysis and Resolution (DAR-3)– Organizational Environment for Integration
(OEI-3)– Causal Analysis and Resolution (CAR-5)
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
17
CMMI Assessment Cheat Sheet
Institutionalization – Ingrained Way of Doing Business that an organization follows routinely as part of its corporate culture
Specific Goals – Required model component that describes the unique characteristics that must be present to satisfy the process area
Specific Practice – Expected model component that is considered important to achieving the associated specific goal. The specific practices describe the activities expected to result in achievement of the specific goals of a process area. (In continuous representation – every specific practice (SP) is associated with a CL, in staged – all SPs are treated equally)
Generic Goal – Required model component that describes the characteristics that must be present to satisfy the institutionalization of the processes that implement a process area
Generic Practice – Expected model component that is considered important in achieving the associated generic goal. The generic practices describe the activities that are expected to result in achievement of the generic goal and contribute to the institutionalization of the processes associated with a process area.
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
18
CMMI Assessment Cheat Sheet (continued)
Managed Process:– Performed process planned & executed in accordance with policy– Employs skilled people– Adequate resources– Produces controlled outputs– Involves relevant stake holders– Monitored, controlled & reviewed– Evaluated for adherence to process description
Defined Process:– Managed process tailored from the organizational standard processes– Maintained process description– Contributes work products, measures & other process info to organizational process assets
Performed Process– Accomplishes needed work to produce work products– Specific goals of the process area are satisfied
Establish & Maintain– Includes documentation & usage:
• Planned• Documented &• Used
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
20
DeMarco & Lister on Process
Organizations driving to be SEI Level 5 (at least level N+1)
Standards are good, but …
Most success centered around standard interfaces
Mandating a “best practice” is a bad practice
Process improvement is good, but process improvement programs aren’t
Competent people improve processes all the time (pride, growth, etc.)
Formal process improvement moves responsibility from the individual to the organization
Process improvement programs focus on process rather than product(making a poor product efficiently is often worse than making a good product poorly)
Focus on process “level” tends to make organizations risk averse
“The projects most worth doing are the ones that will move you down one full level on your process scale!”
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
22
Quality Standard Rationale
Customers want & need assessments of supplier quality
Means:Individually audit (i.e. qualify) vendor:
Specific products
Processes (e.g. manufacturing, design & development, support)
Alternative:Common Quality Assurance standards & audits
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
23
Major Audit Types
First Party Audit
Within own company (aka internal audit)
Used to measure own performance, strengths & weaknesses against internally established procedures & systems
Second Party Audit
Performed by customer on their supplier (aka external audit)
Third Party Audit
Outside, independent auditor contracted to audit on behalf of company or a supplier (e.g. ISO 9000 registration audit)
Assessments (e.g. SCAMPI)
Similar to first party audit, but typically performed by external assessors
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
24
Other Audit Types
System Audit – examination of bigger picture of organization &/or project
Typical cross organizational, cross process & cross product
Process Audit – verify inputs, actions & outputs in accordance with defined requirements (e.g. software inspections)
Product Audit – final product or service for “fitness for use”
Customer oriented
Compliance Audit
Regulatory – audit to government regulations
Management – audit to organizational rules, effectiveness & conformance
Quality – systematic & independent of quality activities vs. established procedures
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
25
ISO 9001:2000
Objective
Provide confidence that vendor can produce quality products
Assumptions: good practices will produce good products
Standard for assessing organization’s Quality Management System (QMS)– Processes
– Activities
– Behaviors
– Training
But, ISO focuses on Quality Assurance not Quality Control
ISO-9001 certification does not guarantee quality products!
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
26
Tenants of ISO 9001
1) Say what you do
2) Do what you say
3) Prove it!
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
27
ISO 9000 Audits
Customers write requirements for current ISO-9001 certification into purchasing contracts
Organizations apply for 3rd party audit,end result is ISO-9001 certification
ISO International Accreditation Forum (IAF) board
Audits national accreditation boards (i.e. one board each nation)
Who register individual registrars (e.g. Lloyd’s, DNV)
Who audit organization internal auditors (e.g. Lucent Optical Networking) & spot check
Who audit design, development, manufacturing & support teams within the organization
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
28
ISO Alphabet Soup
ISO 9000:2000
Overall framework, fundamentals of quality management systems & terminology
ISO 9001:2000
Requirements for quality management systems (qms) & what is required to demonstrate compliance
ISO 90003 2004 (previously 9000-3)
Guidelines for the application of ISO 9001:2000 to computer software
ISO 19011
Guidelines for auditing quality and environmental management systems
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
29
What is wrong with ISO 9001?
Vendors ISO-9001 certified, but quality still elusive!No visibility into supplier quality levels
Not getting quality levels they wanted
Solution:TL9000 (Quest forum, telecommunications)
QS9000 (automotive)
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
30
TL9000
ISO on steroids
Wholly subsumes ISO 9001-2000
Requires vendors prove they are actually improving
Metrics focused on cost drivers of service providers:Know vendor is measuring
Visibility into quality improvement results
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
31
TL9000 Top Management Requirements
Monitor & improve customer satisfaction
Set long & short term objectives for organization effectiveness
Set targets for TL9000 product performance metrics
Use an explicit life-cycle model
Establish a quality improvement program
Periodic management review of quality system
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
32
TL9000 Metrics
Cross-discipline metrics– # of problem reports
– Problem report fix response time
– Overdue problem report fix responsiveness
– On-time delivery
Hardware & Software measurements– System Outages
Hardware measurements– Return rates
Software measurements– Software installation & release application aborts
– Corrective patch quality
– Feature patch quality
– Software update quality
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
33
TL9000 Common Audit Questions
• Do you know how to find your Quality Policy, QMS and the processes you should be using for your work?
• Do you know your organization’s product delivery & improvement goals and what you must do to support them?
• Do you know what skills you should have?
• Do you know what you have to do to approve/baseline/finalize your documents, designs & code?
• Do you know how to store & find records of reviews, inspections, key decisions, etc.?
• Do you know what to do if a problem is found with the product or process?
• Do you know your organization’s performance with respect to customer satisfaction, quality of delivered products & process execution?
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
34
TL9000 Sample Requirements
Planning– Must have methods for estimating & tracking
– Determine where you will do reviews & tests
– Risk management plans, customer, user & supplier involvement in reviews & evaluation
Software Outputs– Requires architecture, detailed designs, code & user documentation
– Each design thread must be reviewed at some point prior to integration or system test
Software Testing– All testing must have test plans; test process must be documented
– Plans must include test cases with inputs, output & test success criteria
– Plans must include types of testing, requirements traceability, coverage definition & measurement, test environment, defect handling, et.al.
– Integration testing specifically required
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
35
Team Project Postmortem
Tracking process improvements during projectProcess Improvement Proposals (PIP)
Port-Mortem
Areas to considerBetter personal practices
Improved tools
Process changes
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
36
Postmortem process
Team discussion of project data
Review & critique of roles
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
37
Postmortem process
Review Process DataReview of cycle data including SUMP & SUMQ forms
Examine data on team & team member activities & accomplishments
Identify where process worked & where it didn’t
Quality ReviewAnalysis of team’s defect data
Actual performance vs. plan
Lessons learned
Opportunities for improvement
Problems to be corrected in future
PIP forms for all improvement suggestions
Role EvaluationsWhat worked?
Problems?
Improvement areas?
Improvement goals for next cycle / project?
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
38
Cycle Report
Table of contents
Summary
Role ReportsLeadership – leadership perspective
Motivational & commitment issues, meeting facilitation, req’d instructor support
DevelopmentEffectiveness of development strategy, design & implementation issues
PlanningTeam’s performance vs. plan, improvements to planning process
Quality / ProcessProcess discipline, adherence, documentation, PIPs & analysis, inspections
Cross-team system testing planning & execution
SupportFacilities, CM & Change Control, change activity data & change handling, ITL
Engineer Reports – individual assessments
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
39
Role Evaluations & Peer Forms
Consider & fill out PEER forms
Ratings (1-5) on work, team & project performance, roles & team members
Additional role evaluations suggestions
Constructive feedback
Discuss behaviors or product, not person
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
40
Project Notebook
Updated Requirements & Design documents
Conceptual Design, SRS, SDS, System Test Plan, User Documentation*
Updated Process descriptions
Baseline processes, continuous process improvement, CM
Tracking forms
ITL, LOGD, Inspection forms, LOGTEST
Planning & actual performance
Team Task, Schedule, SUMP, SUMQ, SUMS, SUMTASK, CCR*
7/31/2007 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt
41
August 4 Class
CMMI Introduction & Configuration Management Appraisal
ISO 9000/TL-9000
Due July 31: Cycle 2 Design & Code, hand off to System TesterSystem Test Plan Inspected & BaselinedProject notebook updates including inspection records, meeting minutes, etc.
Deliverables for August 7Project Postmortem (cycle report)Cycle 2 presentationsPeer Feedback formsCompleted project notebooks
Cycle ExitCompleted project (source, documents & all quality records)
top related