ch6 performinng to asessment
TRANSCRIPT
SE423 SPICH6-Performing Assessment & ISO/IEC15504
Kittitouch Suteeca
Outline The Process Assessment Process Process Assessment Model Process Assessment Model Activities in Process Assessment ISO 15504 The Measurement Framework Roles and responsibilities
Performing an Assessment The requirements for performing an assessment defined in
ISO/IEC 15504 aim at achieving a greater degree of uniformity in the approach to process assessment, so as to maximize the reliability of different approaches and provide a degree of comparability between the results of different assessments.
It may make sense to verify the requirements prior to and during the course of the assessment so that corrective actions can occur.
The Process Assessment Process
ROLES AND RESPONSIBILITIES
SponsorCompetent Assessor
Assessors
ASSESSMENT PROCESS
PlanningData CollectionData Validation
Process Attribute RatingReporting
INPUTSponsor identity
PurposeScope
ConstraintsAssessment
Team
OUTPUTIdentification of Evidence
Process UsedProcess Profiles
PROCESS REFERENCE MODEL
Domain and ScopeProcess Purpose
Process Outcomes
PROCESS ASSESSMENT
MODELScope
IndicatorsMapping
Translation
MEASUREMENT FRAMEWORK
Capability LevelsProcess Attributes
Rating Scale
PRM:Process Reference Model The specific requirements for a candidate
process reference model A Process Reference Model shall contain:
A declaration of the domain of the PRM. Descriptions of the process within the scope of the
PRM., in term of process outcome and purpose. a description of the relationship between the
Process Reference Model and its intended context of use.
Example of PRM. ISO 9000, 12207, 29110, CMMi
Process Assessment Model
PAM:Process Assessment Model A Process Assessment Model forms the basis for the
collection of evidence and rating of process capability.
Any Process Assessment Model is related to one or more Process Reference Models.
A Process Assessment Model shall contain: a definition of its purpose, scope, elements and
indicators; its mapping to the Measurement Framework and the
specified Process Reference Model(s); a mechanism for consistent expression of results.
The Process Assessment Process
ROLES AND RESPONSIBILITIES
SponsorCompetent Assessor
Assessors
ASSESSMENT PROCESS
PlanningData CollectionData Validation
Process Attribute RatingReporting
INPUTSponsor identity
PurposeScope
ConstraintsAssessment
Team
OUTPUTIdentification of Evidence
Process UsedProcess Profiles
PROCESS REFERENCE MODEL
Domain and ScopeProcess Purpose
Process Outcomes
PROCESS ASSESSMENT
MODELScope
IndicatorsMapping
Translation
MEASUREMENT FRAMEWORK
Capability LevelsProcess Attributes
Rating Scale
Activities in Process Assessment
PlanningData collection
DataValidation
Rating Reporting
1.Planning
A plan for the assessment shall be developed and documented, including at minimum:
1) the required inputs defined in this part of ISO/IEC
15504(PAM.); 2) the activities to be performed in conducting the
assessment; 3) the resources and schedule assigned to these activities; 4) the identity and defined responsibilities of the
participants in the assessment; 5) the criteria to verify that the requirements of this
International Standard have been met; 6) a description of the planned assessment outputs.
Assessment input
Identify of the sponsor of the assessment;
Assessment purpose. Assessment scope. Assessment approach. Assessment constraints. The model(s) used within the
assessment.
Assessment input (cont’)
The role and identify of the assessment team, including the competent assessor.
The criteria of competent of the assessor who is responsible for the assessment.
The identify of assesse and support staff.
Any additional information to be collected during the assessment.
Activities in Process Assessment
PlanningData collection
DataValidation
Rating Reporting
2.Data collection
Data required for evaluating the processes within the scope of the assessment and additional information shall be collected in a systematic manner, applying at minimum the following:
1) the strategy and techniques for the selection, collection, analysis of data and justification of the ratings shall be explicitly identified and shall be demonstrable
2.Data collection
2) correspondence shall be established between the organizational unit’s processes, specified in the assessment scope, and the elements in the Process Assessment Model;
3) each process identified in the assessment scope shall be assessed on the basis of objective evidence;
2.Data collection4) the objective evidence gathered for
each attribute for each process assessed shall be sufficient to meet the assessment purpose and scope;
5) the identification of the objective evidence gathered shall be recorded and maintained to provide the basis for verification of the ratings.
Activities in Process Assessment
PlanningData collection
DataValidation
Rating Reporting
3.Data validation
The data collected shall be validated to:
1) confirm that the evidence collected is objective;
2) ensure that the objective evidence is sufficient and representative to cover the scope and purpose of the assessment;
3) ensure that the data as a whole is consistent.
Activities in Process Assessment
PlanningData collection
DataValidation
Rating Reporting
4.Process attribute rating A rating shall be assigned based on validated
data for each process attribute.1) the set of process attribute ratings shall be
recorded as the process profile for the defined organizational unit;
2) during the assessment, the defined set of assessment indicators in the Process Assessment Model shall be used to support the assessor's judgment in rating process attributes in order to provide the basis for repeatability across assessments;
4.Process attribute rating 3) the decision-making process that is used to
derive rating judgments shall be recorded; 4) traceability shall be maintained between an
attribute rating and the objective evidence used in determining that rating;
5) for each process attribute rated, the relationship between the indicators and the objective evidence shall be recorded.
ISO 15504 The Measurement Framework
Level 1 PerformedPA.1.1 Process Performance
Level 1 PerformedPA.1.1 Process Performance
Level 2 ManagedPA.2.1 Performance ManagementPA.2.2 Work Product Management
Level 2 ManagedPA.2.1 Performance ManagementPA.2.2 Work Product Management
Level 3 EstablishedPA.3.1 Process DefinitionPA.3.2 Process Deployment
Level 3 EstablishedPA.3.1 Process DefinitionPA.3.2 Process Deployment
Level 4 PredictablePA.4.1 Process MeasurementPA.4.2 Process Control
Level 4 PredictablePA.4.1 Process MeasurementPA.4.2 Process Control
Level 5 OptimizingPA.5.1 Process InnovationPA.5.2 Process Optimisation
Level 5 OptimizingPA.5.1 Process InnovationPA.5.2 Process Optimisation
Level 0 IncompleteLevel 0 Incomplete IncompleteThe process is not implemented or fails to achieve its purpose
PerformedThe process is implemented and achieves its process purpose
ManagedThe process is managed and work products are established, controlled and maintained.
PredictableThe process is enacted consistently within defined limits
OptimizingThe process is continuously improved to meet relevant current and projected business goals
EstablishedA defined process is used based on a standard process.
29110
Level 0: Incomplete process The process is not implemented, or fails to
achieve its process purpose. At this level there is little or no evidence of any
systematic achievement of the process purpose.
The Incomplete process is one that is either not performed at all, or for which there is little or no evidence of systematic achievement of the process purpose. Systematic achievement is characterized by the routine performance of necessary actions and the presence of appropriate input and output work products which, collectively, ensure that the process purpose is achieved.
Level 1: Performed process The implemented process achieves its
process purpose. The following attribute of the process
demonstrates the achievement of this level:
The Performed process achieves its process purpose through the performance of necessary actions and the presence of appropriate input and output work products which, collectively, ensure that the process purpose is achieved.
Presenting the ResultsProcess Profiles and
Capability Level Rating
Rating process attributes ISO/IEC 15504 provide the basic
elements of a measurement framework for rating process capability. To complete the framework, a scale for rating the extent of achievement of a process attribute is defined.
28
Rating process attributes Process attribute rating values: The ordinal
rating scale defined below shall be used to express the levels of achievement of the process attributes. N Not achieved: There is little or no evidence
of achievement of the defined attribute in the assessed process.
P Partially achieved: There is some evidence of an approach to, and some achievement of, the defined attribute in the assessed process. Some aspects of achievement of the attribute may be unpredictable.
29
Rating process attributes L Largely achieved: There is evidence of
a systematic approach to, and significant achievement of, the defined attribute in the assessed process. Some weakness related to this attribute may exist in the assessed process.
F Fully achieved: There is evidence of a complete and systematic approach to, and full achievement of, the defined attribute in the assessed process. No significant weaknesses related to this attribute exist in the assessed process.
30
Rating process attributes The corresponding values shall be:
N Not achieved 0 to 15% achievement P Partially achieved >15% to 50%
achievement L Largely achieved >50% to 85%
achievement F Fully achieved >85% to 100%
achievement.
Activities in Process Assessment
PlanningData collection
DataValidation
Rating Reporting
5.Reporting The assessment results, including at
minimum the outputs specified PAM., shall be documented and reported to the assessment sponsor or to their delegated representative.
Assessment FeedbackShall contain Introduction of organization context. Assessment Scope Rating Results Process Related Risk Strengths Weaknesses Potential Improvement Actions
Objective No.
O.1 O.2 O.3 O.4 O.5 O.6 O.7
PM. N L P P F P P
SI. L L N F F P P
Attribute
N Not Achieved
P Partially Achieved
L Largely Achieved
F Fully Achieved
Rating Result
The Process Assessment Process
ROLES AND RESPONSIBILITIES
SponsorCompetent Assessor
Assessors
ASSESSMENT PROCESS
PlanningData CollectionData Validation
Process Attribute RatingReporting
INPUTSponsor identity
PurposeScope
ConstraintsAssessment
Team
OUTPUTIdentification of Evidence
Process UsedProcess Profiles
PROCESS REFERENCE MODEL
Domain and ScopeProcess Purpose
Process Outcomes
PROCESS ASSESSMENT
MODELScope
IndicatorsMapping
Translation
MEASUREMENT FRAMEWORK
Capability LevelsProcess Attributes
Rating Scale
Roles and responsibilities
Responsibilities of the Sponsora) verify that the individual who is to take
responsibility for conformity of the assessment is a competent assessor;
b) ensure that resources are made available to conduct the assessment;
c) ensure that the assessment team has access to the relevant resources.
Roles and responsibilities Responsibilities of the Competent Assessora) confirm the sponsor's commitment to
proceed with the assessment;
b) ensure that the assessment is conducted in accordance with the requirements of this part of ISO/IEC 15504;
c) ensure that participants in the assessment are briefed on the purpose, scope and approach of the assessment;
Responsibilities of the Competent Assessor
d) ensure that all members of the assessment team have knowledge and skills appropriate to their roles;
e) ensure that all members of the assessment team have access to appropriate documented guidance on how to perform the defined assessment activities;
Roles and responsibilities Responsibilities of the Assessors a) carry out assigned activities associated
with the assessment, e.g. detailed planning, data collection, data validation and reporting;
b) rate the process attributes.
References ISO/IEC 15504-3 manual