55

Multi-Source Assessment – A Total Quality Management DrivenPerformance Appraisal System: The Issue of Fairness

ZAINI ABDULLAHInstitute of Knowledge Advancement (InKA)

Universiti Teknologi MARA

RUSHAMI ZIEN YUSOFFFaculty of Management of Technology

Universiti Utara Malaysia

ABSTRACT

Performance Management and Total Quality Management are important factors of organizational life.It helps organizations to identify and manage the performance of highs achievers, as well as identifylow and non-performers for intervention and development. Total Quality Management takes a system-oriented perspective in managing individuals and group performance and must be compatible withcontinuous improvement efforts and customer-based strategies. The TQM driven Multi-source Assess-ment provides an alternative to traditional supervisor only appraisal process. This study was under-taken to determine whether Total Quality Management (TQM) based multi-source assessment (MSA)appraisal is perceived as fairer than traditional supervisor only appraisal system. The study was con-ducted using multi-group experimental design in a field setting. The perceived fairness of the perform-ance assessment system was tested using the combined pretest posttest data of the experimental andcontrol group. The t-test results indicated that changes in agreement scores from pretest to posttestresult were significant and respondents perceived the TQM driven MSA process to be fairer. The resultsand discussion of the study show that the MSA process depicts a paradigm shift, not only to manage-ment process, but how employees see their changing roles in total quality management driven organi-zation.

ABSTRAK

Pengurusan Prestasi dan Pengurusan Kualiti Menyeluruh adalah faktor penting dalam kehidupanorganisasi. Ianya membantu organisasi mengenal pasti dan mengurus prestasi mereka yang cemerlangdan juga mengenal pasti ahli-ahli organisasi yang lemah dan kurang berprestasi untuk usaha intervensidan pembangunan. Pengurusan Kualiti Menyeluruh menggunakan perspektif yang berorentasikan sistemdalam pengurusan prestasi individu dan pasukan bagi memastikan ianya selaras dengan usaha yangberterusan dan strategi berpaksikan pelanggan. Penilaian Pelbagai Sumber (Multi-Source Assessment)yang berpaksikan Pengurusan Kualiti Menyeluruh memberikan alternatif kepada proses penilaiantradisional yang berpaksikan proses penilaian daripada ketua. Kajian ini dilaksanakan untuk menilaisejauh manakah Sistem Penilaian Pelbagai Sumber yang berorentasikan Pengurusan Kualiti Menyeluruhdianggap lebih adil daripada sistem penilaian tradisional. Kajian yang dijalankan menggunakan rekabentuk eksperimen pelbagai kumpulan. Tanggapan keadilan sistem penilaian prestasi diuji menggunakangabungan data pra ujian dan pasca ujian daripada kumpulan kawalan dan kumpulan eksperimen.Ujian-t menunjukkan keputusan yang signifikan di mana responden mendapati Penilaian Pelbagai

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

56

INTRODUCTION

Performance appraisal is the key to effective hu-man resources management. Latham and Wexley,(1982) identified selection, appraisal, training andmotivation as the four key systems necessary forensuring the proper management of an organiza-tion’s human resources. Of these four, they arguethat performance appraisal is the most importantbecause it is the prerequisite for establishing theother three. The salience of an effective perform-ance appraisal system as a tool to enhance humanproductivity is well documented (Glueck, 1978).Appraisal system is a critical function in organi-zations because they enhance productivity by tar-geting individual developments needs and at thesame time, identifying high, medium and low per-formers (Bernardin & Betty, 1984; Latham &Wexley, 1982; and Henderson, 1989).

Organizational Challenges in Today’s Environ-ment – Its Effects on Performance AppraisalA number of changes are taking place in manyorganizations, be it public or private. These large-scale organizational changes are causing manyrules to be re-evaluated (Mohrman, Mohrman,Ledford, Cummings & Lawler, 1988), a re-evalu-ation which raises questions about the perform-ance appraisal process. Changes driving the or-ganizations include both environmental and cul-tural. Changes in the environment, such as fastchanging technology, development of self-directedteams and the incorporation of customer satisfac-tion as the organizational objective, reflect a morerapid change in society

Technological ImpactThe impact of changing technology has an effectboth on employees’ access to information and ontheir ability to manage large amounts of data.Contemporary workers often have extensiveknowledge or access to technology-based infor-mation. These knowledgeable workers may over-

see day-to-day responsibilities, and be involvedin managerial decision-making. This access toinformation has changed the power relationshipwithin the organization.

Self-Directed TeamsThe increasing use of self-directed work teamsdrives new adaptive responses (Mohrman,Mohrman & Lawler, 1987). Self-directed workteams provide a new model for getting work ac-complished. The self-directed work team devel-oped from quality initiatives, fosters employees’participation. Performance assessment becomesa critical factor for the self-directed team. Perform-ance measures are needed to identify the degreeto which the team is meeting its goals and objec-tives, and who within the team is and is not con-tributing to the success. These changes requirenew approaches to the traditional performanceappraisal process, which relies on the supervisoralone to make the performance assessment.

Customer Satisfaction as OrganizationalObjectiveIn meeting the challenges faced in this customer-oriented era, many organizations adapt their strat-egies so as to ensure that they can deliver the needsand wants of their customers. Customer focus isthe corner stone of the principle of quality. Theseorganizations use customer satisfaction to drivetheir business (Stephen & Weimerskirch, 1998).Quality management system such as ISO 9000 hasmade a significant change in their concept by in-cluding customer satisfaction analysis as one ofthe main drivers of the system.

Shortcomings of the Current AppraisalsThere is widespread dissatisfaction with most ofthe appraisal system currently used. Most of itfocuses on the single-source (supervisor only)appraisal process. Substantial research indicatesthat traditional performance appraisal does notaugur well with current organizational settings.

Sumber (MSA) lebih adil daripada penilaian tradisional (SSA). Hasil kajian juga menunjukkan prosesMSA memperlihatkan perubahan paradigma, bukan sahaja kepada proses pengurusan, begitu jugabagaimana ahli organisasi melihat perubahan peranan mereka dalam organisasi yang berorientasikanpengurusan kualiti menyeluruh.

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

57

The issues of bias and practicality are ofgreater interest to this discussion. The traditionalperformance appraisal process is subjective andbiased (Latham & Wexley, 1994). Other notableflaws are the vagueness of standards, subjectivityof rater and inconsistency among appraisers, thatis, from one supervisor to another (Basnigth &Wolkinson, 1977).

Total Quality Management and Human ResourcePerformance Appraisal SystemThe total quality approach is changing the role ofhuman resource management by changing theexpectation of stakeholders, within and outsidethe organization, with regards to human resourceresponsibilities. Today the objectives of total qual-ity human resource approach are to increase cus-tomer satisfaction, both internal and external,through improved quality and to develop a moreflexible, loyal and innovative work environment.Improvement in internal efficiencies is impera-tive to ensure organizational effectiveness and thepractice of quality human resource managementtechniques can enhance internal efficiency(Gunasekaran, Korukonda, Virtanen, & Yli-Olli,1994) and competitiveness (Petrick & Quinn,1997). Performance assessment of the employeescan be reflected in the performance measurementsystem that help to monitor and maintain organi-zational control. This is a process of ensuring thatorganizational resources e.g. human resource canpursue strategies that can lead to the achievementof overall goals and objectives (Nani, 1987).

Total Quality Management (TQM)means that the organization’s culture is definedby and supports the constant attainment of inter-nal customer satisfaction through an integratedsystem of tools, techniques and training. This in-volves the continuous improvement of organiza-tional process resulting in quality ‘products’ andservices (Sashkin & Kaiser, 1993).

Organization invests considerable re-sources while implementing TQM and it will bebeneficial to quantify the derived benefits in termsof improved effectiveness and competitiveness sothat resource utilization can be improved. Totalquality will remain a strategic issue in managinghuman resource for ensuring competitiveness inthe knowledge economy characterized by intense

competition and a continually increasing demandfor a better management system.

Deming, (1986) proposed a “theory ofprofound knowledge”. A key aspect of this theoryis that the success of quality management effortsdepends on effective integration of various man-agement sub-systems. The maximum effective-ness of TQM driven Performance ManagementSystem may hinge on whether performance man-agement sub systems are integrated and support-ive of each other. In the pursuit of effectiveness,one sub system cannot be optimized at the ex-pense of another.

TQM proponents are quick to criticizeperformance appraisal practices, which are basedon the assumption that the individual employee islargely in control of his or her own performancelevel (Deming, 1986; Scholtes, 1987; and Walton,1990). An individual based focus has been theprimary paradigm used by researchers in the areaof performance management. This includes anemphasis on individual performance appraisal,goal setting and feedback. Performance appraisalhas been a subject of much interest to HRM re-searchers. Most of this effort has been devoted tocharacteristics of instruments and raters, with un-derlying goals to eliminate errors of bias, betterunderstanding of performance-related informa-tion-processing, and ultimately to improve ratingaccuracy.

Research on performance managementhas included other individually focused phenom-ena that may occur before, during and after ap-praisal. These include planning processes and in-terventions designed to maintain and improveperformance. For example, individual feedbackand goal setting have both received much acclaimas effective ways to manage tasks performance(Ashford & Cummings, 1983).

Characteristics that include thespecificity of feedback, and the extent to which itincludes both behaviors and outcomes, and thedegree of difficulty to achieve goals have beenassociated with high performance (Chokar &Wallin, 1984; and Locke, Shaw, Saari & Latham,1981). The interconnectedness of goal setting andfeedback has also been discussed. Tolchinsky andKing, (1980) and Bandura and Corvone, (1983)proposed that the effect of feedback depends on

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

58

the beliefs that such feedback generates the goalsthat are set in response to the feedback.

Another problem associated with an in-dividually based approach is the assumption thatby enhancing individual task performance, the per-formance of the greater unit or organization willbe enhanced. TQM proponents have been criticalof the use of such individualized goal-setting prac-tices as management by objectives for two rea-sons. Firstly, individuals may attempt to set or ne-gotiate less challenging or easy to achieve goals.Secondly and perhaps more importantly from aTQM perspective, goals tend to be stressing onshort-term outcomes. As noted by Deming, (1986)the traditional appraisal system will reward peo-ple who do well “within the system” but do littlein an attempt to improve the system. These arepotentially growing problem in the organizationparticularly in effectively managing performanceof human resources “humanly fair”.

Dobbins (1991) argued that “perform-ance management activities would be much morefruitful if directed at system”. A system-orientedapproach would be primarily oriented towardsimproving processes, which affects the perform-ance of all the individuals within the system. Forexample in performance appraisal exercise, thework flow or technical process need to be evalu-ated and then improved in such a way as to en-hance the effectiveness and fairness of the evalu-ation process.

Oakland, (1993) suggested that appro-priate performance measurement plays importantroles in quality and productivity improvements.These are to include ensuring that customers re-quirements are being met; providing standards forestablishing comparison, and providing visibilitiesand “score board” for people to monitor their ownperformance levels. The measurement shouldhighlight quality problems and determine areasrequiring prior attention. It should also providefeedback for driving improvement effort. For anyperformance appraisal system to be Total QualityManagement (TQM) based it must be aligned toTotal Quality Management (TQM) principles.Knouse (1996) and Stephen and Weimerskirch(1998) suggested that human resource system ingeneral and specifically performance appraisal andcompensation system should be customer driven,

team oriented, based on observable and measur-able behaviors and results, encourage full em-ployee participation, include timely feedback andenhance continuous improvements.

The Multi-Source Assessment (MSA) Model –ATQM based Appraisal SystemMulti-source Assessment model, a relatively newappraisal method, recommends that performanceinformation come from multiple individuals whointeract with the assessment receiver (Lawler,1967 & Bernardin, 1986). The multi-source modelwhich is a resemblance of the 360 degrees feed-back model, may include the supervisor’s assess-ment plus that other work associates. These workassociates may be colleagues, associates, peers,direct reports, the second level supervisor, andinternal and external customers. This fits well withthe team oriented and customer driven TQM prin-ciples discussed earlier.

Multi-source assessment introduces animportant concept to the process of performanceassessment. It is highly regarded that individualshould be assessed by work associates with whomthey interact frequently. This approach combinesinformation from several sources that leads to“more valid and reliable performance appraisal”of individual performance (Baruch & Harel,1993).

Issues of reliability and validity are criti-cal to any performance appraisal model. Perform-ance appraisal systems that rely solely on super-visor evaluations assume that supervisor is ableto observe closely their employees’ behavior. Thecharacteristic change of MSA is that it expandsthe evaluation team beyond the immediate super-visor, to include others such as peers, direct re-ports, internal and external clients and a self-ap-praisal. A key concern of the traditional single-source appraisal model is that it is biased becauseit relies on a single perspective to provide the as-sessment.

In today’s organization, where employ-ees may have considerable autonomy, the assump-tion that supervisors may have large spans of con-trol may not hold true. Using different viewpointsprovides a more comprehensive picture of indi-vidual performance (London & Beatty, 1993). TheMSA process compensates for the limited perspec-

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

59

tive of the supervisor by expanding the evalua-tion team to include those in the performer’s cir-cle of influence. Adding additional raters obvi-ously can help improve the reliability and valid-ity of the ratings. The MSA process represents notjust an incremental change to the performanceappraisal process but one that also fundamentallyalters the power dimension of performance assess-ment by soliciting input from multiple levels ofthe organization.

The MSA process focuses on organiza-tional culture and its human processes on howhuman intact. This is consistent with the organi-zational development interventions, described asfacilitating an effective transformation ofbehaviors. This transformation requires

Developing skills for inducing effec-tive and satisfying interpersonal andinter-group relationships at work; in-creasing competencies to induce andsustain desired and desirable organi-zational climates; augmenting skillsin conflict resolution; enhancingskills and attitudes to satisfying andproductive choice or change; and en-forcing specific counter-bureaucraticattitudes and behaviors, for examplethose related to risk-taking, opennessand so on (Golembiewski, 1985).

The organizational culture and human processfocus of MSA thus aligned well with TQM prin-ciples that any appraisal system must be based onobservable and measurable behaviors and results(Knouse, 1996; and Stephen & Weimerskirch1998)

OBJECTIVE OF THE STUDY

The MSA process provides several unique char-acteristics to the performance appraisal process.The most significant change is the expansion ofthe performance evaluation team. This majorchange provides the basis for the hypotheses be-ing tested in this study. The overall thesis is thatby expanding the evaluation team from one per-son (the supervisor) to a team of co-workers with

whom the appraise interacts, it is expected thatthe fairness of the appraisal, both perceptually andempirically, will increase.

Perceived fairness of performance ap-praisals may be based upon a number of factors;organizational trust, input, methods of data col-lection, rater training, reliability and anonymity.A key factor is, the knowledge the rater has of theindividual being assessed. It is proposed that ex-panding the evaluation team from one perspec-tive to multiple perspectives, and with those per-spectives including people of different status inrelation to the subject, will be perceived as pro-viding a fairer appraisal rating by the appraise.Multi-source assessment appraisal process is ex-pected to be perceived as fairer than supervisoronly appraisals because the evaluation team ismade up of individuals with high degree of knowl-edge of the appraisee’s performance.

Based on the above background, thestudy was carried out with the objective of inves-tigating and confirming empirically whether theMulti-Source Assessment (MSA) is perceived tobe fairer than the Single-Source Assessment (SSA)

Research HypothesisThe hypothesis for this study are as follows:Hypothesis 1: The TQM based Multi-source ap-

praisal (MSA) is perceived to befairer than traditional supervisor-only performance appraisals(SSA).

Based on the literature of fairness, the study in-vestigate the issue of fairness from the followingdimensions:- Comprehensiveness of the assessment- Accuracy of assessment on performance- Ability to provide effective feedback to

appraisee- Perceived to improve performance of the

appraisee- Ability to provide more confidence in the as-

sessment feedback- Ability to provide more safeguards

In investigating the issue of fairness from theabove dimensions, the study established the fol-lowing sub-hypothesis:

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

60

Hypothesis 1a: TQM based Multi-source ap-praisal (MSA) is a more com-prehensive assessment than thetraditional Single-Source Ap-praisal (SSA).

Hypothesis 1b: TQM based Multi-source ap-praisal (MSA) process is per-ceived to be a more accurateassessment of performance thanthe traditional Single-SourceAppraisal (SSA)

Hypothesis 1c: TQM based Multi-Source Ap-praisal (MSA) assessment isperceived to provide a moreeffective feedback to theappraisee than the traditionalSingle-Source Appraisal(SSA).

Hypothesis 1d: TQM driven Multi-Source Ap-praisal (MSA) outcome is per-ceived to improve performanceof the appraisee more than tra-ditional Single-Source Ap-praisal (SSA).

Hypothesis 1e: TQM based Multi-Source Ap-praisal (MSA) is perceived toprovide more confidence in theassessment feedback than thetraditional single-source ap-praisal (SSA).

Hypothesis 1f: The TQM - Multi-source As-sessment is perceived to pro-vide more safeguards than thetraditional single-source ap-praisal (SSA).

METHODOLOGY

This study’s research design is an experimentaltest of the Single-Source Appraisal model andMulti-Source Appraisal model for performanceappraisal. The experimental research design isbased on the modified version of the Four GroupDesign established by Solomon (1949) and promi-nently employed by Campbell and Stanley (1963)in a simillar experimental setting. The empirical

data has been gathered from a Multi-Source Ap-praisal research project implemented at a publicinstitution of learning. The primary test is regard-ing appraisee perceptions of fairness. This in-volved the use of pre- and post-experiment (treat-ment) survey to collect information from appraiseeon their perception of fairness on the evaluationexercises.

The research design has been modifiedto address the need to compare issues of fairnessbetween the three groups and the types of appraisalsystem used. This modified version of the Multi-ple- Group Experimental Design has the follow-ing structure:

O1 X1 O2O1 X2

X3 O2Whereas X1 represent experiment group I (pre andpost treatment) using single-source and multi-source appraisal, X2 represent control group (pretreatment) using the existing single source ap-praisal method and X3 represent experiment groupII (post treatment) using the multi-source appraisalassessment. Os are the observations of the depend-ent variables, where O1 using single source per-formance appraisal and O2 using multi-sourceappraisal.

The population for the study included allthe ninety-eight academic staff of one of the Tech-nical school in Perlis, Malaysia. After discount-ing those who were involved in the focus groupdiscussions for the development of the instrumentsfor the study and those involved in the pilot test,fifty of the academic staff were taken to be thesample for this experiment. The pre-treatmentsurvey was given to participants after completionof their traditional supervisor-only assessmentregarding their perceived level of fairness of theprocess and the resultant rating. After the imple-mentation of the Multi-Source Assessment, post-treatment surveys were conducted to assess theperceived level of fairness of the Multi-SourceAssessment process. The ratings of perceptualfairness from the pre- and post- test instruments(SSA and MSA) on each response groups werecompared using t-test. t-test was also used to testall other related sub-hypothesis.

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

61

The Performance Evaluation Format: The Single-Source Appraisal vs. the Multi-Source AppraisalFormsPerformance evaluation format based on SSA andMSA were developed to measure the work per-formance of the appraisee. These performanceevaluation formats were developed through a fo-cus group discussion and interviews with the of-ficers from the State Education Department,school teachers and higher institutions lecturers.Altogether four evaluation formats were devel-oped and used. There are as follows:

1. Single Source Appraisal format (based onexisting PSD’s format)

2. Multi-Source Appraisal format for Superiorappraisal

3. Multi-Source Appraisal format for Peer ap-praisal

4. Multi-Source Appraisal format for Students(customers) appraisal

The Research InstrumentThe research questionnaire on the perception offairness in the performance assessment was de-signed to answer the research questions of thisstudy. It contained statements and questions aimedat tapping the aspects of general demographicbackground information on respondent and degreeof perceptual fairness on SSA vs. MSA appraisalsystem was based on:

. The level of comprehensiveness in the per-formance appraisal system. The effectiveness of the appraisal system. The level of the performance appraisal out-comes and achievements. The process and procedures of evaluationConfidence in the appraisal scores. Provision for safeguards against biasness anddiscriminatory practices

The Pre-Experiment (Treatment) Survey Question-naireThe purpose of this questionnaire is to measurethe degree of perceptual fairness of the Single-Source Appraisal process as a performance assess-

ment measure. The single-source appraisal ap-proach is identical to the commonly practicedperformance assessment method used in the pub-lic service employment. Perceptual fairness of anindividual can be represented by three main ap-proaches - outcome, procedure and system. Out-comes (rewards or punishment) come about dueto performance ratings. The procedure is the man-ner and method by which the decisions on out-comes are made. The system defines the organi-zational context in which the procedure operates.The system both influences the outcome and pro-cedure.

The elements of the perceptual fairnessin the survey have been operationalized into a setof feedback response statements. The perceptualfairness statements can be categorized into thefollowing constructs: system and structural at-tributes, instrument or task process profile andachievements/outcomes factors. The system andstructural attributes dimension discusses the per-ception of fairness from the organizational con-text in which the procedures operates. The Hageand Aiken Formalization Inventory (based onMiller, 1991) was adapted to formulate statementsfor this dimension. In this context perceptual fair-ness was measured by the comprehensiveness andeffectiveness of the appraisal system.

The instrument or task process profilesdescribe the procedures and processes in evaluat-ing employees fairly. Procedural justice (fairness)represents the processes for ensuring fairness, orequity in appraising performance (Sheppard,Lewicki & Minton, 1992). Significant research hasargued that the “perception of procedural fairnessis at least as important as perception of outcomesof fairness” (Lind & Tyler.1988). The achieve-ments or outcomes factors are key components toperceptual fairness in any evaluation system. Theoutcomes may be extrinsic in nature such as re-wards, promotion or for purpose of training or rec-ognition. Distributive justice (fairness) articulatesthe equity perspective of fairness. People expectto receive rewards that commensurate their efforts.Perceptual fairness has been closely linked to per-formance and productivity (Sashkin & Williams,1990).

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

62

The Post -Experiment (Treatment) SurveyQuestionnaireThis post-test questionnaire is designed to meas-ure the degree of perceptual fairness of the Multi-Source Appraisal process. The multi-source ap-praisal approach is not a commonly practiced per-formance assessment method in the public sector.Again, the measure of perceptual fairness of anindividual can be represented by the three mainapproaches - outcome, procedure and system - asdescribed earlier.

Validity and Reliability of the Research Instru-mentsThe content validity of all the research instrumentsused in the study were established through focusedgroup discussions comprising experts from abroad spectrum. The reliability coefficients of thescales used in the study are given in Table 1. Thecoefficient of the alphas of the scales used in thestudy ranged from 0.8489 to 0.9356; none of thecoefficient was below 0.7. The reliability coeffi-cients of the research constructs for comprehen-sive assessment system, effective assessment sys-tem, process and procedures and achievements andoutcomes were 0.9394, 0.9446, 0.8877,0.9525respectively.

Table 1:Reliabilities of Scales Used (Alpha Coefficients) (Pretest data)

Scales No. of items Alpha Coefficient

Comprehensiveness 8 .9403Accuracy on performance 8 .9451Effective feedback to appraisee 8 .8877Outcome on performance ,10 .9534

Similar lines of questions (statements) will beposed with emphasis on the Multi-Source Ap-praisal. Summation of all the questions (state-ments) on the various dimensions for allappraisees are categorized as follows: system andstructural attributes, instrument or task processprofiles and the achievements and outcomes fac-tors.

RESULTS AND ANALYSIS

This section focuses on the data analysis of theperformance ratings. All employees (teachers)were required to participate in the research pro-cess that involved the Single Source and Multi-Source Assessment

FindingsThis section presents the results of the sevengroups of hypothesis consisting of one primaryand six sub-hypothesis. These hypothesis were

tested based on participants’ responses to perform-ance assessment method, effectiveness of the as-sessment; procedures and process of assessment,and the achievements of the appraisal approach.

Summary of Research Hypothesis ResultsThe primary hypothesis agrees that the Multi-Source Assessment (MSA) approach provides afairer method of assessment than the traditionalSupervisor-only (SSA) performance assessmentas perceived by appraisees.. Sub hypothesis (1a) agrees that the multi-

source assessment is perceived as fairer be-cause it represents a more comprehensive(complete) assessment of the performancethan the single source assessment.. Sub hypothesis (1b) agrees that the multi-source assessment is perceived as more ef-fective because it represents a more accurateassessment of the performance than the sin-gle source assessment.. Sub hypothesis (1c) agrees that the feedback

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

63

evaluation from the evaluating team, includ-ing co-workers and internal customers (stu-dents), will be perceived to be more effec-tive to the appraisee than feedback only fromthe supervisor.. Sub hypothesis (1d) agrees that the multi-source assessment outcome is perceived toimprove performance of the appraisee morethan traditional source single source assess-ment.

. Sub hypothesis (1e) agrees that Individualappraisees have more confidence in the multi-source assessment than the traditional single-source assessment.. Sub hypothesis (1f) agrees- that the multi-source assessment will be viewed byappraisees as providing more safeguards thanthe traditional single-source appraisal proc-ess.

Table 2 :SSA and MSA Survey Comparison

Construct Hypothesis Mean SSA Std. Mean MSA Std. t-test- and Sub- dev Median dev Medianhypothesis

Perceived fairness 1 2.35 2.5 .50 2.97 3.0 .05 .0.001**Comprehensiveness 1a 2.37 2.5 .63 3.0 3.0 0 .0.05*Accurate assessment on performance 1b 2.47 2.75 .57 3.0 3.0 0 .0.05*Effective feedback to appraisee 1c 2.25 2.07 .45 2.9 3.0 .19 .0.001**Outcome on performance 1d 2.32 2.25 .61 2.9 3.0 .03 .0.001**Provide confidence 1e 2.7 3.0 .48 3.0 3.0 0 .0.001**Provides safeguards 1f 2.1 2.0 .87 3.0 3.0 0 .0.05*

* significant at p<.05 level** significant at p<.001 level

DISCUSSION

The results of this study show that a change inhow performance management is undertaken hasan effect on user perception of performance ap-praisal fairness in organizations. Multi-source as-sessment appears to provide a tool that fostersperceived fairness in the performance assessmentfeedback process. It overcomes the issues ofbiasness and subjectivity that are inherent in thetraditional single source performance appraisalprocess as pointed out by Latham and Wexley(1994). It also manages to solve the problems thatare related to the vagueness of standards, subjec-tivity of rater and inconsistency among apprais-ers that are present in the single source appraisalprocess as reported by Basnigth and Wolkinson(1977). The process may increase the level of trustwhere the system is treating individuals fairly. This

ultimately would also provide greater satisfactionto the employees and supervisors. Therefore,multi-source assessment process can facilitateorganization’s response to the dramatic changesin the workforce and the environment.

For many employees, the introductionof co-workers or peers to the rating process en-hanced the perspective of performance. It was rec-ognized by employees that supervisors could notprovide a comprehensive review of performance.This could be due to their broadened job respon-sibility, lower levels of technical skills, or remotereporting situation. The addition of co-workersbroadens the perspective of performance, leadingthe individual to view performance rating as morecomprehensive and complete. This corroborateswith Kaplan’s (1993) reiteration that co-workersshould be treated as internal customers and there-fore they can provide useful feedback to enhance

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

64

workers performance. The MSA process that in-cludes peer evaluation therefore will broaden theperspective of performance measurement. Theevaluation team consists of members of the knowl-edge network around the individual performers,e.g. co-workers, who have greater knowledge ofthe job performance than the supervisor or rater.These co-workers may spend more time either inwork or non-work settings, plus more likely tohave similar technical information that is relevantto the task or job done. For this reason, co-work-ers feedback becomes more motivational becauseit is more credible and reliable, coming from asource that is “on site” and more aware of the jobresponsibility. This agrees with Cummings andSchwab’s (1973) and Baruch and Harel’s (1993)views that peer appraisal is a reliable and validsource of performance measures as ratings of peerswhich show a correlation with work performance.

The process gives appraisees confidencein the results. Organizational trust is fosteredthrough competency and participation. The Multi-source assessment process focuses on specificbehavioral competencies identified as relevant tothe organization’s specific responsibility. As ar-gued by Dobbins (1991), the focus on competen-cies is important, as it will enhance the useful-ness of the appraisal process in improving work-ers performance. Employee participation enhancesconfidence in the overall process; the involvementof the individual’s co-workers in providing per-formance evaluation, enhances confidence, ortrust, in the ratings results that the appraisal proc-ess provides. This is in aggreement with Schneir(1977), that MSA process is highly participatorybecause it uses multi-rater and increases partici-pation beyond the supervisors in the performanceappraisal process. This MSA approach is consist-ent with the high involvement oriented manage-ment practises.

The process provides safeguards that arenot a feature of the single-source supervisory as-sessment. Multi-source assessment provides ad-ditional options with regards to the design mecha-nism of the assessment system that provide safe-guards to the process. The ability to organize andtrack raters and indicate degree of inter-rater agree-

ment is an enhancement to ensuring a fairer ap-proach to performance assessment. This concurswith London and Beatty’s (1993) findings thatMSA incorporates safeguards to fairness andtherefore enhances participation and confidencein accuracy of the feedback receivers. This is nota characteristic or even feasible within the formatof the single source appraisal system. Providinganonymity to feedback providers or raters is nec-essary for assessing co-workers to identify indi-vidual or ratee strengths and weaknesses. With-out this safeguard, evaluators may not be willingto identify and discuss weaknesses. This wouldundermine the Multi-source assessment processby neutralizing the feedback and creating undif-ferentiated reports such as that provided by thesingle-source appraisal method.

Rater tracking is made available in theMulti-source assessment processes which ensurerater anonymity. Rater anonymity must be sus-tained to ensure perceived fairness of the MSAprocess. London and Beatty (1993) indicated thatanonymity is fundamental to a fair appraisal proc-ess and this is supported by their research find-ings. However, rater tracking relies on the usageof a software program that can balance the desirefor anonymous ratings while protecting appraiseesor ratees from personal attacks or retaliations. Ifthe number of raters is too small and it is virtuallyimpossible to remain anonymous, the peer assess-ment (as assigned) could be justified as an offi-cial system of the assessment process just as thesingle supervisor assessment.

Many supervisors perceived the Multi-source assessment process as supporting theirongoing responsibility to provide performanceassessment by giving them information beyondthe normal range to which they had access. Multi-source assessment enhances their roles as coachesand developers. This support the views pointedout by Latham and Wexley (1982), that MSA proc-ess moves supervisor from the role of judge tothe role of coach. Through the Multi-source as-sessment mechanism, they have total responsibili-ties for providing a comprehensive and accurateappraisal. Information was provided to help themfacilitate employees’ development.

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

65

CONCLUSION

Performance management is a necessary and im-portant process in organizational life but it has notworked as well as the theory suggests. Many or-ganizations have changed so radically in recentyears, requiring mechanism to respond to custom-ers, stakeholders, and employee requirements.TQM initiatives are underway in many public sec-tor organizations. These initiatives present addi-tional opportunities to the performance manage-ment process regarding employee involvement,open communication and “driving out of fear ”.While the orientation in public organization mustcontinue to focus on efficient and effective deliv-ery of services, other considerations must also beaddressed.

This research tested an Multi-source as-sessment process as a model for performance as-sessment that addresses multiple areas of concern.The results of this research may change how qual-ity driven performance management is undertakenin the future for organizations. The Multi-sourceassessment process provides a new approach lead-ing to a fairer system of performance assessmentratings. It gives higher quality information thanjust the single-source appraisal system. It is morespecific and viewed as being fairer and more ac-curate as it provides a more comprehensive per-spective of performance. Consequently, the moti-vational impact on the assessment is higher thanthe traditional single-source appraisal process. Themulti-source assessment fosters continuous im-provement in performance by enhancing commu-nication and improving cooperation and increas-ing the level of confidence and trust.

The Multi-source assessment processevaluated in this research indicates that it addsvalue to the performance appraisal process. Par-ticipants evaluated the process as being fairer thanthe previous assessment process, the single-sourceassessment. Additionally, employees who oftenexperience an “unfair” assessment results ratedthe new assessment process Multi-source assess-ment (MSA) as significantly fairer.

The issue of fairness is critical to publicorganizations. The focus on providing efficientand effective services continues to be a basic valueand driving force of the public sector departments.

The focus of this research on fairness asserts thatemployees’ rights are also important. Fairness isa fundamental value inherent in the structure ofthe democratic process. Fairness is a critical vari-able in supporting the performance assessmentprocess in organizations. Enhancing employees’beliefs that the performance assessment processis fair serves all parties – the organization, the in-dividual and the public served.

This research demonstrates an alterna-tive approach for public sector agencies like theschool system to meet the challenge of changingorganizational realities, including employee andclient expectations. The call to bring aboutchanges in government’s rigid mechanism of per-formance ratings and assessment requires signifi-cant changes to how government employees areevaluated on their performance. Allowing theemployee to develop performance assessment cri-teria that is tied to the organization’s mission andvalues provides a compelling mission on perform-ance expectations. Expanding the evaluation teamfrom the traditional supervisor’s appraisal only,to a group of co-workers and related clients, whosee the employee in a variety of settings and morefrequently, strengthens the performance assess-ment process.

It is envisioned that this research willprovide a benchmark model for multi-source as-sessment appraisals. This study will assist thePublic Services Department of Malaysia in de-veloping and implementing a performance assess-ment process in line with the Malaysian Remu-neration System of managing public sector per-formance management system. This research alsoprovides a process for organizations to fosteremployees’ involvement in the TQM initiatives.It provides an insight for organizations to use To-tal Quality Management (TQM) driven perform-ance assessment approach that is viewed as a morecomprehensive, accurate and fairer process thanthe single- source model.

REFERENCES

Ashford, S. J., & Cummings, L. L. (1983). Feed-back as an individual resource: Personalstrategies of creating information. Or-

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

66

ganization Behavior and Human Per-formance. 32, 370-398.

Bandura, A., & Carvone, D. (1983). Self-evaluation and self-efficacy mechanism gov-erning the motivational effects of goalsystem. Journal of Personality and So-cial Psychology. 45, (5), 1017-1025.

Baruch, Y., & Harel, G. (1993). Multi-sourceperformance appraisal: An empirical,and methodological note. PAQ, 17, (1),96-111.

Basnigth, T. A., & Benjamin W. W. (1977). Evalu-ating managerial performance: Is yourappraisal system legal? Employee Rela-tions Law Journal. 3, 240-54.

Bernardin, H. J. (1986). Subordinate appraisal: Avaluable source of information aboutmanagers. Human Resource Manage-ment, 25, 421-39.

Bernardin, H. J., & R. W. Beatty. (1984). Per-formance appraisal: Assessing humanbehavior at work. Boston: Kent.

Campbell, D. T., & Stanley, J. C. (1963). Experi-mental and Quasi Experimental Designfor research. Chicago: Rand- Mc Nally.

Chokar, J. S., & Wallin, J. A. (1984). Improvingsafety through applied behavioral

analysis. Journal of Safety Research. 15, (4), 141-151.

Cummings, L., & Shcwab, D. (1973). Perform-ance in organizations. Chicago: Scott,Foresman.

Deming, W. E (1986). Out of the crisis. Cam-bridge: MIT Center for Advanced Engi-neering Study.

Dobbins, G. H., Cardy, R. L., & Carson, K. P.(1991). Examining fundamental as-sumptions: A contrast of person and sys-

tem approaches to human resources man-agement. In G. Ferris, Ed. Research inpersonnel and human resource manage-ment. 4, 2-20.

Glueck, W. F. (1978). Personnel- A diagnosticapproach. Dallas, TX: Business Publi-cations, Inc.

Golembiewski, R. T. (1985). Humanizing publicorganizations. Mt. Airy, MD: Lomand.

Gunasekaran, A., Korukonda, A. R., Virtanen, I.,& Yli-Olli, P. (1994), Improving produc-tivity and quality in manufacturing or-ganizations. International Journal ofProduction Economics. 36, (2),169-183.

Henderson, J. D. (1989). Private sector manage-ment: Promoting efficiency, and cost-effectiveness. Corrections Today. 12,(October), 98-102.

Kaplan, R. E. (1993). 360-degree feedback plus:boosting the power of co-worker ratingsfor executives. Human Resource Man-agement, 32, (2), 299-314.

Knouse, S. B. (1996). Human resources manage-ment perspectives on TQM: Concepts apractices. Milwaukee: ASQC QualityPress.

Latham, G. P., & Kenneth, N. W. (1982, rev. 1994.)Increasing productivity through perform-ance appraisal. Reading, M.A: Addison-Wesley.

Latham, G. P., & Kenneth N. W. (1982, rev. 1994).Increasing productivity through perform-ance appraisal. Reading, MA: Addison-Wesley.

Lawler, E. E., III. (1967). The multirait-multitraiterapproach to measuring managerial jobperformance. Journal of Applied Psy-chology. 51, (2), 369-81.

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y

67

Lind, E. A., & Tyler, T. (1988). The social psy-chology of procedural justice. New York:Plenum.

Locke, E. A., Shaw, K. N., Saari, L. M., & LathamG. P. (1981). Goal setting in task

performance: 1969-1980. Psychological Bulletin.90, 125-152.

London, M. & Beatty, R.W. (1993). 360-degreefeedback as a competitive advantage.Human Resource Management. 32, 353-72.

Miller, C. D. (1991). Handbook on research de-sign and social measurement. NewburyPark, California: Saga Publication.

Mohrman, A. M., Jr., Susan A. M., & Edward E.L. III. (1987). The performance manage-ment of teams. CEO Publication. G91-2, 187.

Mohrman, A. M., Jr., Susan A. M., Gerald E. L.Jr., Thomas G. C., & Edward E. L. III,and Associates. (1988). Large-scale or-ganizational change. San Francisco:Jossey-Bass.

Nani, L. R. (Nov, 1987). Corporations as persons:Objections to goodpaster’s ‘principle ofmoral projection. Journal of BusinessEthics. 6, (8), 633-635.

Oakland, (1993). Total quality management: Theroute to improving performance, Oxford:Butterworth-Hainemann.

Petrick, J., & Quinn, J. (1997). Management eth-ics: integrity at work. Thousand Oaks,CA: Sage.

Sashkin, M., & Kiser, K. J. (1993). Total qualitymanagement. San Francisco: Berett-Koehler.

Sashkin, M., & Richard L. W. (1990). Does fair-ness make difference? OrganizationalDynamic, 1, 56-71.

Schneir, C. E. (1977). Multiple rater groups andperformance appraisal. Public PersonnelManagement, 6, (1), 13-2.

Scholtes, P. R. (1987). An elaboration onderming’s teachings on performanceappraisal. In Gary McLean, SusanDamme, & Richard Swanson, Eds.,Performance appraisal: perspectives ona quality management approach.Alexandria, VA: ASTD, and Univ. ofMinnesota, 24-52.

Sheppard, B. H., Lewicki, R. J., & Minton, J. W.(1992). Organizational justice –thesearch for workplace. New York:Lexington.

Solomon, R. L. (1949). An extension of controlgroup design. Psychology Bulletin, 46,137-150.

Stephen, G., & Weimerskirch, A. (1998). Totalquality management: strategies andtechniques proven at today’s most suc-cessful companies. 2nd New York: Wiley.

Tolchinsky, P. D., & Donald, C. K. (1980). Dogoals meditate the effects of incentiveson performance? Academy of Manage-ment Review. 5, 455-467.

Walton, M. (1990). The deming managementmethod. New York: Putnam

Malaysian Management Journal 8 (1), 55-67 (2004)

http

://m

mj.u

um.e

du.m

y