HEALTHECONOMICS, VOL. 5: 525-529 (1996)

ECONOMIC EVALUATION

BIASES IN COST MEASUREMENT FOR ECONOMIC EVALUATION STUDIES IN HEALTH CARE

PHILIP JACOBS' AND JEAN-FRANCOIS BALADI' 'Department of Public Health Sciences, University of Alberta, Edmonton, Alberta

2Canadian Coordinating Ofice for Health Technology Assessment, Ottawa, Ontario

SUMMARY

This paper addresses the issue of biases in cost measures which are used in economic evaluation studies. The basic measure of hospital costs which is used by most investigators is unit cost. Focusing on this measure, a set of criteria which the basic measures must fulfil in order to approximate the marginal cost (MC) of a service for the relevant product, in the representative site, was identified. Then four distinct biases-a scale bias, a case mix bias, a methods bias and a site selection bias-each of which reflects the divergence of the unit cost measure from the desired MC measure, were identified. Measures are proposed for several of these biases and it is suggested how they can be corrected.

KEY wows-Health care costs, cost effectiveness

Authors of several summaries of costing method- ology for economic evaluations in health care state that marginal cost (MC) is the appropriate cost concept to use for economic assessment studies.'-3 It is also asserted that, when conducting a study from a social viewpoint, all resources which are used in the intervention should be included in the analysis and valued at their opportunity It is implied that the marginal cost which is measured should be appropriate to the cases which are being studied. There is some question as to whether the cost measure should reflect resource use in those sites where the service is typically provided, or where it is most efficiently provided.

In areas where fees or charges are nof available as indicators or surrogates for costs, there are three widely used measures of hospital and other organizational costs: cost per day,5 cost per case mix weighted case or weighted day6 and case

specific costs of service^.^ No criteria currently exist to help us evaluate and compare these three (or any other) costing methods in the light of the concepts referred to in the previous paragraph. Based on these concepts, we have identified four biases which are relevant to costing. Reference to these biases can help in the selection of appropriate costing methods and also in adjusting cost statis- tics to reduce these biases,

GENERAL COSTING CRITERIA

The base statistic which is used as a reference point to measure margin$ cost is TC/Q ( = AC), where TC is total cost, Q is a quantity measure of a given output and AC is average cost. There are several assumptions or techniques which are used to interpret this measure as the desired marginal

Address for comspondence:Philip Jacobs, Department of Public Health Sciences, 13-103 Clinical Sciences Building, University of Alberta, Edmonton, Alberta T6G 2G3 CANADA.

CCC 1057-92301961060525-05 0 1996 by John Wiley & Sons, Ltd.

Received I I December 1995 Accepted 15 July 1996

526 P. JACOBS AND J-F BALADI

cost (MC) measure. First, we assume that AC=MC. This means that MC is constant, or MC/AC = 1. Second, we include in our measure of TC only those items which are variable. Currently economists' recommend that all direct operating costs, plus allocated overhead expenses and imputed costs for unpaid capital and other resources, be included in the calculation of TC. In this case, TC will be interpreted as (very) long run (variable) cost and average total cost (ATC), which is the same in the long run as AVC, equals MC. Third, the measure of Q can be very refined (e.g. workload units), or very rough (e.g. the number of laboratory tests). Fourth, the sites which are selected for costing can be a single hospital or ward, or all hospitals in a given region or country. Fifth, the costing can be conducted over all cases in the hospital or region, or for a select group of cases. Sixth, costing can be done on an actual basis or occasionally (called standard costing).

BIASES IN COSTING

Related to these techniques or assumptions are four biases which reflect the divergence of the actual AC measure from the ideal MC measure. A scale bias occurs when the department or facility is operating at a point where MC/AC does not equal 1, but where the investigator makes the assumption that MC/AC = l.9 This bias would be difficult to detect and is seldom addressed in evaluation studies. We will not address it further in this paper, but its existence is a distinct possibility.

A methods bias occurs when the measurement technique leads to costs which differ from long run variable cost. For one thing, if some costs are truly fixed (such that resources do not shift when ser- vices do), then we create a bias by treating these costs as variable. There are innumerable ways to allocate fixed costs to direct service departments and then on to specific products; all are arbitrary. Some overhead costs are in fact variable and it is necessary to attribute these to the appropriate cost centres; but the allocation of fixed costs is an arbitrary task and it is difficult to identify what a correct allocation is."

There are other potential causes of methods biases. Investigators may attribute inappropriate resource costs to various interventions. For example, one study which compared inpatient and outpatient surgery omitted certain fixed costs from

the outpatient unit because they were already in place; the authors concluded that such costs were sunk and therefore should be omitted." Such a practice would put one intervention, or the other, at a disadvantage, simply because of the specific circumstances in the hospital where the interven- tion was measured. For generalizability, all interventions should be costed under similar rules. Additionally, the cost which is included should be appropriate to the study's purpose. A study which takes a social (economic) viewpoint should include costs of all services from resources. Trans- fer payments, which do not stem from resource use, should be excluded. Biases can arise from the inappropriate inclusion or exclusion of 'costs. '

A case or service mix bias occurs when the measure of Q which is chosen is not refined and so the unit cost (TC/Q) for that service or case over- or underestimates the actual amount of resources used at the low and high cost end of the scale. This bias can occur at the departmental or patient level. For example, the costs of a laboratory test which has above average resource intensity will be underestimated if the measure of Q is all labora- tory tests. The bias can be eliminated by using a more refined measure of Q, that of workload units. An example of bias at the patient (global) level occurs when patient days are used as the measure of Q. Cases which are very resource intensive will have their costs understated. A more appropriate measure would be case mix weighted days.

Finally, there is a site selection bias. This type of bias might occur when costs are measured at a single site. A measure which is taken at one sight will misrepresent the marginal cost in the average site (or the most efficient site, if that is preferred as a standard). This bias can be adjusted for when one has data for all the sites; such data are now avail- able in many countries.

IDENTIFYING THE BIASES IN HOSPITAL COST MEASURES

In this section we identify the biases in three commonly used measures of hospital costs'*-a provincial per diem cost, a national or provincial Case Mix Group (CMG) or Diagnosis Related Group (DRG) per diem cost and costs calculated according to the Hotel and Ancillary (H&A) method. The first method, the provincial per diem, has been used in several studies in According to this method, cost is measured as the

BIASES IN COST MEASUREMENT 527 total cost, including all overhead expenses (except imputed capital and depreciation costs), at the provincial level. This measure has both a methods and a case mix bias. The methods bias occurs because total reported expenses exclude unpaid costs such as depreciation and the opportunity costs of capital and any other unpaid resources. The extent of the case mix bias will depend on which cases the investigator intended to measure. If the investigator chooses a regional or provincial average as the standard, then the provincial per diem method will reflect this.

The second measure is the cost per day for a given CMG. In Australia, a standard cost per weighted day was estimated for the entire country.6 This approach eliminates a site selection bias. A methods bias would have to be estimated on the basis of the costs that were actually included in the measure. The cost per weighted day measure is preferable to the straight per diem cost indicator because it is a more refined indicator. However, it is best used when we want to capture cost differ- ences which arise because of cases with different types of diagnoses. If we require more case detail and want to focus on differences between individ- ual cases within a CMG, then the use of the cost per weighted day method is not appropriate. When there is systematic variation in cases within a CMG, individual case costing methods should be used.

The third measure is the H&A method and its use in part eliminates the case mix bias. According to this method, unit costs (direct and overhead) for each service are calculated and summed over all of the services which are attributed to each case. The per case cost is then expressed as (;)(CPS) x Qi, where CPS is the cost per service and Qi is the number of service i which are used in each case. When the Q's are refined, the case mix bias in the H&A method will be minimal. However, when the Qs are crude, then case or service mix biases could result. For example, a flat per diem output measure is almost always used in conjunction with the H&A method for nursing services, which account for roughly 30% of all direct services in hospitals. A Canadian study of hospital costs which adjusted for nursing acuity indicated that per diem nursing costs for surgical patients was 8% above the average for nursing services and for medical patients it was 20% below the average. l4 The H&A method exaggerates nursing costs for medical patients and underestimates nursing costs for surgical patients.

The H&A method is subject to a site selection bias. Because of its computational complexity, the H&A method has only been applied at single sites. In Ontario the cost per weighted cases, measured with the standard method at individual teaching sites where the H&A method has been used, varies from 30% above the provincial average and 55% below the provincial average (source: Ontario Ministry of Health).

The H&A method can also be subject to a methods bias. Proponents of the H&A method usually allocate overhead costs according to one of several formulae: direct allocation, l5 step-down allocation" and simultaneous equations allo- cation.16 There is very little information available to compare the biases in these methods.

An issue which should be mentioned is the appropriate volume of output to use when deter- mining unit capital costs. These costs are fixed and will be sensitive to the volume indicator which is chosen. In many instances capital costs form a large component of the total cost of an intervention (e.g. lith~tripsy).'~ It would be useful to have a commonly agreed upon method for determining Q when costs are fixed; one candidate measure is quantity at full capacity output. Using such a volume for the denominator would ensure that a consistent measure was used across studies. We should point out that the use of alternative measures of volume do not lead to biases; there being no standard for volume in these cases, there is a range of legitimate choices which one can use.

ADJUSTING FOR BIASES

Biases can be eliminated directly or they can be accounted for in sensitivity analyses. If used in a sensitivity analysis, the focus on specific biases can provide an economic rationale (as opposed to a statistical rationale) for including alternative values for cost related variables.

The adjustment for a site selection bias requires that one first chooses a target site as the standard. The investigator can choose the average site or the most efficient site. We propose use of the average site, on grounds of practicality; it would certainly be more difficult and controversial to identify which is the most efficient site. One statistic which can be used to adjust for a site selection bias is (CMC, - CMC), where CMC, is hospital i's cost per CMG weighted case and CMC is the cost per weighted case for the standard hospital or hospital

528 P. JACOBS AND J-F BALADI

group. In Ontario, where the H&A method is widely used, data on CMC for each hospital are readily available.

The magnitude and direction of the case mix bias will depend on the costing method that is used. If a straight provincial per diem cost measure is chosen, then this statistic can be adjusted by (Ci - 0.2197) times the average cost per weighted case, where C, is the CMG per diem weight for the relevant type of case and 0.2197 is the average per diem weight for all cases. It should be noted that 1.oooO is the value of the average weighted case and the value 0.2197 will change every year and is unique to the Canadian database. As an example, the per diem CMG weight for an extensive gastrointestinal procedure (Case Mix Group 250, 1995/6 Resource Intensity Weight from Canadian Institute for Health Information) is 0.3236. The use of a provincial cost per day as an approxima- tion of the cost per day for this procedure would therefore understate hospital per diem costs by roughly 50%."

If the investigator wishes to capture resource use differences between cases within the same CMG or DRG, then neither the provincial per diem nor the per CMG per diem is the appropriate costing method to use and it would be difficult to deter- mine what the bias was when either of these measures was used. The H&A method would be the more appropriate indicator. As we mentioned above, the H&A method can have a service mix and a site selection bias and so if it is the chosen indicator, the investigator should make adjust- ments for bias.

A methods bias which results from excluded costs can be adjusted for by making an allowance for these costs. Adjustments for other types of methods biases, such as the those resulting from inappropriate overhead allocation techniques, will be difficult to detect.

CONCLUSION

We have identified four separate biases which can occur in a costing study: a scale bias, a case mix bias, a site selection bias and a methods bias. In this paper we focused on the last three; using these, we identified the specific biases which are present in three popular methods for costing hospital services: provincial per diem costs, pro- vincial or national case weighted costs and unit service costing. All three methods contain biases

which could be of considerable magnitude. Cur- rently, information is only available to adjust for some of these biases. We have suggested several such measures.

ACKNOWLEDGEMENTS

We thank Andre Lalonde of the Canadian Institute for Health Information for providing us with data.

REFERENCES

1. Donaldson, C. The state of the art of costing health care for economic evaluation. Community Health Studies 1990; 14: 41-356.

2. Dranove, D. Measuring costs. In: Sloan, F. (ed.) Valuing health care. London: Cambridge Univer- sity Press, 1995, pp. 61-75.

3. Tolley, K. and Gyldmark, M. The treatment and care costs of people with HIV infection or ADS. Health Policy 1993; 24: 55-70.

4. Johannesson, M. The concept of cost in the econ- omic evaluation of health care. International Journal of Technology Assessment in Health Care

5. Detsky, A. S. and Jeejeebhoy, K. N. Cost effective- ness of preoperative parenteral nutrition in patients undergoing major gastrointestinal surgery. Journal of Parenteral and Enteral Nutrition 1984; 8:

6. Australia Commonwealth Department of Health, Housing and Community Services. Manual of Resource items and their associated costs. Can- berra: Commonwealth Department, 1992.

7. Jaakimainen, L., Goodwin, P. J., Pater, J. et al. Counting the costs of chemotherapy in a National Cancer Institute of Canada randomized trial in non- small cell lung cancer. Journal of Clinical Oncol-

8. Drummond, M. F., Stoddart, G. L. and Torrance, G. W. Methods for the Economic evaluation of health care programs. Oxford: Oxford Medical Publications, 1987.

9. Lipscomb, J., Raskin, I. E. and Eichenholz. J. The use of marginal cost estimates in hospital cost containment policy. In: Zubkoff, M., Raskin, I. and Hanft, R. (eds) Hospital cost containment. New York: Prodist, 1978, pp. 514-538.

10. Horngren, C. T., Foster, G. and Datar, S. M. Cost accounting, 8th edn. Englewood Cliffs, NJ: Pren- tice Hall, 1994.

11. Pineault, R., Contandriopoulos, A.-P., Valois, M. et.al. Random clinical trial of one day surgery. Medicalcure 1985; 23: 171-182.

12. Jacobs, P. and Bachynsky, J. Costing methods in the Canadian literature on the economic evaluation

1994; 10: 675-682.

632-637.

ogy 1990; 8 : 1301-1309.

BIASES IN COST MEASUREMENT 529 of health care: a survey and assessment. Inter- national Journal for Technology Assessment in Health Care in press.

13. Baskett, T. F. and Parsons, M. L. Prevention of Rh(D) alloimmunization. Canadian Medical Association Journal 1990; 142 : 337 -339.

14. Jacobs, P., Hall, E., Henderson, I. et al. Episodic acute care costs: linking inpatient and home care. Canadian Journal of Public Health 1995; 86:

15. Gabriel, S., Jaakkamainen, R. L. and Bombardier, C. The cost effectiveness of misoprostal for nonster- oidal antiinflammatory drug-associated adverse gastrointestinal events. Arthritis and Rheumatism

200-205.

1993; 36: 488-459.

16. Churchill, D. N., Lemon, B. C. and Torrance, G. W. A cost effectiveness analysis of continuous ambulatory peritoneal dialysis and hospital dialysis. Medical Decision Making 1984; 4: 489-500.

17. Labelle, R. J., Churchill, D. N., Martin, S. E. etal. Economic evaluation of extracorporeal shock wave lithotripsy, percutaneous ultrasonic lithotripsy and standard surgical treatment of urolithiasis-a Canadian perspective. Clinical Investigative Medi- cine 1987; 10: 86-95.

18. Canadian Institute for Health Information. Resource intensity weights: Summary of methodology, 19951 96. Ottawa: Canadian Institute for Health Informa- tion, 1995.