Severity_and_Outcome

Severity and Adjusting Outcomes This discussion focuses on outcome (mortality) data collected by providers, and other data that are in or could be included in the patient's medical or administrative records, that could be used to adjust outcomes. The availability of such data and the integrity of their collection is the Achilles heel of such systems. Unless collection is audited routinely, the data cannot be relied upon. Not only could providers manipulate data, but their ability to provide accurate data depends on their competence, the very attribute the system is supposed to measure. Unless diagnoses were validated, comparisons among providers might be risky. For the near future, at least, validating diagnoses will likely mean examining individual patient records. If such information were available, a provider's cases could be divided into two groups: (1) those meeting criteria, for which mortality rates could be calculated, and (2) those not meeting diagnostic validation criteria, which is not the same as saying the patient did not have the problem. Some adjustment for patient factors would still have to be made, because different providers are likely to treat different percentages of patients with characteristics associated with mortality. Again, such factors are likely to have been drawn from the medical record, and the adequacy of medical records varies. The use of severity scores for adjusting outcomes assumes their validity. Treatment difficulty scores are not useful for adjusting outcomes, because they are predictions of what one is measuring. A valid treatment difficulty score (were it to exist) could be compared to an observed score, at discharge for example, to assess provider performance. The treatment difficulty score represents a prediction of posttreatment outcomes; the observed score represents actual outcome. The approach is another form of population-based quality assessment. Perhaps a valid approach to mortality data could be developed that does not depend on diagnosis, or obtaining medical record data. However, given the complexity of the problem, it may take some time to develop. Moreover, a parallel data collection system would have to be developed, if one were not to rely on the medical record. Experience to date is not too encouraging. For example, the model used by Shortell and Hughes to examine inhospital mortality rates for selected diagnoses in states with and without stringent rate and certification-of-need controls explained only 10 percent of the variance.(1) An excellent model, if one could be devised, would explain at least 90 percent of the variance. A mortality adjustment system could be validated as follows. A small number of diagnoses could be selected for test purposes. The cases would be taken from a random sample of institutions. The providers' mortality experience would be calculated with and without adjustment and rank-ordered, from best (lowest mortality) to worst. Simultaneously, individual medical records would be reviewed independently by panels of expert clinicians to determine if they met practice standards. Each clinician would follow a structured review protocol, and any disagreements among reviewers would be discussed and resolved. The proposition of providers' cases meeting practice standards would be calculated, and providers would be rank ordered accordingly. The panels' judgments would be considered definitive. If the adjustment method were valid, there would be a perfect correlation between adjusted mortality rates and levels of acceptable care, both for the sample as a whole and for each of its constituent diagnoses. A less acceptable, but nonetheless good, result would be a perfect correlation among providers' rank order using the two measures. A lesser correlation coefficient could be accepted, but the lower limit of acceptability would have to be set prior to the analysis to prevent the result from influencing the standard. One consideration makes population-based approaches to assessing provider performance inherently flawed. To be truly valid, any system would have to adjust not only for technical factors that influence patient outcomes but also for patients' choices. If the patient's choice of therapy influenced the outcome, e.g., mortality, and varied from that which would produce the lowest mortality rate, resultant data would have to be adjusted to permit valid comparisons. For example, assume that there are two treatments for a hypothetical stage II of a health problem. One is simple and expedient, the other disfiguring, but the mortality rate for the first is twice that of the second. If one provider's patients freely choose the simple therapy more often and another's the disfiguring one because of the provider's advice, the first provider's performance (even if adjusted for technical factors) would appear worse than the second's, even though by permiting the patient to choose more freely, the provider could be considered to be delivering better quality care. References [1]Shortell, S., and Hughes, E. "The Effects of Regulation, Competition, and Ownership on Mortality Rates among Hospital Inpatients." New England Journal of Medicine 318(17):1100- 7, April 28, 1988. [2]Jencks, S., and others. "Evaluating and Improving the Measurement of Hospital Case-Mix." Health Care Financing Review, pp. 1-11, Nov. 1984, Supplement.