Abstract
International data on quality of medical care allow countries to compare their performance to that of other countries. The Commonwealth Fund International Working Group on Quality Indicators collected data on twenty-one indicators that reflect medical care in Australia, Canada, New Zealand, England, and the United States. The indicators include five-year cancer relative survival rates, thirty-day case-fatality rates after acute myocardial infarction and stroke, breast cancer screening rates, and asthma mortality rates. No country scores consistently the best or worst overall. Each country has at least one area of care where it could learn from international experiences and one area where its experiences could teach others.
Most industrialized countries share an interest in measuring, reporting, and improving the quality of medical care. Despite this interest, there have been limited internationally comparable data available on quality indicators, especially in areas involving medical care interventions. Collaboration between countries to produce internationally comparable data permits benchmarking and allows policymakers and clinicians to identify specific areas where individual countries could improve.
In the United States, sparked by Institute of Medicine (IOM) reports focusing attention on gaps in the quality of medical care, interest in improving quality has expanded rapidly among policymakers, corporations, clinicians, the media, and the public.1 Despite this concern about the quality of care, U.S. policymakers and clinicians often recite the mantra, "Americans have the best medical care in the world."2 The empirical basis for this statement is unclear. The limited empirical international data on quality that exist—life expectancy and infant mortality statistics—place the United States in the bottom quartile of industrialized countries, although most observers do not attribute this poor performance primarily to the performance of the medical care system.3
This paper presents data collected for twenty-one quality indicators in five countries. Our intent is to draw attention to potential opportunities to improve medical care in the five countries; raise questions about why some countries do well on some measures and others do poorly; provoke debate within countries about health care priorities and policies; and stimulate efforts to examine, refine, improve, and collect additional data.
Methodology
In 1999 the Commonwealth Fund convened a working group of quality measurement experts from governments in Australia, Canada, New Zealand, England, and the United States, along with academic researchers and representatives of institutions involved in medical care quality measurement.4This group examined a variety of working definitions of quality, ultimately choosing one developed by the IOM: "the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge."5
The next step was to choose among the various measurement domains that give structure to the quality reporting effort. The five countries have each created similar measurement frameworks. Canada’s framework was adopted to guide the data collection exercise mainly because of its comprehensiveness. The working group focused on developing indicators of the appropriateness and effectiveness of care—the extent to which care is delivered in accordance with established standards and achieves its desired results. Other quality measurement domains (such as equity and responsiveness) were left for future work, because of measurement difficulties (for example, continuity and safety) or because they are the subject of other measurement activities (for example, efficiency and access).
The working group then identified indicators for collection, starting with lists of potential indicators reflecting each domain of health system performance. Indicators were evaluated using the following criteria. (1) Feasibility: Only indicators that were already being collected by one or more countries were candidates. (2) Scientific soundness: Only indicators that were deemed valid and reliable were considered. Since all of the indicators considered were already in use, determination of scientific soundness relied on existing reviews of the scientific evidence and approval by a consensus process or similar method in one or more countries. (3) Interpretability: Only indicators that allowed a clear conclusion for policy-makers were included. This meant that the indication had to have a clear direction (higher is either good or bad). (4) Actionability: Only measures of processes or outcomes of care that could be directly affected by health care policy or health care delivery system intervention were eligible. (5) Importance: Only indicators that reflected important health conditions accounting for a major share of the burden of disease, the cost of care, or policymakers’ priorities such as vulnerable populations were pursued.
These criteria were applied in a five-step process. First, all indicators currently available in at least one country—an initial set of more than 1,000 indicators—were assembled. The fifty most promising indicators were then selected based on the five criteria above. We then further assessed these indicators by collecting information on definition, numerator and denominator specifications, the population represented, periodicity of collection, and data sources for each country. Indicators with irreconcilable differences in specifications or that were not nationally representative in several countries were discarded. For the remaining thirty-five indicators, we applied an iterative process of collecting data in the five countries, evaluating the comparability of the specifications, and making adjustments, such as revising coding classifications or age standardization. Finally, we compared the face validity of preliminary data and investigated any unusual differences to increase the reliability of the indicators. We also reviewed the final data with experts in each country.
There are numerous reasons to explain why this specific list of twenty-one indicators was selected and many others were not. Many potential indicators would require a review of medical records, which would be very costly without routine access to electronic medical records, and medical record-keeping practices vary considerably across countries. Some indicators were deemed difficult to interpret.6 Others were eliminated because of a relative lack of importance.7 Among the available indicators meeting all other criteria, several were deemed not to be internationally comparable.8
Results
Data for all twenty-one indicators are summarized in Exhibit 1
. The results are standardized so that indicators with different measurement units can be compared. In Exhibit 1 the country with the worst result for an indicator is given a score of 100. All other countries are given scores relative to the country with the worst result. The scales are structured so that higher scores always indicate better quality. For example, the breast cancer survival rate is 14 percent better (higher) in the United States than in England, and the suicide rate is 55 percent better (lower) in England than in New Zealand. The actual value of the indicator for the country with a score of 100 is given in the right-hand column, so that any country’s actual value can be calculated from its score.
None of the five countries consistently scores the best or worst on all of the indicators. In addition, each country has either the best or the worst score on at least one indicator. In other words, no country scores consistently the best or worst overall, and each country has at least one area of care where it could learn from international experience. Each country also has an area where it could teach others.
The results in Exhibit 1
are intended to stimulate additional inquiry by policy-makers and clinicians in each country. There are many reasons why a country could score well or poorly on a particular indicator. We have grouped the twenty-one indicators into outcome indicators (survival rates and avoidable events) and process indicators for presentation purposes.
Outcome indicators: survival rates. The first five indicators are five-year relative survival rates for various types of cancer.9 The relative survival rate is the ratio of the number of cancer survivors to the number of people of that age and sex in the same country who would have been expected to be alive after five years if they did not have cancer. It measures the additional deaths attributable to cancer, controlling for differences in underlying mortality patterns between countries.
On these indicators, the range in performance was usually small. On most survival rates, the countries are within 10 percent of each other. One pattern that does stand out is that England is consistently at the low end of the distribution for cancer survival. This is consistent with previous comparisons of cancer survival between the United Kingdom and other European countries.10
Higher cancer survival rates are unquestionably a desired health care goal. Primary care, including health promotion and screening, can make a difference in the stage of diagnosis for the cancers studied, particularly cervical, breast, and colorectal cancer.11 Secondary and tertiary cancer care can also make a difference. Other factors such as financial barriers to care, waiting lists, and reluctance to seek care could also influence rates.
Two related indicators of the outcomes of health care are the survival rate following a kidney or liver transplant.12 The survival rates for both were relatively low in the United States. Differences in the characteristics of patients receiving transplants could influence survival rates. Assuming that transplant recipients in the five countries are similar, the remaining differences are more likely to be attributable to differences in medical care.
Data on acute myocardial infarction (AMI) and ischemic stroke are also presented, although comparable data are available in only three of the five countries.13 AMI case-fatality rates are highest in Canada and lowest in Australia (Exhibit 2). The higher case-fatality rate among older people in Canada is an area that war rants investigation. Exhibit 3 shows only small differences in the case-fatality rate for ischemic stroke. Noticeable differences are seen only in the 75–84 age group. In addition to medical care received, these rates could be affected by factors including the average severity of AMI and ischemic stroke in the three countries; the rate at which emergency services transport people to the hospital; and hospital discharge, admission, and length-of-stay characteristics.14
Outcome indicators: avoidable events. The second group of outcome indicators shows the rates of certain health outcomes that are considered avoidable had appropriate care been delivered. These indicators include suicide rates, the incidence of vaccine-preventable diseases, asthma mortality rates, and smoking rates.
Suicide rates were almost equal in Australia, Canada, New Zealand, and the United States but lower in England.15 Suicide rates among two groups of younger people show bigger differences.16 New Zealand had much higher suicide rates among young people than the other four countries; suicide is an area that is already receiving attention in New Zealand.
The incidence rates of three vaccine-preventable diseases—pertussis, measles, and Hepatitis B—show that some countries have these diseases under better control than others do.17 Pertussis incidence was particularly high in Australia and Canada; measles incidence was higher in England than elsewhere; and Hepatitis B incidence was highest in the United States and Canada.
Another appropriateness indicator reflecting an avoidable outcome is the asthma mortality rate for people ages 5–39, the ages at which asthma is most reliably diagnosed (Exhibit 4).18 Deaths resulting from asthma are considered preventable if the condition is managed appropriately.19 New Zealand’s asthma mortality rate was much higher than that of England and Australia in the early 1980s. Since then, it has declined markedly to a level close to those of the other countries, as clinicians discontinued use of fenoterol, an adrenergic bronchodilator, and began using inhaled corticosteroids.20 Nevertheless, the asthma mortality rate in New Zealand remains higher than in the other countries. The rate in England and Australia has also declined over time, reflecting improvements in asthma care. The United States is the only country where the asthma mortality rate has been increasing recently. In 1990 asthma mortality was lowest in the United States, but by 2000 it was higher than in Australia and England and approaching the rate in New Zealand. The reason behind this increasing U.S. trend is an important area for investigation.
Smoking rates (as percentage of the population) were lowest in the United States and Canada (Exhibit 1).21 The health care system does not have perfect control over people’s decisions to smoke, but advice and treatment provided by physicians have been shown to have an impact on smoking cessation.22
Process indicators. The five countries were similar in performance on several process indicators of appropriate care delivery and widely different on others. The differences in mammography rates between countries were relatively small.23 However, sizable differences were seen between countries in the cervical cancer screening rate for the population for whom screening is indicated.24Cervical cancer screening was much more common in the United States than elsewhere.25
Influenza vaccination rates show that all five countries could prevent more influenza-related deaths among older people through vaccination.26 New Zealand, in particular, might investigate how it could increase its rate to the level of other countries. Polio vaccination rates were above 90 percent in Australia, England, and the United States and above 80 percent in Canada and New Zealand.27 Although polio has recently been absent in these countries, these low vaccination rates could allow it to recur, particularly in Canada and New Zealand.28
If one considers cost-effectiveness, it is hard to identify the clear ideal level for these process indicators; higher rates are better, but at a certain point the marginal returns are likely to be small. In the absence of such an ideal level, it is useful for countries to benchmark their rate against those in other countries. These comparisons show that countries deliver these health care interventions at generally similar rates, although opportunities exist for countries to raise their level to that of the best-performing country.
Summary And Potential For Improvement
The comparisons on this initial set of quality indicators show that each country performs well in some areas and poorly in others compared with other countries. Each country could improve the quality of care.
Australia performed well on many of the indicators. In particular, cancer survival rates were generally high (excepting childhood leukemia); breast cancer screening rates were high; asthma mortality was relatively low; and influenza and polio vaccination rates were high. However, the incidence of pertussis was much higher than elsewhere, suggesting an opportunity for improvement.
In Canada, cancer survival rates were generally average or above average and were highest for childhood leukemia. Stroke case-fatality rates were relatively low. Transplant survival was also relatively high in Canada. However, AMI case fatality was higher in Canada than Australia and New Zealand in older age groups. This confirms previous findings and deserves further investigation.29Pertussis incidence was much higher than in the other countries (except for Australia).
Suicide rates were notably lower in England than in the other four countries. The polio vaccination rate there was the highest. However, cancer survival rates were lowest in England, as were breast and cervical cancer screening rates. This confirms previous European comparisons and suggests an opportunity for improvement. Measles incidence was also higher in England than elsewhere.
In New Zealand, the improvement in asthma mortality over the past twenty years is a true success story, although there may be room for further improvement. The colorectal cancer relative survival rate was highest. However, the suicide rate in New Zealand, particularly among younger people, was much higher than elsewhere. Stroke case-fatality rates were higher among older age groups. Breast cancer screening and influenza and polio vaccination rates were relatively low.
In the United States, breast cancer survival rates were higher than in the other countries. Cervical cancer screening rates were very high. One area for concern is that asthma mortality rates were increasing in the United States but decreasing in the other countries. Transplant survival rates were also relatively low in the United States.
While the United States often performs relatively well for this set of indicators, it is difficult to conclude that it is getting good value for its medical care dollar from these data. The huge difference in the amount the United States spends on health care compared with the other countries could very well be justified if the extra money provided extra benefits. Population surveys have shown that the extra spending is probably not buying better experiences with the health care system, with the exception of shorter waits for nonurgent surgery.30 Earlier studies have shown the United States to be in the bottom quartile of population health indicators such as life expectancy and infant mortality.31 Our results also fail to reveal what the extra spending has bought, although there are many important places to look.
The limitations of this indicator list preclude the definite conclusion that any country has the best quality of care. It should be emphasized that this initial set of twenty-one quality indicators was distilled from a starting compendium of more than 1,000. It is an opportunistic list, rather than a comprehensive list. Some indicators relate to health conditions that account for a large share of the burden of disease in these countries, while others (such as transplant survival rates) have smaller implications for population health. Some conditions that represent a large share of the disease burden, such as diabetes, are not represented at all. More work is clearly needed to expand the scope and depth of the indicator set so that it can be used to judge overall health system performance, and further investment in data collection and international harmonization of indicators to allow valid international comparisons are necessary.
During the time frame of this project, which began in 1999, major improvements in quality measurement capabilities have been made in many countries, which indicates the potential for improvement. Most importantly, building on the Commonwealth Fund’s International Working Group on Quality Indicators and a similar effort undertaken by a group of five Scandinavian countries, the Organization for Economic Cooperation and Development (OECD) has undertaken an initiative to move this work forward, expand the number of countries involved, develop additional quality indicators, and institutionalize the collection of these indicators. We hope that these twenty-one indicators will be a first installment in ongoing efforts to conduct international quality comparisons.
No comments:
Post a Comment