Methodology in coronary artery bypass surgery quality assessment

Zhongmin Li, Nicole Shirakawa, Amy Chen, Fu-Hai Ji, Hong Liu,

1. Department of Anesthesiology and Pain Medicine, University of California Davis Health, Sacramento, USA; 2. Department of Anesthesiology, First Affiliated Hospital of Soochow University, Suzhou, China

ABSTRACT Coronary artery bypass graft (CABG) is associated with a high risk of mortality and morbidity; thus, assessment of surgery quality is necessary. In this perspective, we will focus on the structure, process, and outcomes measured as quality assessment. A set of 21 evidence-based structure, process, and outcome measures were selected as National Quality Forum. Of these, the Society of Thoracic Surgeons ultimately chose 11 individual quality measures grouped them into four domains used to assess the quality of CABGs. These four domains consisted of perioperative medical care, operative care, risk-adjusted operative mortality and postoperative risk-adjusted major morbidity. These measures have been useful as quality improvement tools in assessing the quality of CABG surgery.

Coronary artery disease (CAD) is the leading cause of morbidity and mortality worldwide and coronary artery bypass graft(CABG) is the most common cardiac surgical procedure performed and it is among one of the costliest surgeries, especially in aged population.[1-3]This procedure generally has favorable outcomes but may be associated with a high risk of mortality and morbidity.[4]Therefore, assessment of the quality of surgery is of great importance to both patients and health care providers. In the United States, public reporting of CABG surgery outcomes has been found to have a profound impact on transparency and quality improvement.[5,6]Since the initiation of public performance reporting (PPR), beginning in the early 1980s, outcomes of CABG surgeries have allowed for surgeons and hospitals to focus on quality improvement. Existing data have shown an association with improved mortality rates of CABG procedures and quality improvement measures by hospitals and surgeons over the years,[7,8]however, not all states in the United States and other parts of the world participate in PPR. This review is to exam a set of 21 evidence-based structure, process, and outcome measures and the importance in CABG outcomes.

Although conceptually, the term “quality” is an abstract construct that cannot be directly measured.Therefore, people may rely upon measurable surrogates to quantify the concept of “quality”. Those surrogates used to be related to the results that we would expect from excellent health care.[9]In 1966, Donabedian introduced the concepts of structure, process and outcome for measuring health care quality.[10]These concepts remain to be the dominant paradigm to evaluate the quality of health care to date. Thoracic surgery is not excluded from this concept and thus quality outcomes may be measured using this idea.

Structure refers to inherent characteristics of health care providers that are believed to be associated with higher quality. These characteristics may include material resources (modernization of facility and equipment, information and surgery technology, etc), human resources (such as nurse staffing ratio, surgeon qualifications and experience) and organizational characteristics (size, surgery volume, participation in an outcomes database registry, etc). Although structural measures may be useful when specific outcomes data are unavailable, the strength of the relationship between structural measures and desired surgical outcomes is not well defined. Furthermore,structural measures are often not readily actionable by health care providers, thus this diminishes their usefulness as quality improvement tools. Because the development in technology, surgical techniques and equipment used in surgical coronary interventions, these measures be re-examined and updated based on scientific evidence. For example, some studies suggested that the cardiac surgery volume affected the surgical outcomes[11,12]and other studies did not support it.[13,14]The relationship of observed/expected ratio of operative mortality and CABG surgery volume represented stability of year-to-year outcomes. However, the center-level, risk-adjusted CABG mortality varies significantly from one year to the next.[14]Surgical technique, cardiac anesthesia specialty training, quality measures, centers for excellence, and the advancement of cardiopulmonary bypass technology contribute to the overall low mortality rate despite an older, sicker population.[15]Other keys to improve outcomes included surgeon leadership and engagement, regularly sharing unblinded data, development of standardized quality improvement processes, improvement and standardization of care delivery, target setting for quality improvement, and a shared vision for improved patient outcomes.[16]

Process measures reflect the extent to which a provider complies with evidence-based care guidelines.These measures cover the activities that constitute health care such as screening and diagnoses, treatment and rehabilitation, education and prevention.Generally, assessing the processes of care provides a more immediate path to improvement in patient care since it involves measurement of the care patients actually receive. If diagnostic and therapeutic strategies with clear links to outcomes are monitored,some healthcare quality problems can be detected long before demonstrable health outcome differences occur. In the acute care setting, appropriate process measures might include the administration of aspirin and β-blockade to reduce mortality in acute coronary syndromes, “door to balloon” time for acute myocardial infarction, and the internal mammary artery(IMA) usage for eligible patients undergoing CABG.Process measures are particularly useful for procedures and medical conditions in which outcome measures are unavailable or impractical.[17-21]They have the advantage of being actionable by providers, and in the United States, they are the dominant quality metric utilized in many pay-for performance initiatives.However, when payment depends on compliance of particular process measures, providers may focus on maximizing their performance in these specific areas. In some cases, this may be a perverse incentive, leading to tests or interventions that are unnecessary, costly, or inconsistent with the patient’s wishes.[22,23]Ideally, process measure compliance should reflect the number of eligible patients who received the treatment or therapy. In practice, agreement about what constitutes a legitimate exclusion has often been difficult even for expert panels, and mechanisms to document such exclusions are both problematic and susceptible to gaming.

Outcomes are the most obvious and intuitive indicators used to measure quality. These measures include mortality, complications, readmission, functional status and patient satisfaction. These measures are generally the most relevant and important to patients themselves. In the United States, the first attempt to quantify health care outcomes on a large scale was at near the end of the 20thcentury, the Health Care Financing Administration (HCFA), now known as The Centers for Medicare & Medicaid Services (CMS),publicly released hospital mortality data for all institutions that received payment from HCFA. However, because of serious deficiencies with the program, including lack of risk adjustment and the use of an administrative database not designed for outcomes analysis, the program was abandoned shortly after its inception.

In recognition of its important public health implications, the National Quality Forum (NQF), a private, not-for-profit organization established in 1999,recently convened a Cardiac Surgery Performance Measures Steering Committee and associated Technical Advisory Panel, both of which included representatives of the Society of Thoracic Surgeons (STS),payers, regulators, health policy experts, and consumers. Their charge was to identify and endorse a set of evidence-based measures that would accurately reflect the performance of cardiac surgery programs. Ultimately, a set of 21 structure, process, and outcomes measures (Table 1) were selected.[24]

Because all of those outcome measures are consistent with the Institute of Medicine (IOM) goals for health care (safe, effective, patient-centered, timely,efficient, and equitable),[25]these measures have become the set of nationally recognized parameters for assessing quality in cardiac surgery in the United States. Based on those 21 relevant CABG process and outcomes measures currently endorsed by the NQF,STS Quality Measurement Task Force (QMTF) ultimately chose 11 individual quality measures and grouped them into four domains (Table 2).[26]The measures further selected by STS were mainly due to data availability in STS National Adult Cardiac Surgery Database (NCD).

Table 1 Structures, process, and outcomes measures of cardiac surgery performance measures.

Table 2 The Society of Thoracic Surgeon quality measures.

Table 3 Multivariate logistic regression risk model for operative mortality, 2007, California.

PERIOPERATIVE MEDICAL CARE

The use of preoperative β-blockade as a quality measure stems from its cardioprotective effects from the physiologic stress of major surgery, and from its antiischemic effects in patients with severe coronary artery disease (CAD). However, there are important considerations in choosing to use this process measure as an indicator of quality.

Firstly, β-blockade has important clinical contraindications that may preclude their use. Amongst the most common contra-indications are hemodynamic instability (e.g., shock), severely decompensated heart failure, significant bradycardia, and severe reactive airway disease. Second, the limits which define when β-blockers should be withheld are highly patient and physician dependent. For example, in a patient with severe systolic heart failure and a heart rate of 110 beats/min, β-blockers may be dangerous even in the presence of normal blood pressures and relative tachycardia. On the other hand, β-blockers may be withheld in patients with heart rates of 55 beats/min simply because of physician preference.

The use of discharge β-blockade, aspirin and lipidlowering therapy are also process measures recommended by the STS QMTF. These measures have been extensively studied and are recognized in the 2006 American College of Cardiology (ACC)/American Heart Association (AHA) Guidelines for Secondary Prevention for Patients with Coronary and Other Atherosclerotic Vascular Disease.[27]However, these measures are not static, they should be modified according to the emerge evidence.

OPERATIVE CARE: INTERNAL MAMMARY ARTERY

In most cases of first-time isolated CABG surgery where the operative status is either elective or urgent and the left anterior descending (LAD) was bypassed, the surgeon has the option of using the IMA, also known as the internal thoracic artery. Clinical literature strongly supports use of the IMA to promote long-term graft patency and patient survival, and recent research also suggests a reduction in immediate, operative mortality associated with use of the internal mammary artery as opposed to saphenous vein revascularization. However, the target coronary artery to which the IMA is grafted to is also an important technical consideration. In general, the IMA is grafted to the LAD artery, or a dominant diagonal branch of the LAD. This is because the LAD is considered the most important coronary artery after the left main coronary artery as it supplies a large portion of the left ventricle, including the anterior wall and septum. Furthermore, grafting the IMA to the circumflex may be technically more difficult due to the course of the circumflex, the distance of the IMA from the circumflex with resultant stretching of the IMA pedicle, and the usually smaller size of the circumflex and its marginal branches.Therefore, in reporting the use of the IMA, it is equally important to report on the target recipient coronary artery. For example, the use of an IMA graft for the circumflex artery would not be considered optimal surgical technique in the context of large-scale public reporting. For similar reasons, the IMA is not usually grafted to the right coronary artery (RCA).

In the United States, many nationally respected organizations encourage the use of IMA when appropriate. Currently, the Leapfrog Evidence-Based Hospital Referral program endorses 80% hospital adherence to IMA use. The National Quality Forum(NQF) does not endorse a specific rate but states that the goal is to raise the IMA usage rates of hospitals with low utilization. The Society of Thoracic Surgeons (STS) states that IMA use should be given primary consideration in every CABG surgery patient.Furthermore, a number of healthcare quality advocates recommend public reporting of IMA usage rates for CABG surgery.

OPERATIVE MORTALITY

The mortality rate is the most widely used indicator for measuring the quality of cardiac surgery by hospitals and surgeons. Mortality is used as a measure because it is severe and unambiguous. Differing from the in-hospital mortality that only counts death up until discharge, the operative mortality also counts deaths that occur anywhere after hospital discharge but within 30 days of the CABG. Use of operative mortality as the outcome measure, instead of in-hospital mortality, avoids potential manipulation of outcomes through discharge practices and holds hospitals or surgeons performing CABG surgeries accountable for patients who died at home shortly after discharge or were transferred and died at other facilities. However, some researchers have suggested that the 30-day rule should apply to both patients who are discharged within 30 days of surgery and patients who suffer in-hospital death, since those who have prolonged community hospital admissions may succumb to reasons unrelated to the CABG. In the United States, there are five states that publicly report CABG outcomes using clinical data.Of these, New York, New Jersey and California have adopted operative mortality as the key outcome measure, while Massachusetts measures 30-day operative mortality. Pennsylvania however, uses operative mortality, in-hospital mortality and 30-day mortality as their outcome measures to report to the public.[28-32]

POSTOPERATIVE MORBIDITY

Among the postoperative morbidity outcomes recommended by STS, renal insufficiency refers to acute or worsened renal failure resulting in one or more of the following: (1) increase of serum creatinine to > 2.0 and 2 × most recent preoperative creatinine level and/or (2) a new requirement of dialysis postoperatively. Deep sternal wound infection refers to whether patients within 30 days postoperatively develop a deep sternal infection involving muscle,bone, and/or mediastinum that requires operative intervention. It must have all of the following conditions: (1) wound opened with excision of tissue or reexploration of mediastinum; (2) positive culture; and(3) treatment with antibiotics. The re-exploration for any cause includes: (1) reoperation for bleed/tamponade: indicating whether the patient returned to the operating room for mediastinal bleeding /tamponade; and/or (2) reoperation for graft occlusion: indicating whether an operative re-intervention was required for graft occlusion due to acute closure, thrombosis, technical or embolic origin. Stroke refers to whether the patient suffered a postoperative stroke (i.e., any confirmed neurological deficit of abrupt onset caused by a disturbance in cerebral blood supply) that did not resolve within 24 h. Finally, prolonged ventilation refers to whether the patient required prolonged pulmonary ventilation > 24 h postoperatively.

Because of the quantity of potential outcome measures for postoperative complications, researchers have suggested creating a composite measure. However, the weighting scale for each measure in the development of a composite measure is subjective,and could be biased; an all-or-none approach has therefore been proposed to create a composite measure.[33]

Risk Adjustment

Regardless of the type of outcome measures utilized for measuring surgical quality, risk-adjustment is the key measure to make fair comparisons of CABG outcomes among different healthcare providers. Riskadjustment is a statistical process where the selected outcome measures are adjusted to account for variation in the preoperative health condition of patients. Since mortality or morbidity is a binary variable(yes/no), most researchers use a multivariate logistic regression models to determine the relationship between each of the demographic and preoperative clinical risk factors with the probability of mortality/morbidity, and to compute the predicted mortality/morbidity for each patient. Each patient’s predicted mortality/morbidity would be summed up by the hospital or surgeon as the provider’s expected mortality/morbidity. First, the risk-adjusted mortality/morbidity by the provider is computed by dividing the observed mortality/morbidity rate by the provider’s expected mortality/morbidity rate to obtain an observed/expected (O/E) ratio.[16]If the O/E ratio is close to 1, the performance is judged as expected, or as better if the O/E ratio < 1, or worse if the O/E ratio is larger than 1. The O/E ratio is then multiplied by the overall population mortality/morbidity rate to obtain the provider’s risk-adjusted mortality/morbidity rate. The risk-adjusted rate represents the best estimate, based on the risk model, of what the provider’s mortality/morbidity rate would have been if the provider had a patient case mix identical to the overall patient population mix. Thus,this rate is comparable among providers since the differences in the severity of illness amongst patients have been accounted for. To prevent a misinterpretation of differences caused by chance variation, most public reporting programs in the United States identify quality outliers by using a 95% confidence interval (CI) of the risk-adjusted mortality/morbidity rate instead of point estimates. The performance rating of a provider is based on a comparison of the 95% CI of each provider’s risk-adjusted rate to the population rate. If the entire 95% CI of a provider’s risk-adjusted rate is below the population rate, indicating the provider’s rate is significantly lower than the population rate, the performance rating will be “Better”; if the entire 95% CI of a provider’s risk-adjusted rate is above the population rate, indicating the provider’s rate is significantly higher than the population rate, the performance rating will be “Worse”; and if the population rate is within the 95% CI of a provider’s risk-adjusted rate, the performance rating will be “As Expected”.

To develop a multivariate logistic regression model for a binary outcome measure or a multivariate linear regression model for a continuous outcome measure (e.g., length of stay), researchers often split the source data into two parts: about 50% of records are used for model development and the other 50%are used for model test or verification. The model usually is considered valid when the validation data proved that the model had good predictive power(i.e., discrimination) and goodness-of-fit (i.e., data calibration). Table 3 and 4 present California’s riskadjustment models for operative mortality and pos-toperative stroke. The models had C-statistic of 0.82 and 0.72 respectively for discrimination, indicating the models distinguish well between patients who have an adverse event and those who do not. The models also hadP-value of 0.147 and 0.152 for data calibration tests respectively, indicating that the predicted number of adverse events were consistent with actual number of adverse events in the data.

Table 4 Multivariate logistic regression risk model for postoperative stroke, 2006-2007, California.

Continued

The risk-adjustment model is not only significant for measuring surgery outcomes. It is equally important for appropriately grouping patients into different cohorts before risk-adjustment. Patients undergoing isolated CABG surgery are usually at much lower risk of adverse events compared to those who undergo combined CABG and valve repair/replacement. Thus, the risk-adjustment model should be developed and validated for each cohort separately.

CONCLUSION

In summary, CABG is still the most expensive and common cardiac surgery performed today despite the presence of percutaneous coronary intervention. Since California launched the CABG Outcomes Reporting Program in 2003 for all nonfederal hospitals performing this surgery, it has made a profound impact on transparency and quality improvement by using evidence-based measures (structures, processes, and outcome measures) that accurately reflect the performance of cardiac surgery programs. Looking ahead, implementing these measures along with a mandatory reporting programs for CABG and all the cardiac surgeries throughout the nation will likely lead to improved outcomes of cardiac surgery and will allow patients to make informed decisions regarding their care.

FUNDINGS

This work was supported by the Departments of Anesthesiology and Pain Medicine of University of California Davis Health and NIH grant UL1 TR00 1860 of the University of California Davis Health.