ABSTRACT

The empirical literature on hospital report cards typically assumes that report card cannot interact with consumer’s private learning. This study examines the impact of the implementation of FL hospital quality reporting system in late 2004 on hospital admission patterns using a pre-post difference-in-difference design. The estimation model allows for the possibility that report-card learning may interact with non-report-card learning. The study sample is comprised of all patients admitted to any FL hospital between 2000 and 2008 with a principal diagnosis of acute myocardial infarction (AMI). We find that hospital admission patterns for AMI patients did not respond to report card information. However, we find evidence consistent with the possibility that the implementation of a report card system may stimulate consumers (either patients or physicians) to seek higher quality hospitals through private information channels.

1. INTRODUCTION

For more than two decades, there have been interests to provide comparative provider quality information to consumers to aid consumer choices based on quality of care. To date, the US federal government, more than 30 states, and many private organizations have initiated some forms of report cards that publicly disclose selected financial, process, volume, and outcomes data on health plans, hospitals, and physicians. In principle, report cards can improve market performance in two ways. First, report cards can generate patient (demand) responses, by bringing critical information to patients to identify and switch to high quality providers. Assuming that patients would “vote with their feet,” providers are incentivized to improve quality in order to subsequently attract higher demand. The second mechanism is provider responses. Even when there are no demand responses, report cards may still generate incentives for providers to improve quality if providers care about their performance relative to the peers [1].

Empirically, there is a large volume of literature examining the effect of releasing provider report cards on patient volume or provider choices. The main challenge of the literature is that most studies did not distinguish between consumer’s learning from report cards and that from non-report-card mechanisms that would occur independently of report cards [2-4]. While many other studies did either control for the effect of private learning or estimate the magnitude of it [5-12], the main issue with this set of studies is that they commonly assume that consumer learning from public report cards is independent of that from private channels, which may be not plausible.

This study focuses on testing whether report cards may interact with consumer’s private learning of quality, instead of assuming that they are independent from each other. Conceptually, it is unclear in which direction private learning should be affected by the release of report cards. For example, report cards may decrease the effectiveness of private learning, if report cards provide information that people would otherwise have to rely on private channels prior to the release of report cards. Assuming that public information is less costly, report cards thus may substitute away the need and usefulness of private information. On the other hand, report cards may increase private learning of provider quality. For example, the publicity that came with the implementation of report cards can increase public awareness and demand for provider quality. If the demand for quality increases, while critical information is not completely released on the report cards, consumers may turn to seek more information from private channels, or doctors may be pressured to better respond to quality in their referral decisions. This will result in an increase in the effect of private quality information on provider choice. Lastly, it is still possible that report cards may have no effect on private learning, if report cards simply do not affect consumer knowledge or behavior in any dimension. In any case, it would be problematic to assume the private learning is unaffected by report cards without testing it directly.

Our study addresses this gap in the literature by examining, specifically, whether consumer learning of hospital quality may be substituted away or enhanced by the release of report cards in the hospital setting. In this study, we term the learning of all possible channels other than the information from report cards as “non-reportcard learning”. To our knowledge, only one existing study, while not its main focus, allows non-report-card learning to interact with report card learning in the fertility clinic setting [13]. In that study, the 2 quality measures for quality, reported and non-reported, were highly correlated at 0.59, because one is the 3-year lag of another, and thus the estimates may not be stable. We improve the prior study by using 2 different quality measures to better separate out report-card vs non-report-card learning and testing them explicitly in a different hospital setting. The second contribution of our study is that we analyze the effect of report cards on physicians’ admission patterns. Taking advantage of the study data that allow us to track the same physician over time, we examine how a physician’s referral pattern across hospitals changes before and after the release of report cards using a difference-in-difference estimate at the physician level. Lastly, we analyze the effect of an aggregated 3-category quality ranking, rather than detailed numeric quality measures. The prior reporting method is of particular importance, as many states and the Center of Medicare and Medicaid Services that adopted hospital report cards use the 3-category reporting—“better than expected/above average,” “as expected/same as the average,” and “worse than expected/below average”. It is unclear whether the highly simplified information is useful to patients and there are relatively less empirical studies testing the effect of the simplified reporting.

2. METHODS

The study uses a pre-post difference-in-difference model to estimate separate effect of quality information on physician admission patterns using the unique information contained in the FL report card system. The state of Florida passed the Affordable Health Care for Floridians Act in 2004, requiring the Agency for Health Care Administration (AHCA) to “offer information about health care services, costs, and quality of care, including hospitals, nursing homes, ambulatory surgery centers, pharmaceuticals, health plans and physicians”. Beginning in November of 2004, the agency launched a new website, FloridaCompareCare.gov, to publish the cost and process indicators for over 150 medical conditions, surgeries, and procedures. For hospitals, the data also includes infection/complication rates and mortality rates for several common diseases/conditions. The implementation of the reporting system was a major undertaking at the AHCA with widely disseminated publicity.

The critical information that enables the present study is 2 distinct hospital quality measures available in the system: one aggregated 3-catogory quality ranking that was made available to public by report cards, and another detailed hospital-level actual risk-adjusted quality score that was never disclosed to the consumers but available to researchers before and after the report cards. This unique information allows us not only to use the reported quality ranking to estimate demand responses due to report cards, but also to control and estimate the change in demand response to non-report-card learning as a result of report cards.

The empirical difference-in-difference model is specified as in the following:

(1)

2.1. Dependent Variable

We use the percent distribution of a physician’s AMI patients across hospitals to describe a physician’s admission pattern. In other words, the dependent variable measures the percent AMI patients coming from a physician at a hospital in a given year. For example, if a physician has a total of 50 AMI patients in a year, and sends 20 patients to Hospital A and 30 to Hospital B. The dependent variable will record 40% for Hospital A and 60% for Hospital B for this particular physician. We use the percentage term instead of the absolute number of cases to adjust for the scale differences between physicians, and within a physician over time (which can be changing at different rates). Analyzing volumes at the physician-hospital level provides another advantage that the well-known volume-outcome endogeneity will be less of a concern at the physician-hospital level. Higher hospital quality may attract physicians to send more patients to that hospital, but a single physician’s volume at a hospital is less likely to be large enough to drive a hospital’s overall quality outcomes (with few extremely highvolume exceptions).

2.2. Key Independent Variables

The key independent variables of interest are “better/ worse” and “risk-adjusted mortality”. “Better” is an indicator variable which takes on the value of 1 if a hospital was ranked as “better than expected” in the prior year’s report card, and “Worse” indicates those hospitals that ranked “worse than expected,” respectively. Coefficient α indicatesa change in hospital admissions to hospital h per physician p due to a change in hospital h’s ranking in the report card to be “better” than expected, and coefficient β measures the effect of a “worse” report card ranking. The reference group is hospitals whose AMI mortality rate was “as expected.” Each quarter beginning the third quarter of 2004, the Florida AHCA calculated a hospital’s expected risk-adjusted mortality rateusing the AHRQ’s IQI tools with FL hospital discharge data. It then ranked hospitals into three categories: 1) better than expected, if the actual mortality rate is lower than expected, 2) as expected, if the actual mortality rate is within the expected range, and 3) worse than expected, if the actual mortality rate is higher than expected, and disclose only the 3 categories in their public reporting. Hospitals with too few cases in the condition will not receive a ranking. We obtain the actual reported data (three categories) from the agency directly between 2004 (4^th quarter) and 2008 yearly and these are entered as 1-year lag in the regressions to avoid volume-outcome endogeneity.

γ and r are the other 2 coefficients of interests. The variable “risk adjusted mortality” was the calculated but unreported hospital-specific risk-adjusted AMI mortality serving as a proxy of a hospital’s underlying quality. 2001-2004 was defined as the pre-report card period, and 2005-2008 as the post-report card period. Coefficient γ identifies the relationship between private knowledge of hospital quality (observable to consumers but unobservable to researchers) and admission patterns in the prereport card period, and coefficient r captures whether this relationship changed in the post report card period. The critical assumption is that because report cards never report the detailed, hospital-level quality score, any learning about the hospital-level quality has to come from non-report-card learning. We reproduce AMI risk-adjusted mortality rates using the same public-available AHRQ IQI tool¹, because the Agency was unable to share their calculated risk-adjusted mortality rates. AMI risk-adjusted mortality rates were also entered as 1-year lag.

2.3. Control Variables

Distance is calculated by vincenty method, which calculates the distance between two points on the surface of a spheroid using zip codes. We included 2 distance measures. The first is the distance between the referring physician’s office to the hospital where an admission occurs, which measures the preference/convenience of the referring physician. The second measure is the conventional travel distance between a patient’s residence and the admitted hospital, which measures a patient’s willingness to travel. It is worth noting that we cannot decompose changes in travel distance as pure physician’s or patient’s response. Patients could be switching to physicians who are willing to admit them to desired hospitals, or changes could come from physicians redirecting patients to certain hospitals. In this study throughout we treat any observed volume change as the result of joint physicianpatient responses.

Control variables also include several sets of fixed effects (FE). It can be argued that some hospital factors not observed by the researchers may be observable to physicians/patients. For example, a physician can learn from experience the true productivity level of a hospital that is not directly observed by hospital characteristics. If this underlying productivity is correlated with patient outcomes, then the estimates on the effect of “better” or “worse” are biased. We use a stronger hospital FE specification that controls for all fixed hospital characters that may be correlated with the level or change in hospital quality. Lastly, year fixed effects are used to control for year-specific time trends that affect all hospitals/providers commonly.

We use three specifications for regression model. The first is the hospital fixed effects model as described above. In the second specification, we also include physician FE in addition to the hospital FE to control for specific time-invariant provider characteristics that may influence a physician’s choice of hospital. This specification uses the change in “better” or “worse” ranking to identify its effect on the change in physician level AMI volume. In the third and strongest specification, we replace hospital FE and physician FE with physician-hospital pair FE. The physician-hospital pair FE essentially removes all the possible confounding variations between hospitals for a given physician, and uses only withinphysician and within-hospital variation over time to identify the effect of a change in report card ranking on a physician’s admission to that hospital.

2.4. Alternative Model

To better understand the timing of the effect of report cards, we replace the post period dummy in the main regression model with year FE and interact the unreported AMI risk-adjusted mortality with year FE:

(2)

where “RsikAdjusted Mortality” again are the calculated but unreported risk-adjusted AMI mortality of a hospital in the previous year, which is interacted with the year FE from year 2002 to 2008, where 2001 is the reference year. The set of coefficients γ_k will inform whether there is any systematic change between unobserved hospital quality and admission patterns over time, and how these changes may be correlated with the timing of the release of report cards. The regression model in Equation (2) is identical to the main model in Equation (1), except for the change of time period dummy in the interaction terms. We also used three specifications for Equation (2) as did for Equation (1)—one used hospital FE only, another used hospital FE and physician FE, and a third one used physician-hospital pair FE.

3. DATA

We obtain FL hospital discharge data from FL AHCA between 2000 and 2008. This data set contains the whole universe of inpatient discharges occurred during 2000 to 2008 at any hospital in FL. Data on hospital ranking as appeared on the report cards, starting from the fourth quarter of 2004 to the last quarter of 2008 was merged to the main discharge data, by 1-year lag. Hospital characteristics are obtained from the American Hospital Association Annual Surveys. Additional information on physician office address comes from Medical Quality Assurance Services of Florida Department of Health, and is linked to the master data by physician license number. Hospital-level risk-adjusted mortality rates are computed using the Agency for Healthcare Research and Quality (AHRQ) Inpatient Quality Indicators (IQI) tool, which is the tool FL AHCA uses to generate the 3-category of AMI mortality ranking (higher than expected, as expected, and lower than expected).

Overall, there are 336,142 AMI discharges occurring at about 200 hospitals between 2001 and 2008. We limit the study period to 2001 and 2008 due to the use of one-year lag variables. There may be concerns that some doctors may not routinely admit AMI patients, and thus whose admitting patterns can skew the estimates. Hence, we restrict the sample further to physicians who specialize in Internal Medicine, Family Medicine, or Cardiac Surgeons. This exclusion eliminates about 19% of the total hospital-year observations. In addition, AMI patients do not travel far. We exclude admissions that have long travel distances, measured as the distance between the zip code of admitted hospital or admitting physician’s office and patient’s home being greater than 50 miles or the admissions with any of zip code information missing. This drops out 20% observations. Furthermore, we exclude hospitals with less than 20 AMI cases per year and physicians who have only one-year records. These procedures together exclude another 2%. Our final sample includes 206,515 AMI admissions in 2001-2008 for the regression analysis.

4. RESULTS

Table 1 showed the descriptive statistics of the study sample. The number of AMI patients in FL was on a steady decline, from 27,370 in 2001 to 24,209 in 2008. Risk-adjusted mortality rates were declining as well. Interestingly, the number of physicians treating AMI patients also declined over time. The statistics in the last 2 columns indicated that the number of hospitals used by a physician declined from an average of 1.56 hospitals in 2001 to 1.36 hospitals in 2008, suggesting that physicians were narrowing their hospital referral network for AMI patients.

Table 2 reported the results using the main regression model in Equation (1) to examine the effect of report card information vs non-report-card information on a physician’s admission pattern. The first column controlled the hospital FE, the second column controlled the hospital FE and physician FE, and the third column controlled the physician-hospital pair FE. All the three columns have year FE. The results of the three columns reported a consistent finding. The coefficients of “better” and “worse” than expected mortality were not statistically significant, indicating that there was no direct effect of report card ranking on physicians’ admission patterns. The coefficients of “risk-adjusted mortality” in the prereport period, in years 2001-2004, were all positive and insignificant in Columns 1 and 3. In Column 2, there was