By Rafael Fonseca MD & John A Tucker MBA, PhD
A Critical Analysis of a Recent Study by Hadland and colleagues
Association studies that draw correlations between drug company-provided meals and physician prescribing behavior have become a favorite genre among advocates of greater separation between drug manufacturers and physicians. Recent studies have demonstrated correlations between acceptance of drug manufacturer payments and undesirable physician behaviors, such as increased prescription of promoted drugs. The authors of such articles are usually careful to avoid making direct claims of a cause-effect relationship since their observations are based on correlation alone. Nonetheless, such a relationship is often implied by conjecture. Further, the large number of publications in high profile journals on this subject can only be justified by concerns that such a cause-and-effect relationship exists and is widespread and nefarious. In this article, we will examine a recent paper by Hadland et al. which explores correlational data relating opioid prescribing to opioid manufacturer payments and in which the authors imply the existence of a cause-and-effect relationship.1
We propose the relationship between transactions between the private sector (e.g., meals provided, consulting payments) and prescribing habits can fall into one of three categories:
|0||There is no cause-effect relationship between these transactions and prescribing habits. Correlative observations may merely be reflections of practice patterns, and likelihood to use a drug category.||No harm exists.|
|Ia||There is a demonstrable cause-effect for transactions and prescribing patterns. However, this relationship is associated with increased use of drugs that have been proven to be an improvement over the current standard.||The effect is beneficial for patients. “Beneficial marketing.”|
|Ib||An adverse causative effect is documented with establishment of causation. There is a possibility of patient harm.||Patient harm occurs because the wrong medication is administered and contravenes medical standards. A minor damage is done but arguably exists, if a physician prescribes a more expensive medication when a cheaper alternative exists.
Hadland et al.: Opioid Prescriptions and Manufacturer Payments to Physicians
The authors of this paper linked physician-level data from the 2014 CMS Open Payments database to 2015 opioid prescribing behavior described in the Medicare Opioid Prescribing Database. They explored the hypothesis that meals and other payments increase physician opioid prescribing by examining the association between receipt of meals and other financial benefits with the number of opioid prescriptions written. Specifically, they found the following:
- A nearly linear relationship between the number of opioid manufacturer-provided meals accepted by a prescriber and the number of opioid prescriptions written. The relevant data is provided in Figure 1 below. Prescribers who received nine meals from opioid manufacturers in 2014 prescribed opioid analgesics at slightly more than 3x the rate of those who accepted only one meal.
- When broken down by physician specialty, those who accepted any payment from opioid manufacturers wrote between 1.2% more and 11% more opioid prescriptions as those who did not accept any such payments (Table 1).
Figure reproduced from JAMA Internal Medicine 2018, volume 178, 861-3 under the Fair Use provisions of Section 107 of the U.S. Copyright Act.
Table reproduced from JAMA Internal Medicine 2018, volume 178, 861-3 under the Fair Use provisions of Section 107 of the U.S. Act.
Hadland et al. conclude that
Amidst national efforts to curb the overprescribing of opioids, our findings suggest that manufacturers should consider a voluntary decrease or complete cessation of marketing to physicians. Federal and state governments should also consider legal limits on the number and amount of payments.
While no cause-and-effect relationship between payments and prescribing habits has been demonstrated by this correlative study, the implication that one exists is made clear in the authors’ recommendations. In our analysis below, we attempt a deeper dive to determine whether such a cause-and-effect relationship exists.
Our View: It is More Complicated than That….
To better understand the issues presented by the Hadland’s correlative study, we undertook an independent analysis of the same data. We repeated the Hadland data extraction from the CMS sources cited in the paper. We associated payments with prescribing behavior using physician name and geographical information as described by Hadland. Despite the lack of detail provided in the publication, we closely reproduced the number of opioid prescribers, the number of opioid prescribers accepting payments, and the total number of payments described in the Hadland paper. The only discrepancy we found between our data and that reported by Hadland is that we found a more substantial total payment amount of $13.1M vs. the $9.1M reported by Hadland et al. We found no simple explanation for this discrepancy, as the total payment amount was consistently about 50% higher than that described by Hadland when stratified by source or by payment type. While we are not able to firmly assess the source of this difference given the lack of a detailed protocol in the paper, we believe that part of the difference may have arisen by including a more comprehensive range of opiate products in our analysis relative to that used by Hadland.
How Large is the Association Between Manufacturer Payments and Prescribing Volume?
Our first criticism of the Hadland analysis is directed at the non-standard presentation of the data in Figure 1. The most widely accepted way to present the relationship between two continuous variables such as payments and the prescription count is a correlation diagram. We present the data in this manner in Figure 2 (Note the logarithmic Y axis). Doctors who accepted no free meals from opioid manufacturers wrote between 0 and 1000 opioid prescriptions in 2015. As did those who accepted 50 or more.
Figure 2. Correlation Diagram Relating Number of Opioid Prescriptions Written to Number of Drug Maker Meals Accepted
This graph gives a very different impression than the presentation of the same data in Figure 1. Why is that? Here we have shown every data point, though some are hard to see because there are so many of them (345K to be exact). In Hadland’s presentation of the data, they grouped the prescribers into categories based on the number of meals that they accepted. They calculated the mean for each group, which hides the tremendous variation in prescribing behavior within each group. The error bars are shown in Hadland’s figure are not standard deviations (a measure of within-group variation) but standard errors (A measure of how precisely the mean has been estimated). The latter value is derived from the former by dividing by the square root of the number of data points, which ranges as high as 8468 for some of the categories in Hadland’s figure. So a clear representation of within-group variation would show error bars as much as 92-fold larger than those shown.
A similar criticism can be directed at the presentation of the data in Table 1. Comparing mean prescribing rates between those who accepted any payment and those who accepted none gives a non-representative picture because the distributions are highly skewed. Imagine a cancer trial in which 5 patients live 2, 3, 3, 4, or 20 months. Reporting that the average survival was 7.5 months and the standard deviation was 8.3 months really doesn’t give a very meaningful picture of what happened in the trial. Similarly, Hadland et al. report that physicians who accepted payments in 2014 wrote 539 +/- 945 prescriptions in 2015, while those who did not wrote 134 +/- 281. Who are the physicians who wrote less than zero prescriptions in 2015, and what does a negative prescription look like? This type of bizarre result arises from applying statistical methods appropriate to a normal distribution of values to a data set that is decidedly non-normal.
The problems become even more apparent when we compare these numbers to the authors’ statement in the text that those who accepted payments in 2014 increased their prescription count in 2015 by 1.6, while those who did not accept payments in 2014 reduced their prescription count by 0.8. How is the difference (2.4 prescriptions) equal to 9.3% of 134 prescriptions (Table 1)? And doe a relative increase of 2.4 prescriptions per year from a base of 539 prescriptions merit publication in JAMA Internal Medicine and a call for legislation?
Are Drug Companies Paying Doctors to Write Prescriptions?
While the correlation between meals and opioid prescriptions is much weaker than implied by the figures presented in Hadland et al., a reasonable person might still object that ANY exchange in which prescriptions result from a conscious or unconscious quid pro quo for free lunches is morally unacceptable (Type Ib). We would certainly take that position. So let’s analyze whether the relationship is causative or merely correlative. Hadland’s implicit hypothesis is that doctors are writing opioid prescriptions in “exchange for pizza.” An alternative explanation might be that attending manufacturer informational sessions at which meals are served and prescribing opioids might both be driven by having a practice that involves treating many pain patients. Let’s look at the data and see if we can distinguish between these possibilities.
- If Doctors are writing prescriptions in exchange for payments, one would expect that the number of prescriptions would rise predictably with the payment amount.
In practice, we find this is not the case.
Regressing the number of opioid prescriptions written on total payments received, we find r2 for the correlation is 0.01. Thus only 1% of the total variation amongst prescribers is associated with variation in the amount of payment received. (The gap in the graph between $0 and $10 arises because CMS does not require reporting of payments below $10).
Figure 3. Relationship Between the Number of Opioid Prescriptions Written and Total Payments Received
- If doctors are writing prescriptions as quid pro quo for industry payments, one would expect that non-meal payments would show a correlation with prescribing similar to the correlation with meals shown in Figure 1.
Alternatively, if both attendance at educational sessions at which meals are served and opioid prescribing are driven by having a practice that involves treating many pain patients, one might expect a very modest or no correlation of prescribing with non-meal payments.
In practice, we see the latter (Figure 4).
Figure 4 was drawn using Hadland’s categorical style of presentation to allow direct comparison to Figure 1. While Hadland found that opioid prescribing tripled as the number of industry-sponsored meals increased from one to nine, we find no trend in toward increased prescribing among those who received between $0.01 and $65,536 in non-meal payments from opioid manfacturers. In fact, the geometric mean rate was nearly identical for those receiving less than $1 in non-meal payments (711 prescriptions) and for those receiving $32,000 to $64,000 (718 prescriptions). For the 58 physicians who received more than $65,536, the rate of prescribing was increased by nearly twofold relative to those receiving less than a dollar, but due to large within group differences, this difference was not statistically significant.
The fact that opioid prescribing correlates with the number of meals accepted but not with the total amount of non-meal payments received suggests that attendance at educational events at which meals are served and opioid prescribing are both driven by practice characteristics. In contrast, these data are difficult to accommodate within the theory that the association of prescribing rates with meals accepted is due to quid pro quo, or that companies are bribing doctors to prescribe their products.
Figure 4. Geometric Mean Prescribing Rates by Total Non-Meal Payments Received
- If doctors are writing prescriptions in exchange for free meals, one would not expect meals provided by the manufacturer of non-opioid pain treatment to be associated with increased opioid prescribing. If doctors with large pain practices are more likely to attend informational lunches about pain products, such an association is expected and natural.
In practice, we find that the association of increased opioid prescribing with attendance at informational lunches offered by the manufacturers of pain therapeutics is independent of whether the pain product is an opioid!
St. Jude Medical is a medical device company that sells neuromodulation devices for the treatment of chronic pain. Those who attended St. Jude lunches prescribed opioids at the same rate as doctors who attended an equal number of lunches sponsored by opioid manufacturers. This observation holds up equally well when looking only at those who attended St. Jude lunches but did not attend any opioid lunches. We found similar associations with lunches provided by manufacturers of other non-opioids products (data not shown).
Figure 5. Relationship Between Attendance at Industry-Sponsored Lunches and Opioid Prescribing: St. Jude vs. Opioid Manufacturers
Correlation is not causation. While many advocates of reduced interactions between “commercial” interests and physicians have implied or directly suggested a quid pro quo between industry meals and other financial interactions and prescribing habits, correlation alone does not prove a quid pro quo relationship. In the case of opioid prescribing, we believe that we have presented a strong case that 1) the relationship between industry payments and prescribing is much weaker than has been presented in the literature, and 2) that prescribing and attendance at manufacturer-sponsored informational lunches are both driven by practice characteristics, rather than the meals themselves driving prescriptions (Type 0 relationship).
We believe that much of what has been published regarding the correlation of prescribing with industry payments and sponsored meals suffers from the shortcomings described in this short note. In particular, many of these papers conflate causation with correlation. In cases where fairly simple and obvious analyses would serve to differentiate between the authors’ preconceptions and alternative interpretations of the data, these analyses have not been performed. We urge all with an interest in this area to approach these data with the highest possible level of objectivity, as is our responsibility as scientists. We have done our best to do so here, and commit to doing so in our planned analyses of other papers in this area.
We look forward to a stimulating debate with those who have other data bearing on this issue, or other interpretations of the data presented herein.
Hadland SE, Cerdá M, Li Y, Krieger MS, Marshall BL. Association of pharmaceutical industry marketing of opioid products to physicians with subsequent opioid prescribing. JAMA internal medicine. 2018
. This analysis, as well as alternative analyses performed by the present authors, was limited to the prescribing behavior of those who wrote at least ten opioid prescriptions in 2015 due to redaction of counts between 1 and ten by CMS.
About the authors
Rafael Fonseca is a hematologist at the Mayo Clinic in Arizona and John is a medicinal chemist residing in Northern California.