Who wants to be responsible? Whenever anything goes wrong, the first thing they ask is, “Who’s responsible for this?” – Jerry Seinfeld
The National Quality Forum (NQF) recently released its second major report looking at how health insurance companies and others attribute responsibility for patients, visits, or episodes of care to specific health care providers. The potential to affect provider reputations and compensation makes attribution a high-stakes exercise, and it’s important to get it right.
Yet attribution is difficult to do well because reliable information from patients or providers on clinician–patient relationships is frequently unavailable, so attribution must rely on administrative claims or other sources to estimate provider accountability. Even when carefully performed, these estimates are imperfect. Attribution methodologies must seek to minimize error without knowing the truth, an exercise akin to trying to hit the bullseye without being able to see the dartboard. The best methodology varies with the measure, data source, and purpose of the assessment. Accordingly, the NQF report doesn’t offer a single solution, but instead it outlines key considerations for selecting the right methodology given the circumstances.
Despite its importance, attribution receives less attention from researchers when compared with other aspects of performance measurement, such as risk adjustment. Performance assessment programs have adopted a wide variety of approaches, but no consensus exists yet on the best approach in any given circumstance. NQF’s latest guidance is a welcome and much-needed contribution to the field, but there’s still more to do. The good news is that by paying close attention to how cases are attributed to providers, it is possible to begin to address concerns they commonly voice.
Concern 1: “These aren’t my patients.”
The most direct criticism of attribution is that an assignment is simply wrong. This bias can swing in both directions: attributing a patient to a provider who is not truly responsible for the patient’s care (a false positive) or failing to attribute a patient to a provider who is responsible for that patient’s care (a false negative).
It is tempting to favor inclusive approaches to attribution because they enable more providers to be graded based on more cases. Such approaches are less likely to exclude vulnerable subsets of beneficiaries, such as those with multiple chronic conditions and treatment needs that are highly dispersed across providers and care settings. Unfortunately, these approaches are also the ones most susceptible to false positives in the sense that they can hold providers responsible for driving major outcomes—such as improvement in a chronic condition or reduction in the patient’s total medical spending—even if the provider’s involvement with the patient’s care was extremely limited.
Solution: Carefully assess via empirical testing whether the assigned provider–patient relationships are meaningful.
Before being used for performance measurement, any attribution approach should be tested to ensure it reflects meaningful involvement by the provider with his or her assigned cases. For instance, it is common to assign responsibility to the clinician who saw the patient the most during the year. For such rules, it is important to measure the percentage of those visits the assigned provider actually accounted for. Even for the clinician who saw the patient the most, it’s easy to imagine scenarios in which the patient–provider relationship would be more or less meaningful. For example, the relationship would be generally more meaningful if the patient saw the assigned provider for 8 of her 10 doctor’s visits during the year than if the patient saw that provider for only 3 of her 10 visits.
For time-limited measures within a larger performance period (such as hospital readmission measures), one should measure the proximity of the episode to any attribution-related patient–provider encounters. A clinician who treated a patient in February might not be meaningfully involved with that patient’s care by a December hospitalization.
Concern 2: “This score doesn’t reflect my real performance.”
Performance scores might fail to reflect a provider’s true performance for multiple reasons, only some of which relate to attribution. Just as adopting too inclusive a rule can compromise validity (measurement accuracy), adopting too restrictive a rule can compromise the statistical reliability (measurement consistency). Measuring performance based on a small sample increases the opportunity for a small subset of nonrepresentative cases to skew results relative to a provider’s true long-run performance. A clinician that was attributed responsibility for the costs of 10 patients could score poorly if even one or two patients’ care needs are unexpectedly high, but this kind of bad luck is much less likely to drive a score when measurement includes the costs of 100 or 1,000 patients.
Solution: Attribute responsibility at a different level of care, aggregate responsibility of multiple connected entities, or use statistical methods to borrow strength from other data to attribute enough cases for accurate measurement.
Relaxing an otherwise valid attribution approach can exacerbate concerns discussed above, but there are other options:
- One can attribute responsibility at a different level of care. An attribution approach might, for example, assign too few cases to many individual clinicians but be sufficient when grading practices.
- Another alternative is to combine the data of distinct organizations, such as practices, to obtain sufficient volume for assessment. To do so, it must make clinical sense to treat the distinct entities as one for the purpose of assessing performance. Whether that is reasonable will depend on the specific context. Grading practices in the same geographic region on their aggregated data, for example, will make more sense if the practices are already working together to improve their performance than if they simply happen to be located near each other.
- A third approach is to complement attribution with data beyond the provider’s immediate performance data. Random effects models are a type of statistical model that base scores partly on performance of peers—such as clinicians in the same practice, specialty, or geographic region. Hospital measures have used such an approach. Another option is to use the provider’s own data by looking further back in time.
- If none of these solutions are viable, the program should seriously consider whether to assess providers on that measure. Forgoing measurement could be preferable to attributing too few patients to have confidence in the performance score.
Concern 3: “It’s not my fault.”
The question of who should be held responsible for outcomes is particularly challenging for broadly defined or population-based measures such as total cost of care. Multiple providers will typically contribute to the cost of a patient’s care over a year, but many of those will feel they should not be held responsible for, or are not even able to influence, the total. For example, a urologist overseeing treatment of a patient’s prostate cancer might be assigned responsibility for a patient’s medical spending for a year but will not generally be in the position to influence the cost of treatment for the same patient’s heart condition.
Even primary care providers seeking to manage their patients’ care comprehensively find it challenging to control a patient’s total costs in an open network, to say nothing of specialists. And even when clinicians can influence outcomes, properly allocating responsibility among different providers remains challenging.
Solution: To promote provider acceptance, perform thorough validity testing that includes input on attribution from clinicians of all specialties, roles, and practice types that will be assessed.
Using an evidence-based approach is critical to ensuring the reasonableness of an attribution rule. This approach includes looking for evidence in existing literature or through new research that the provider can influence the outcome of this measure for this patient or episode. This is especially important considering the large number of measures that can be influenced by forces outside the provider’s control. Although empirical assessment is important, testing should also include input from the full range of provider types that could be attributed cases, especially for broad population-based measures. Following a review of its approach to attribution to accountable care organizations, for example, the Shared Savings Program clarified which specialties can be attributed beneficiaries under its two-step assignment algorithm and which cannot.
Some initiatives might use attribution in an aspirational way to change how accountability is perceived. One example is assigning responsibility to a single provider for the actions of multiple providers to spur investment in care coordination. Because correctly attributing responsibility is so challenging, taking the additional step of using an attribution rule to drive changes to patient care should be done with extreme caution. Such approaches should be used judiciously and considered experimental until sufficient evidence exists to support their use in public reporting and payment.
It will get easier, eventually.
Performance assessment for quality improvement, public reporting, or payment purposes all require an approach to attribution, and payers have been developing approaches for decades. That NQF and others continue to release guidance on attribution reflects how challenging it has been to get it right. Moreover, significant questions await further research. For example, it is important to consider accountability for deciding on a treatment’s appropriateness: a rule might assign responsibility for a hip replacement to the orthopedic surgeon performing it without considering the surgeon’s or any referring physician’s responsibility in assessing whether alternative treatments such as physical therapy were considered and appropriately ruled out.
As with other aspects of performance measurement, attribution can be improved and made easier over time with careful research and testing. We have already made substantial progress and should not hesitate to accept the challenge of making attribution even better. The success of value-based payment approaches depends on it.
