Moving the Quality Needle: The Evidence Conundrum

Recently CMS convened a series of technical expert panels (TEPs) that were charged with recommending new clinical performance measures (CPMs) that could be submitted to the National Quality Forum (NQF) for endorsement.  As a member of the fluid management TEP I was able to follow the process from start to finish, and we should all be concerned about the process and the way we need to drive better quality for our patients.

The purpose of a CPM is to change behavior on the part of a practitioner.  That is, the CPM is derived from rigorous evidence in the literature, and is then constructed to set a target for a particular outcome based on the evidence.  For example, if best evidence suggests that patients have better survival with Kt/V≥1.2, the CPM would be the fraction of patients achieving this goal, and targets would be set based on current performance and projections of what should be desirable and achievable.

When the four TEPs completed their work, however, it became clear that the lack of rigorous evidence in the literature severely limited what CPMs could be based on solid evidence.  Despite this fact, and the recommendations of a separate data TEP- a group that opined on the availability of the data needed to calculate performance on each CPM proposed- that there were significant gaps in this area, 44 measures were forwarded to NQF for consideration.  A mere handful were endorsed since NQF has an appropriately high bar that CPMs must meet, particularly regarding quality of the evidence base, before they will be endorsed.

Many of my colleagues who participated in the process expressed disappointment or even outrage that the work of experts could be rejected by NQF.  It was clear to me, however, that since CPMs are used by regulators and payers and others to judge the performance of providers it is essential that such measures truly are based on evidence, and unfortunately many of ours were not or the evidence was scant and weak.

Thus the conundrum- does the lack of evidence that meets CPM standards mean that we are now paralyzed in driving the quality agenda forward?  I think we need to step back and think more clearly about the difference between measuring quality for accountability purposes and driving quality improvement for patients.  This to me is best understood using the OJ Simpson analogy.  As many of you will recall this renowned football player was accused of brutally murdering his wife and her friend.  Mr. Simpson was acquitted in the criminal case but lost in a civil case brought by the victim’s family.  In the former, to be convicted in criminal court there needed to be proof beyond a reasonable doubt while in the latter a preponderance of the evidence was sufficient.

We need to ask ourselves, although CPMs need to meet the “proof beyond a reasonable doubt” standard, quality improvement initiatives do not, in my view.  The art of medicine mandates that clinicians make their best judgment, based on whatever evidence is available, and on the balance between risk and benefit, on what should be tried and what shouldn’t.  This approach should be applied not only to the care of individual patients, but to populations as well.  Each of us would be wise to review the 44 CPM recommendations that came out of the TEPs and consider how to apply them to our dialysis populations- they may not meet the proof beyond a reasonable doubt standard, but if implemented they will do more to help improve patient outcomes than will waiting for the next big randomized controlled trial.

I look forward to your comments, until next time.

Striving to bring quality to life,

Allen R. Nissenson, MD

To make sure you receive a notification when a new blog is posted, click here!