The Tower of Babel Problem with PROMs: Is Score Linking the Best Solution for a Unified Language? 

The Tower of Babel Problem with PROMs: Is Score Linking the Best Solution for a Unified Language? 

As patient reported outcome measures (PROMs) have grown in number and variety, key decision makers are unable to agree on a cohesive set of PROMs to support replicability and comparison in healthcare research, clinical care, and value-based reimbursement programs.1 This lack of a common “PROM language” has been described as a Tower of Babel problem.2

 

Today’s PROM users tend to fall into one of two camps: legacy PROMs (classical test theory-based) or modern science PROMs (item response theory-based). These camps are further divided by factors such as preferences for condition-specific or generic PROMs, familiarity and experience, and established clinical practice patterns.

 

High quality PROMs are crucial components of patient-centered care, performance measurement, and quality improvement.3 PROMs form the basis for patient-reported outcome performance measures (PRO-PMs), a type of quality measure prioritized in quality payment programs such as those managed by the Centers for Medicare & Medicaid Services (CMS). Emphasis on PRO-PMs is one aspect of CMS’ drive to elevate the voice of the patient by measuring and improving healthcare in ways that are most meaningful to patients, families, and caregivers.

Universal Foundation Initiative

As described in the recently launched Universal Foundation initiative, CMS is aiming for a streamlined set of quality measures to narrow the focus on priorities and comparable measurement across programs, settings, patients, and providers. It follows that PRO-PMs within this Universal Foundation need to be similarly streamlined, of limited number, and highly impactful.

 

We need a correspondingly finite and cohesive set of PROMs for the development and maintenance of such PRO-PMs; given that scores between different PROMs are not comparable, it won’t work to have a variety of PROMs to choose from within a single PRO-PM. The Tower of Babel problem makes selecting a single PROM as the basis for a single PRO-PM a difficult, if not impossible, task.

Score Linking

Score linking has been proposed as the solution to the PROM Tower of Babel.2,4,5 To understand how PROM score linking (sometimes called “cross walking”) works, check out this short video provided by Northwestern University and the PROsetta Stone project. As these resources explain, advanced psychometric methods are used to develop mathematical links between PROMs so that scores from different PROMs can be placed on a single scoring metric. A main product of score linking is a cross walk table which facilitates the transformation of one PROM score to the metric of another PROM, resulting in a “common PROM metric.”6-8

 

The goal of PROM score linking (cross walks) is to please everyone; providers get keep their choice of legacy or modern science PROMs, condition-specific or generic, based on their preferences and familiar practice patterns, while payers and policymakers get comparability across settings, patients, and providers. But with most things that sound this good, there’s a catch: it’s not simple, and it’s not cheap.

Considerations for Score Linking

While advanced score linking methods may be the best solution to provide common – or a universal set of – PROM metrics as the basis for PRO-PMs within CMS’ Universal Foundation, decisions about whether and how to implement PROM score linking should be made with a comprehensive understanding of all the implications. As examples of such implications, consider added costs and burdens to PRO-PM stakeholders such as:

  • Quality measure developers, who will need to account for score linking development requirements such as large sample sizes with patients who each responded to both PROMs, as well as specialized scientists with an in-depth understanding of the various score linking methods and additional layers of reliability and validity testing and expanded specification details for PRO-PMs. 3,4 Furthermore, in some cases, an additional adjustment factor may need to be scientifically developed to account for different measurement characteristics of the linked PROMs, such as for accurate benchmarking results in a quality payment program.
  • Quality measure evaluators, such as the CMS Consensus Based Entity, will need extra steps and expertise in an already complex process to evaluate the reliability, validity, and feasibility of the score linking components.
  • Electronic health record and other health information technology vendors that implement quality measure components, ideally within the context of seamless, standards-based, end-to-end data collection and reporting, will need to add score linking functionality in the proper sequence of calculating and reporting performance results.
  • Providers at the facility level will need to learn, understand, and interpret the role of score linking in the measures of performance they are graded by. While the score linking steps could theoretically take place fully outside the provider’s realm, it stands to reason that facility administrators will want to understand score linking with respect to the quality measures they report as part of understanding how their performance is being rated and to better harness performance feedback into continuous quality improvement cycles.
  • Providers at the patient care level, for similar reasons, will likely want the linked PROM scores at the point of care. Indeed, one of the most foundational reasons for emphasizing PRO-PMs is to encourage use of PROMs as part of clinical decision-making and to enhance communication with patients and families. However, this invites yet another layer of complexity and educational burden. While score-linking methods may produce accurate results when applied to large groups, scores are less precise on the individual patient level. Thus, such point-of-care interpretation requires deeper understanding since the measurement error around a single linked score may be quite large.

An Alternative Solution

Foregoing the added costs and complexities inherent in the score linking solution while achieving measurement that is cross-cutting for patient conditions, settings, and providers may require narrowing PRO-PM options to a unified and cohesive set of PROMs. If this becomes the desired direction, it should be considered that the PROMIS® measures were designed for this purpose and have continued to evolve toward even greater efficiency and customization to meet the granular needs of specialties and sub-specialties. In lieu of what may be a controversial, one-size-fits-all direction, score linking is likely the most harmonious solution to the PROM Tower of Babel. This path should be chosen wisely, with eyes wide open to recognize and plan for the added complexities it will inevitably bring.

 

References

  1. Kaplan RM, Hays RD. Health-Related Quality of Life Measurement in Public Health. Annu. Rev. Public Health 2022. 43:355–73. https://doi.org/10.1146/annurev-publhealth-052120-012811.
  2. Bjorner JB. Solving the Tower of Babel Problem for Patient-Reported Outcome Measures : Comments on: Linking Scores with Patient-Reported Health Outcome Instruments: A Validation Study and Comparison of Three Linking Methods. Psychometrika. 2021 Sep;86(3):747-753. doi: 10.1007/s11336-021-09778-x. Epub 2021 Jun 18. PMID: 34145529.
  3. National Quality Forum. Building a Roadmap From Patient-Reported Outcome Measures to Patient-Reported Outcome Performance Measures. July 21, 2022. https://www.qualityforum.org/Projects/n-r/PRO-PM/Developer_Feedback_Final_Report.aspx
  4. Schalet BD, Lim S, Cella D, Choi SW. Linking Scores with Patient-Reported Health Outcome Instruments: A VALIDATION STUDY AND COMPARISON OF THREE LINKING METHODS. Psychometrika. 2021 Sep;86(3):717-746. doi: 10.1007/s11336-021-09776-z. Epub 2021 Jun 26. PMID: 34173935.
  5. https://www.prosettastone.org/ Accessed July 31, 2023.
  6. Schalet BD, Revicki DA, Cook KF, Krishnan E, Fries JF, Cella D. Establishing a Common Metric for Physical Function: Linking the HAQ-DI and SF-36 PF Subscale to PROMIS(®) Physical Function. J Gen Intern Med. 2015 Oct;30(10):1517-23. doi: 10.1007/s11606-015-3360-0. PMID: 25990189; PMCID: PMC4579209.
  7. Oude Voshaar, M.A.H., Vonkeman, H.E., Courvoisier, D. et al. Towards standardized patient reported physical function outcome reporting: linking ten commonly used questionnaires to a common metric. Qual Life Res 28, 187–197 (2019). https://doi.org/10.1007/s11136-018-2007-0
  8. Lee, M.K., Schalet, B.D., Cella, D. et al. Establishing a common metric for patient-reported outcomes in cancer patients: linking patient reported outcomes measurement information system (PROMIS), numerical rating scale, and patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Patient Rep Outcomes 4, 106 (2020). https://doi.org/10.1186/s41687-020-00271-0