Navigating the RCT request: Providing alternative evidence
A common request technology companies receive from payers seeking additional clinical evidence to inform coverage decisions is a published article on the results of a Randomized Control Trial (RCT). RCTs are the preferred study design to demonstrate clinical utility; categorized by health plans as level 1 evidence along with meta-analyses of large RCTs. Because patients in RCTs have been randomized into treatment groups, the differences in study cohorts can be more confidently attributed to the intervention instead of differences in study cohort characteristics.
Unfortunately, RCTs are prohibitively expensive and time consuming and can therefore be improbable or even impractical for many companies. However, alternative study designs, while classified by health plans as a lower level of clinical evidence, can be used to provide evidence of clinical utility when designed effectively. Retrospective study designs, including claims data analyses, medical record reviews systematic reviews and open-label trials are all considered level 2 evidence when performed optimally. “Optimal” studies mean the study is comprised of a large, geographically diverse group of patients and an appropriate comparator group is utilized. Without these critical study design components, the analysis will be classified as level 3 evidence.
Suboptimal meta-analyses (those including studies with small sample sizes, non-randomized studies or study design flaws) are also considered level 2 evidence. Case control studies with small sample sizes are considered the lowest level of evidence, as it is difficult to generalize study findings from such a sample to a population such as a large health plan membership.
An alternative to RCTs
Real world data can be utilized to perform level 2 retrospective studies and demonstrate the clinical utility of a device or diagnostic, without requiring a prospective randomized study design. Data to perform these analyses is available in patient medical records, disease or device registries and administrative claims data, all of which have become more readily available to researchers. Medical record reviews do not require researchers to prospectively follow patients, and therefore can take less time to complete than RCTs. Medical records hold additional clinical information not found in claims data, such as medical symptoms, diagnostic test results and reasons for referral patterns that can be evaluated to show the clinical utility of a medical technology. “All-comer” disease and device registries can be extremely helpful to researchers, as these databases identify and collect demographic and medical information on patients with specific conditions, reducing the time to recruit a study population.
Claims databases are also rich with diagnostic and procedural information regarding treatments and interventions in real world populations. Unlike medical record reviews, which limit the study population to those treated at a participating facility, claims data analyses can offer a national perspective. Claims data can be employed to conduct longitudinal analyses and examine a myriad of clinically relevant outcomes such as discharge location, diagnoses present on admission (as opposed to those diagnosed during the hospital stay), cost, length of stay, readmissions and adverse events. Additionally, claims data analyses require significantly less financial and human capital resources to complete compared to RCT and medical record reviews. Further, the increasingly rapid disbursement of claims databases, such as the Medicare files now released quarterly instead of annually, and commercial plan files that are often updated monthly facilitates the retrospective analysis of treatment outcomes ever more quickly after a product is released into the market.
A meta-analysis performed on currently available peer-reviewed articles can also be successful in establishing clinical utility. As mentioned, a meta-analysis of large RCTs is categorized as level 1 evidence but a meta-analysis of smaller RCTs and/or cohort studies still has substantial value. Pooling together results from a range of studies provides a larger sample size of patients from multiple study sites.
To be sure, a meta-analysis of existing literature and retrospective analyses of data available in medical records and claims databases are capable of demonstrating clinical utility and providing evidence to support the overall impact of a medical technology.
Benefits of RCT alternatives
Retrospective study designs offer a multitude of supplementary benefits compared to RCTs. Due to the expense of designing and performing a large scale RCT, many studies result in small sample sizes from limited geographical areas. However, payers prefer studies with large external validity and applications in population health. Additionally, patients with complex medical histories and comorbidities, who make up health plan populations, are often excluded from RCTs. Yet, health plans need to know if the medical technology provides clinical utility to these real world populations.
Claims database analyses allow access to a large population of demographically and geographically diverse patients who represent real world patients, increasing the applicability of study findings to the health plan’s membership. While the ability to include patients across all regions of the U.S. is beneficial to national payers, a sub-analysis of the same data can show geographical differences that would be of interest to smaller regional payers. Further, when utilizing a lower level of evidence it becomes even more essential to have a large sample size, allowing for techniques such as patient matching to control for difference in study cohorts.
Access to a large sample of claims is also advantageous in the evaluation of medical technology treating rare diseases. Due to the prospective nature of an RCT, recruiting an adequate sample size to determine the effect of an intervention, assuming some patients may be lost to follow-up, can take a substantial amount of time. Using a retrospective design, a sufficient number of participants with ample follow-up time can be readily identified for analysis. Registries can be particularly useful in identifying and tracking study populations for rare diseases.
Patient retention is not only a concern in the study of rare conditions, but is also a challenge all RCTs measuring long-term outcomes face. However, long-term clinical impact of medical technologies is of particular interest to payers when making coverage decisions. Retrospective analyses enable follow up time to be pre-determined in the study design with only patients who meet the follow-up criteria included in the analysis, ensuring all outcomes can be evaluated within the desired timeframes.
A further benefit of retrospective analyses utilizing data from claims databases, disease and device registries and medical records is the ability to easily measure numerous study endpoints. Retrospective analyses can evaluate multiple outcomes in a single study, enabling researchers to assess the impact of a medical technology on a range of factors such as care utilization, length of stay, readmission rates, costs, etc. Providing evidence of multiple product benefits is crucial, particularly as the healthcare industry transitions to value-based care and evaluates the full impact of medical technologies over an episode of care. For example, a new and more expensive medical technology may be as effective in treating a condition as the current standard of care but if additional outcomes measured show the technology reduces readmission rates or overall costs (downstream benefits), the incremental cost of the technology can be justified to payers. Additionally, the ability to longitudinally track patients in claims databases enables researchers to profile a patient’s medical history, including prior procedures and diagnoses, to determine if the patient is appropriate for inclusion in the analysis.
Limiting a perceived weakness of retrospective analyses
While there are numerous benefits to retrospective analyses, there are also concerns that should be addressed. Notably, without randomizing patients into study cohorts there is a risk of selection bias and confounding variables impacting study results, making it more difficult to conclude that clinical differences between study cohorts are related to the intervention. However, propensity matching can be utilized to limit the effect of these concerns, strengthening the validity of the analysis. Propensity matching ensures patients in each cohort have similar demographic characteristics and comorbidity profiles so the researcher can more confidently conclude the differences observed between study cohorts are due to the medical technology, instead of an underlying and tangential variable.
With increased access to claims databases, development of registries and adoption of electronic health records (EHRs), retrospective analyses have become a strong source of evidence to demonstrate clinical utility, cost-effectiveness and other technology benefits. The disbursement and deployment of such studies into the payer marketplace are critical to their success and these studies may make smaller payers more receptive to having your company build some “coverage momentum” before gaining widespread acceptance of your technology.
In some cases, these analyses have made the difference in companies achieving positive coverage.
For more information on this topic, or to learn how Baker Tilly specialists can help, contact our team.
