A Paradigm Shift in Digital Health

Innovation is best scaled when pipelines are replaced with platforms.

John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, wrote this article.

The Digital Health Frontier includes cutting edge predictive analytics, machine learning enhanced algorithms and big data analytics. But for these innovations to have their full impact on patient care requires the right strategic and operational foundation. In the past, many technology-focused organizations have relied on a pipeline approach as the foundation to construct innovations and promote growth. But history suggests this approach is less sustainable than a platform approach. A recent article in Harvard Business Review (HBR) sums up the difference: “Platform businesses bring together producers and consumers in high-value exchanges. Their chief assets are information and interactions, which together are also the source of the value they create and their competitive advantage…. Pipeline businesses create value by controlling a linear series of activities — the classic value-chain model. Inputs at one end of the chain (say, materials from suppliers) undergo a series of steps that transform them into an output that’s worth more: the finished product.” While this explanation gets the point across, it’s rather abstract. To really appreciate the advantages of one approach over the other, we need an example or two.

Apple’s handset business follows the pipeline model, making sure there are adequate supplies available to build the device and then overseeing the various other steps to create a finished iPhone, as well as its distribution, sales, and servicing. But when Apple linked the phone to its App store, the situation changed dramatically, turning the operation into a sustainable platform that connected app developers with iPhone owners. In their HBR article, Geoffrey Parke and Sangeet Paul Choudary explain: “The resource-based view of competition holds that firms gain advantage by controlling scarce and valuable — ideally, inimitable — assets. In a pipeline world, those include tangible assets such as mines and real estate and intangible assets like intellectual property. With platforms, the assets that are hard to copy are the community and the resources its members own and contribute, be they rooms or cars or ideas and information.” Apple’s success and the loss of market share by pipeline-oriented companies like Nokia can be explained by such differences.

Like Apple, John Deere has successfully employed a platform approach. They own not just the physical assets — e.g. tractors and combines — but a vast collection of intellectual property — including APIs and apps to help farmers manage what is now being called precision agriculture. The company links third party providers and producers to their farming customers and reaps the benefits. With all these technological tools in place, farmers now have the ability to more efficiently monitor their equipment with data on fuel consumption, location, machine hours, and engine RPMs; and they can improve crop management with weather prediction data, community pricing and the like. Some of John Deere’s more advanced combines incorporate a grain quality camera, grain loss sensor, a Gen4 display monitor, and remote access to an operations center from inside the cab. For every new connected tractor sold, more data flows into the John Deere Platform, enhancing the value of the platform to partners creating new apps and analytics.

Mayo Clinic Platform (MCP) is taking a similar approach. Instead of creating dozens of pipeline businesses or building an organization chart to support pipeline businesses, we are leveraging external collaborators, network effects, and data flowing back to the Platform, which increases its value for producers of products and consumers of services. Mayo Clinic and Commure, a General Catalyst portfolio health care technology company, have launched Lucem Health to connect data from remote medical devices with AI-enabled algorithms. External collaborators who have partners with MCP include nference, Medically Home, Kaiser Permanente, and K Health. The strategic approach allows the Mayo Clinic Platform to offer products and services that fall into 4 broad categories of functionality: Gather, Discover, Validate, and Deliver. For example, in the Deliver “bucket” is the combined ECG/algorithm system that was recently validated and published in Nature Medicine. The digital tool was able to detect low ejection fraction, thereby improving the diagnosis of left ventricular systolic dysfunction. While the ECG/algorithm is improving direct patient care at Mayo Clinic, it can also be offered to external partners and embedded in their ECG waive form viewer, which in turn will improve the relationship between a community hospital and its patient population.  Similarly, the clinical data analytics tools developed by MCP are being made available to outside partners like K Health, which provides symptom checking, access to virtual visits with a clinician. The data analytics function is helping K Health improve its services to their clientele.

As the HBR article emphasized, the chief assets of a platform are information and interactions, which together are also the source of the value they create and their competitive advantage. Such value and advantages are what will sustain healthcare innovators through the next several decades.

When AI Meets SDOH

Artificial intelligence can help identify and address the social determinants of health.

John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, wrote this article.

Machine learning is getting better at predicting things. There are now algorithms that improve the detection of diabetic retinopathy, predict the onset of sepsis, and help determine a critically ill patient’s risk of dying.  But a piece of wisdom from Warren Buffet comes to mind: “Predicting rain doesn’t matter. Building arks does.” Even the most impressive algorithm is relatively useless if it doesn’t allow us to build better “arks” to address the medical  disorder or  complications that the digital tool identifies.  And building the best healthcare interventions requires that clinicians not just identify the right signs, symptoms, and biomarkers, whether they be high cholesterol levels, elevated A1c, or a lump in a woman’s breast.  It requires we understand what’s happening in patients’ everyday lives outside  the clinic, the so-called social determinants of health (SDOH), and then using that data to inform treatment.

A great deal has been written recently about SDOH. Health professionals are slowly beginning to realize that we cannot “remove health and illness from the social contexts in which they are produced,” according to Simukai Chigudu, Oxford Department of International Development, University of Oxford.¹  That begs the questions: What social issues are mostly likely to influence our patients’ clinical course and what do we do about them? How can AI help alleviate the impact of these issues?

The Centers for Disease Control and Prevention (CDC) has numerous data sources to help incorporate SDOH into public health initiatives and medical practice. But as the agency points out, moving from data to action is the hard part. CDC has several programs designed to focus clinicians’ attention on key social issues, including socioeconomic status, educational level, and work history. One initiative, for instance, zeros in on the role of EHRs. Its purpose is to support the incorporation and use of structured work information into health IT systems. How might this SDOH element inform a physician’s different diagnosis? Consider a patient with hypertension who doesn’t respond to a low sodium diet or anti-hypertension medication. Awareness of his 10 year history as a house painter might point the clinician in the direction of lead poisoning, a possible cause of hypertension. Similarly, a nurse practitioner may be at a loss to figure out why a patient with type 2 diabetes has recently seen a spike in her A1c levels.  If at EHR system is linked to work history, when the NP enters the reason for the clinic visit into the EHR field for chief complaint, this might trigger a pop up box that states that the patient works the night shift and that shift work can affect diabetes control. The system would then provide recommendations on diabetes management among shift workers. The same CDC program is also working on a work information data model, as well as national standards for vocabulary, system interoperability, and instructions for health IT system developers. 

At Mayo Clinic, we are also studying the impact of SDOH on health and disease. Young Juhn, MD, MPH, Director of the AI Program and Precision Population Science Lab of Department of Pediatric & Adolescent Medicine at the Clinic, has studied  the effects of socioeconomic status on health since in 2006 when his research work was supported by the NIH. With the support from the NIH, he developed and validated a housing-based socioeconomic measure called the Housing Based Index of Socioeconomic Status or HOUSES index, which is being used in epidemiologic research to help understand health disparities and differences in a variety of health outcomes in both adults and children. The index has enabled researchers to overcome the absence of socioeconomic measures in commonly used data sources (e.g., medical records or administrative data), conduct geospatial analysis in health disparities research, and apply a life course approach.

The HOUSES index is an objective way to measure the individual-level socioeconomic status of  patients because it is based on real property data for individual (not aggregated) housing units and is derived from public records; it uses 4 data points: the number of bedrooms in a person’s residence, as well as the number of bathrooms, square footage of the unit, and estimated building value of the unit. The index can help target patients who are most at risk of poor health outcomes and inadequate access to health care , demonstrating the real value of adding SDOH into the mix by addressing the limitations of the existing SDOH. For example, Stevens et al have shown that patients with a higher HOUSES score (quartiles 2-4) had 53% lower risk of  kidney transplant rejection (adjusted hazard ratio 0.47), when compared to those with the lowest score (quartile 1).² Dr Juhn and his colleagues have found that HOUSES can  predict 44 different health outcomes and behavioral risk factors in both adults and children.

Of course, clinicians still have to be reasonable in their expectations. Even if an algorithm were outfitted with every conceivable SDOH, it still may not reduce disparities in healthcare. Patients and providers may choose to ignore the recommendations of the improved algorithm because they believe the recommended diagnostic test is too expensive or unjustified, for example, because it is too difficult for patients to get to the testing facility, or because a patient’s lack of health literacy prevents them from seeing the value of said test.

Despite these shortcomings, SDOH-enhanced algorithms have the potential to improve patient care. While physicians and nurses have gained tremendous insights into health and disease by measuring countless clinical parameters during office visits, it’s clear now that’s not enough.  The clinical picture generated with these metrics is too often hazy and needs to be supplemented by a long list of social metrics that can influence a patient’s access to care and their long-term outcomes.

References

 1. Chigudu S. Book: An ironic guide to colonialism in global health. Lancet. 2021. 397:1874-1975.

 2. Stevens M, Beebe TJ, Wi Chung-II et al. HOUSES index as an innovative socioeconomic measure predicts graft failure among kidney transplant recipients. Transplantation 2020; 104:2383-2392.

The Digital Reconstruction of Healthcare is Upon Us

The transition from brick and mortar to digital medicine will profoundly impact the way clinicians and patients interact—and will likely improve clinical outcomes.

John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, wrote this article.

Paul Cerrato and I are excited to finally announce the publication of our 5th book together: The Digital Reconstruction of  Healthcare: Transitioning from Brick and Mortar to Virtual Care. In March, we posted the table of contents of the new book. Now that it’s reached the “newsstand,” we wanted to whet readers’ appetite by sharing some additional excerpts.

The logical place to start any discussion on this topic is to explain why  digital reconstruction is necessary, which we address in Chapter 1:

Episodic Medical Care Often Falls Short

White coat hypertension, the tendency for patients to only present with elevated blood pressure during a doctor visit, illustrates a problem that permeates the entire healthcare ecosystem. Any sign or symptom that a patient exhibits during an office or clinic visit may not be a true presentation of their underlying condition. Unfortunately, this phenomenon not only affects a person’s blood pressure but other common parameters. White coat hyperglycemia has also been documented. And since psychosocial stress is likely a contributing cause of such white coat reactions, white coat hypercholesterolemia, asthma attacks, and numerous other conditions probably exist as well, all triggered by stress hormones. Conversely, any normal readings during a physical examination or laboratory test do not necessarily mean a patient is in good health.

The common denominator in all these scenarios is episodic care. In such situations, clinicians are making a judgement about a patient’s health status based on cross-sectional data, which can be misleading. But given the financial restraints and incentives that exist in healthcare today, it has been the only viable option—until now. With the emergence of virtual care and remote patient monitoring (RPM), gathering long-term data for many clinical parameters is no longer out of reach. That steady stream of online data can be inserted into predictive analytics algorithms to help locate patients at high risk. Some thought leaders refer to this shift in priorities as the movement from episodic to life-based care.

Such digitally enhanced patient engagement is the future of healthcare. No responsible practitioner would conclude a diabetic patient is in good metabolic control based on a single blood glucose reading, and yet that is often the same reasoning we use when a routine metabolic panel comes back stating LDL cholesterol, serum calcium, white blood count, blood pressure, and numerous other parameters are all “within reference range.” We now have the technology to move beyond this outdated mindset. That technology enables us to detect longitudinal patterns of change in patients’ health status. By way of example: Longitudinal data on systolic blood pressure has been linked to patients’ risk of cardiovascular disease.

The Power of Remote Patient Monitoring

Many patients and healthcare professionals have yet to appreciate the power of remote patient monitoring. When executed correctly, it can be truly transformative, combining medical self-care, objective physiological data, and expert advice to improve both preventive and therapeutic care. And as RPM continues to mature, it has the potential to completely reinvent healthcare, especially among those motivated patients who see it as a source of self-empowerment. The power of RPM in the hands of motivated asthmatic patients was well illustrated in an experiment conducted by University of Wisconsin and Centers for Disease Control and Prevention researchers. Using an electronic medication sensor that was attached to inhalers of 30 patients, Van Sickle et al. tracked patients’ use of inhaled short-acting bronchodilators for 4 months. To evaluate patients’ health status, investigators asked them to fill out surveys, including the Asthma Control Test (ACT). One month into the study, they also received weekly emails that summed up their medication usage for the preceding week and offered suggestions on how to comply with the National Asthma Education and Prevention Program guidelines. No changes were observed in ACT scores after the first month, but they increased by 1.40 points each month after that. Patients also reported significant decreases in daytime and nighttime symptoms. They also noted “increased awareness and understanding of asthma patterns, level of control, bronchodilator use (timing, location) and triggers, and improved preventive practices.” That last statement is worth closer inspection.

Very often, patients do not understand the triggers that cause symptoms, unless they are actually attuned to subtle changes in their physiology. Providing graphic displays of their symptoms paired with the medication usage can be eye opening for many patients who never noticed patterns of use before. These newfound revelations were summed up by several patients participating in the study:

“I learned that I used my inhaler more than I remember. I was able to see and relate to my doctor that my asthma is not under control.’’ Participants also reported that the receipt of information about the time and location where they used their inhaler helped to highlight locations and exposures to triggers that led to symptoms. ‘‘I’ve been more keen to note surroundings when I feel shortness of breath,’’ one participant said. ‘‘It opened my eyes to triggers I wasn’t aware of in the past.’’

The results of this experiment highlight 2 important lessons for patients and clinicians, summed up in a few choice words from Kamal Jethwani, MD, MPH, from Partners HealthCare: “The future of health is proactive, self-managed wellness. We want to put the onus back on the person. We’re saying: It’s your health, and I’m no longer your babysitter.”

AI-Enhanced Cardiology Takes Another Step Forward

Combining a convolutional neural network with routine ECGs detected low ejection fraction, a signpost for Asymptomatic left ventricular systolic dysfunction

John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, wrote this article.

Asymptomatic left ventricular systolic dysfunction (ALVSD) may not be the most familiar disorder in medicine, but it nonetheless increases a patient’s risk of heart failure and death. Unfortunately, ALVSD is not that easily detected. Characterized by low ejection fraction (EF) — a measure of how much blood the heart pumps out during each contraction — it’s readily diagnosed with an echocardiogram. But because the procedure is expensive, it’s not recommended as routine screening for the general public. A recently developed AI-enhanced algorithm that’s used in conjunction with an ECG can identify low EF, one of many advances that will eventually make machine learning an essential part of every clinician’s “tool kit.”

The new algorithm, a joint effort between several of Mayo Clinic’s clinical departments and Mayo Clinic Platform, was published online by Nature Medicine. The EAGLE trial included over 22,000 patients, divided into intervention and control groups and managed by 358 clinicians from 45 clinics and hospitals. The algorithm/ECG was used to evaluate patients in both groups but only those clinicians allocated to the intervention arm had access to the AI results when deciding whether or not to order an echocardiogram. In the final analysis, 49.6% of patients whose physicians had access to the AI data underwent echocardiography, compared to only 38.1% (Odds ratio 1.63, P< 0.001). Xiaoxi Yao, with the Kern Center for the Science of Health Care Delivery, Mayo Clinic, and associates reported that “the intervention increased the diagnosis of low EF in the overall cohort (1.6% in the control arm versus 2.1% in the intervention arm) and among those who were identified as having a high likelihood of low EF.” Using the AI tool enabled primary care physicians to increase the diagnosis of low EF overall by 32% when compared to the diagnosis rate among patients who received usual care. In absolute terms, for every 1,000 patients screened, the AI system generated five new diagnoses of low EF compared to usual care.

Earlier research on the neural network used to create the AI tool had shown that it’s supported by strong evidence. A growing number of thought leaders in medicine have criticized the rush to generate AI-based algorithms because many lack a solid scientific foundation required to justify their use in direct patient care. Among the criticisms being leveled at AI developers are concerns about algorithms derived from a dataset that is not validated with a second, external dataset, overreliance on retrospective analysis, lack of generalizability, and various types of bias, issues that we discuss in The Digital Reconstruction of Healthcare. The EAGLE trial investigators addressed many of these concerns by testing its algorithm on more than one patient cohort. An earlier study used the tool on over 44,000 Mayo Clinic patients to train the convolutional neural network and then tested it again on an independent group of nearly 53,000 patients. And while this study was retrospective in design, other studies have confirmed the algorithm’s value in clinical practice by using a prospective design. The most recent study, cited at the beginning of our blog, was not only prospective in nature, it was also pragmatic, which reflects the real world in which clinicians practice. Traditional randomized controlled trials consume a lot of resources, take a long time to conduct, and usually include a long list of inclusion and exclusion criteria for patients to meet. The EAGLE trial, on the other hand, was performed among patients in everyday practice.

A Unique Partnership Delivers Acute and Holistic Home Care

Mayo Clinic, Kaiser Permanente, and Medically Home join forces to offer patients the best of both worlds, forging a partnership that has the potential to redefine the hospital industry.

John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, wrote this article.

One of the problems facing most patients who require hospital admission is removing their familiar surroundings and emotional supports. While these resources might be considered less critical than specialized clinical expertise, there's little doubt that these "less important" factors play a crucial role in the healing process. Even the most dedicated nursing staff can never replace having a loved one available 24/7 at home. Nor can the most nutritious hospital food ever replace appetizing home-cooked meals. Equally important are the familiar wake/sleep cycles that patients are accustomed to at home, which usually must give way to hospital routines that demand blood pressure checks at three in the morning.

A new partnership[FJA1]  between Mayo Clinic, Kaiser Permanente, and Medically Home launched recently to expand access to care that combines the comforts of home with the expertise of hospitalists, helping patients receive the holistic care needed to speed long-term recovery and the acute care services to address their immediate medical needs. Stephen Parodi, MD, executive vice president of the Permanente Foundation, summed up the philosophy behind the new initiative succinctly, "Treating patients in their home allows physicians to treat the whole patient. We see their individual needs and can integrate critical information, such as diet, physical environments and social determinants of health, into their care plans."

The challenge, of course, is how to turn this philosophy into a cost-effective, safe program. In fact, that transition has already begun.  As we discussed in a previous blog, Mayo Clinic launched its advanced care at home program last summer at Mayo Clinic in Florida and Mayo Clinic Health System in Eau Claire, Wisconsin, to deliver complex, comprehensive care and restorative services to qualifying patients in their homes. These services, which are provided in-person and virtually, include:

• Infusions.
• Skilled nursing.
• Medication delivery.
• Laboratory and imaging services.
• Behavioral health.
• Rehabilitation services.

Similarly, Kaiser Permanente launched its hospital-at-home program in two regions last year, admitting patients from multiple hospitals across both its Northern California and Oregon locations. In this model, Kaiser Permanente has a single medical command center in each region, supporting multiple hospitals to care for patients longitudinally across their acute and restorative phases.

The new partnership will scale Medically Home’s operations, allowing more providers to offer this unique care model. The model includes a 24/7 medical command center (Figure 1) staffed with clinicians in regular communication with a care team in the community that contains EMTs and nurses who provide bedside care. Among the elements that make the new program unique:

• Required protocols for high-acuity care in the home.
• Rapid response logistics systems and providers of care in the home.
• Integrated communication, monitoring and safety system technology in the home.
• A software platform, the Cesia® Continuum, for orchestrating high-acuity care in patients’ homes. (Figure 2)

One of the problems with writing about a significant health care event while at the same time being a key player in the event is some outsiders will question our objectivity and immediately assume we exaggerate its importance to gain a competitive advantage. The plain truth is this partnership is not just about Mayo Clinic, Kaiser Permanente, and Medically Home. The partnership's ultimate goal is to bring better care to patients across the country and the globe. With that in mind, the program will provide all the necessary outcomes data, tools, systems, training and technology to enable the model’s widespread adoption.

Figure 1

Figure 2

Health Data Privacy Gets the Attention It Deserves

The Partners in Privacy Conference gathered world-class experts to address some of health care’s most vexing problems.

John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, wrote this article.

The challenges involved in keeping patient and consumer health data private may seem daunting. Still, a recent virtual conference hosted by Mayo Clinic brought together over 80 world-class experts to address the issues. Their insights are worth a closer look.

During his opening remarks at Partners in Privacy Conference: The Ethical and Responsible Use of Data to Drive Cures (April 22, 2021). Gianrico Farrugia, M.D., CEO of Mayo Clinic, acknowledged the delicate balance required to respect the public’s desire to keep its data confidential and the health care community’s desire to use that data to improve patient care, “What the right thing is is not a simple question to answer — it is complex and can vary in different countries and even under different circumstances. A person may be less or more willing to share information and have a different view on data privacy at different times in their lives.  What degree of data privacy seems right to a healthy 40 year is likely not going to be the same for that same person with advanced cancer or neurodegenerative disease. Moreover, the world of research and collaboration is changing. There are global opportunities for new partnerships among medical centers, industry and government that increasingly involve data sharing.”

With these concerns in mind, Dr. Farrugia introduced the keynote speaker, Micky Tripathi, Ph.D., MPP, the National Coordinator for Health Information Technology for the U.S. Department of Health and Human Services. Micky briefly reviewed the achievements of the HITECH Act and the implementation of EHRs around the country. Still, he also pointed out that the speed with which this rollout occurred has made us all realize that “technology has outpaced policy.” One of the ways in which this disconnect is being addressed is through the 21st Century Cures Act. As of April 5, 2021, the law now requires that providers, health care information networks, and technology developers give the public friction-free access to and control of their health data through apps. But that access also means patients can more easily share that information with third parties that are not required to follow the rules spelled out in HIPAA. And even when such data remains within the confines of a health care provider organization bound by HIPAA regulations, keeping it private and secure remains a challenge, despite the fact that there are ground rules on de-identifying it. 

These challenges were among the many questions addressed by the four breakout groups that followed Micky’s presentation. We discussed privacy laws and regulations; state-of-the-art methods for protecting data privacy used to advance health care; consumer and patient attitudes about privacy; and balancing privacy protection with the benefits of research and commercialism.  The lessons learned from the conference will be presented in a white paper that is currently being developed by the thought leaders involved in the project. But in lieu of that, consider a few takeaways:

• Patient consent will evolve. In the future, there will be more granular control options; we will likely go from a black and white consent decision to a few more controls based on the use of the data and the actors who have access to it.
  
  
• There will be more transparency in data use. That can take the form of a “nutrition label” type description for every algorithm that clearly spells out the data used to create it, as well as its performance and characteristics.
  
  
• Technology is evolving, with machine learning and natural language processing accelerating very quickly. It will fundamentally change how we deliver care, but it will also mean more data being used for more purposes. It is incumbent upon us to keep that data safe and to respect patient preferences as we develop algorithms. Fortunately, privacy and security technology is evolving as well, including advances in de-identification, encryption, allow lists and tokenization. However, the conference attendees emphasized that these approaches are imperfect, which means we will need a multi-layered approach.

Cris Ross, the CIO at Mayo Clinic, summed up many of the observations gleaned from the conference: “We came into this conference wondering whether matters of law, regulation, policy, technology, and market practice are fully established, or if there’s a need for further exploration and consensus. We concluded there is a need for more exploration, and a need for a group like this to help create a consensus, with the goal of advancing cures with the ethical use of data and preservation of patient privacy?" Cris emphasized that this virtual meeting demonstrated that this need exists and we would like to issue a call to other leaders in this field to advance the agenda. Partners in Privacy Conference: The Ethical and Responsible Use of Data to Drive Cures was only the first step in a journey that will require the input and expertise of stakeholders around the nation and the world.

It’s OK to Break the Rules Now and Then

Technological innovation sometimes requires we take risks — and question the tenets of evidence-based medicine.

John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, wrote this article.

It’s challenging at times to know when to follow the rules and “color inside the lines” and when to ignore those lines and forge ahead. That’s true whether we’re navigating everyday life, creating new technology, or devising the best patient care initiatives. Which brings to mind a quote from Elbert Hubbard: “The greatest mistake you can make in life is continually fearing that you'll make one.” 

Over the years, we have discussed the strengths and weaknesses of evidence-based medicine and randomized controlled trials (RCTs) in several books and articles, our point being that fear of investing in a treatment approach because it not supported by the RCT “gold standard” can create a kind of inertia that ultimately hurts patients.^1,2  And even if we put aside the fear factor that Hubbard mentions, mounting evidence strongly suggests that an in-depth analysis of large data sets can supplement — and in some cases be substituted for — RCTs to support the clinical decision-making process.

That doesn’t imply that RCTs should be abandoned.  The list of treatments that have been supported or retired due to a well-designed RCT is long. For decades, surgeons used radical mastectomy to treat breast cancer until a controlled trial demonstrated that less disfiguring alternatives were just as effective in managing the disease.³ Similarly, clinicians used to freely prescribe hormone replacement therapy to women in menopause until the Women’s Health Initiative, also an RCT, demonstrated that it increases the risk of heart disease, stroke, and breast cancer. But on the other hand, there have been many recent non-RCT investigations that have taken advantage of the power of massive data sets and have generated actionable insights. With the VA Boston Health System, Julia Prentice and her colleagues, using administrative data from more than 80,000 veterans, have shown that among patients with Type 2 diabetes, sulfonylurea drugs increased the risk of dying or being hospitalized when compared to patients on thiazolidinediones.⁴ Similarly, David Graham created a stir when he analyzed the patient records of approximately 1.4 million patients who belonged to Kaiser Permanente in California. They aimed to determine if rofecoxib (Vioxx) increased the risk of acute myocardial infarction and sudden cardiac death. Graham et al. reviewed the equivalent of 2,302,029 person-years of follow-up. They detected 8,142 cases of serious coronary heart disease (CHD), 2,210 of which were fatal. The odds of developing CHD among patients taking any dose of the medication were 59% greater than it was among controls. Among patients who took high doses, namely more than 25 mg daily, the odds of heart disease were 258% greater. ⁵

More recently, nference, a data analytics firm with a partnership with Mayo Clinic, spearheaded several data-intensive studies that did not use the traditional clinical trials protocol. One study used deep neural networks to evaluate 15.8 million clinical notes in an EHR from over 30,000 patients who underwent COVID-19 diagnostic testing.⁶ When investigators compared patients with clinically apparent COVID-19 with negative patients about a week before they had PCR testing to confirm the diagnoses, they found loss of taste and smell was more than 37-fold more likely to occur in those whose infection was confirmed versus those who tested negative. Shweta et al. state, “This study introduces an augmented intelligence platform for the real-time synthesis of institutional knowledge captured in EHRs.” One caveat that the researchers acknowledge in the report was that they had yet to conduct prospective validation of the augmented EHR curation approach.

A second nference-based investigation reviewed the records of patients who had received more than 94,000 doses of the Pfizer COVID-19 vaccine, more than 36,000 doses of the Moderna vaccine, and 1,745 doses of the Johnson & Johnson vaccine. The study’s goal was to determine the incidence of cerebral venous sinus thrombosis (CVST), which has been reported in a small number of patients after receiving one of the vaccines.⁷ The preprint study found no significant association between any of the vaccines as CVST.

One of the strengths of RCTs is their prospective nature, a design that is more likely to eliminate confounding variables and bias when compared to retrospective studies. But at the same time, several RCTs have fallen short because they were underpowered, resulting in false-negative results. Also, RCTs are expensive and often require many years to generate results that clinicians can use at the bedside. On the other hand, retrospective analyses can generate results much more quickly, and under the right circumstances, can provide actionable insights and inform the clinical decision-making process.

Thomas Frieden, MD, MPH, a former director of the CDC, has pointed out the real-world advantages of retrospective cohort studies, which have been used to assess the prognosis and treatment of various types of cancer. That, in turn, has led to better treatment protocols. Similarly, such cohort studies have successfully been used to evaluate survival among pediatric cancer patients and made clinicians aware of the “increased risk of post-treatment cardiac complications, enabling better clinical care.”⁸ Frieden summed up the controversy this way, “Elevating RCTs at the expense of other potentially highly valuable sources of data is counterproductive. A better approach is to clarify the health outcome being sought and determine whether existing data are available that can be rigorously and objectively evaluated, independently of or in comparison with data from RCTs, or whether new studies (RCT or otherwise) are needed.”

When comparing research methodologies, it’s important to remember that’s it’s not a sports competition; there doesn’t have to be a clear winner and loser. Big data analytics and RCTs each have their strengths and weaknesses and can be deployed accordingly. When there's enough time and resources available to conduct a controlled trial, it is often the best way to evaluate potentially useful treatment approaches. Still, when clinicians need to quickly make diagnostic and therapeutic decisions, especially during an international crisis, we don't always have the luxury of time.

 

References

1. Cerrato P, Halamka J. Realizing the Promise of Precision Medicine. 2017, Academic Press/Elsevier, Cambridge, MA, pp. 87-91.

2. Cerrato, P, Halamka J. The Transformative Power of Mobile Medicine. 2019, Academic Press/Elsevier, Cambridge, MA, pp 57-58.

3. Treasure T, Takkenberg JM. Randomized trials and big data analysis: we need the best of both worlds. Eur J CardioThoracic Surg. 2018; 53:910-914.

4. Prentice JC, Conlin PR, Gellad WF et al. Capitalizing on Prescribing Pattern Variation to Compare Medications forType2Diabetes. Value in Health. 2014; 17:854-862.

5. Graham DJ, Campen D, Hui R, et al. Risk of acute myocardial infarction and sudden cardiac death in patients treated with cyclo-oxygenase 2 selective and nonselective non-steroidal anti-inflammatory drugs: nested case-control study. Lancet 2005;365:475–581.

6. Shweta F, Murugadoss K, Awasthi S el al. Augmented Curation of Unstructured Clinical Notes from a Massive EHR System Reveals Specific Phenotypic Signature of Impending COVID-19 Diagnosis. eLife. Published online July 7, 2020. https://elifesciences.org/articles/58227

7. Pawlowski C, Rincon-Hekling J, et al. Cerebral venous sinus thrombosis (CVST) is not significantly linked to COVID-19 vaccines or non-COVID vaccines in a large multi-state US health system. medRxiv. 2021, April 23. https://www.medrxiv.org/content/10.1101/2021.04.20.21255806v1

8. Frieden TR. Evidence for Health Decision Making —Beyond Randomized, Controlled Trials. N Engl. J Med.2017;377:465-475.