It’s OK to Break the Rules Now and Then

Technological innovation sometimes requires we take risks — and question the tenets of evidence-based medicine.

John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, wrote this article.

It’s challenging at times to know when to follow the rules and “color inside the lines” and when to ignore those lines and forge ahead. That’s true whether we’re navigating everyday life, creating new technology, or devising the best patient care initiatives. Which brings to mind a quote from Elbert Hubbard: “The greatest mistake you can make in life is continually fearing that you'll make one.” 

Over the years, we have discussed the strengths and weaknesses of evidence-based medicine and randomized controlled trials (RCTs) in several books and articles, our point being that fear of investing in a treatment approach because it not supported by the RCT “gold standard” can create a kind of inertia that ultimately hurts patients.^1,2  And even if we put aside the fear factor that Hubbard mentions, mounting evidence strongly suggests that an in-depth analysis of large data sets can supplement — and in some cases be substituted for — RCTs to support the clinical decision-making process.

That doesn’t imply that RCTs should be abandoned.  The list of treatments that have been supported or retired due to a well-designed RCT is long. For decades, surgeons used radical mastectomy to treat breast cancer until a controlled trial demonstrated that less disfiguring alternatives were just as effective in managing the disease.³ Similarly, clinicians used to freely prescribe hormone replacement therapy to women in menopause until the Women’s Health Initiative, also an RCT, demonstrated that it increases the risk of heart disease, stroke, and breast cancer. But on the other hand, there have been many recent non-RCT investigations that have taken advantage of the power of massive data sets and have generated actionable insights. With the VA Boston Health System, Julia Prentice and her colleagues, using administrative data from more than 80,000 veterans, have shown that among patients with Type 2 diabetes, sulfonylurea drugs increased the risk of dying or being hospitalized when compared to patients on thiazolidinediones.⁴ Similarly, David Graham created a stir when he analyzed the patient records of approximately 1.4 million patients who belonged to Kaiser Permanente in California. They aimed to determine if rofecoxib (Vioxx) increased the risk of acute myocardial infarction and sudden cardiac death. Graham et al. reviewed the equivalent of 2,302,029 person-years of follow-up. They detected 8,142 cases of serious coronary heart disease (CHD), 2,210 of which were fatal. The odds of developing CHD among patients taking any dose of the medication were 59% greater than it was among controls. Among patients who took high doses, namely more than 25 mg daily, the odds of heart disease were 258% greater. ⁵

More recently, nference, a data analytics firm with a partnership with Mayo Clinic, spearheaded several data-intensive studies that did not use the traditional clinical trials protocol. One study used deep neural networks to evaluate 15.8 million clinical notes in an EHR from over 30,000 patients who underwent COVID-19 diagnostic testing.⁶ When investigators compared patients with clinically apparent COVID-19 with negative patients about a week before they had PCR testing to confirm the diagnoses, they found loss of taste and smell was more than 37-fold more likely to occur in those whose infection was confirmed versus those who tested negative. Shweta et al. state, “This study introduces an augmented intelligence platform for the real-time synthesis of institutional knowledge captured in EHRs.” One caveat that the researchers acknowledge in the report was that they had yet to conduct prospective validation of the augmented EHR curation approach.

A second nference-based investigation reviewed the records of patients who had received more than 94,000 doses of the Pfizer COVID-19 vaccine, more than 36,000 doses of the Moderna vaccine, and 1,745 doses of the Johnson & Johnson vaccine. The study’s goal was to determine the incidence of cerebral venous sinus thrombosis (CVST), which has been reported in a small number of patients after receiving one of the vaccines.⁷ The preprint study found no significant association between any of the vaccines as CVST.

One of the strengths of RCTs is their prospective nature, a design that is more likely to eliminate confounding variables and bias when compared to retrospective studies. But at the same time, several RCTs have fallen short because they were underpowered, resulting in false-negative results. Also, RCTs are expensive and often require many years to generate results that clinicians can use at the bedside. On the other hand, retrospective analyses can generate results much more quickly, and under the right circumstances, can provide actionable insights and inform the clinical decision-making process.

Thomas Frieden, MD, MPH, a former director of the CDC, has pointed out the real-world advantages of retrospective cohort studies, which have been used to assess the prognosis and treatment of various types of cancer. That, in turn, has led to better treatment protocols. Similarly, such cohort studies have successfully been used to evaluate survival among pediatric cancer patients and made clinicians aware of the “increased risk of post-treatment cardiac complications, enabling better clinical care.”⁸ Frieden summed up the controversy this way, “Elevating RCTs at the expense of other potentially highly valuable sources of data is counterproductive. A better approach is to clarify the health outcome being sought and determine whether existing data are available that can be rigorously and objectively evaluated, independently of or in comparison with data from RCTs, or whether new studies (RCT or otherwise) are needed.”

When comparing research methodologies, it’s important to remember that’s it’s not a sports competition; there doesn’t have to be a clear winner and loser. Big data analytics and RCTs each have their strengths and weaknesses and can be deployed accordingly. When there's enough time and resources available to conduct a controlled trial, it is often the best way to evaluate potentially useful treatment approaches. Still, when clinicians need to quickly make diagnostic and therapeutic decisions, especially during an international crisis, we don't always have the luxury of time.

 

References

1. Cerrato P, Halamka J. Realizing the Promise of Precision Medicine. 2017, Academic Press/Elsevier, Cambridge, MA, pp. 87-91.

2. Cerrato, P, Halamka J. The Transformative Power of Mobile Medicine. 2019, Academic Press/Elsevier, Cambridge, MA, pp 57-58.

3. Treasure T, Takkenberg JM. Randomized trials and big data analysis: we need the best of both worlds. Eur J CardioThoracic Surg. 2018; 53:910-914.

4. Prentice JC, Conlin PR, Gellad WF et al. Capitalizing on Prescribing Pattern Variation to Compare Medications forType2Diabetes. Value in Health. 2014; 17:854-862.

5. Graham DJ, Campen D, Hui R, et al. Risk of acute myocardial infarction and sudden cardiac death in patients treated with cyclo-oxygenase 2 selective and nonselective non-steroidal anti-inflammatory drugs: nested case-control study. Lancet 2005;365:475–581.

6. Shweta F, Murugadoss K, Awasthi S el al. Augmented Curation of Unstructured Clinical Notes from a Massive EHR System Reveals Specific Phenotypic Signature of Impending COVID-19 Diagnosis. eLife. Published online July 7, 2020. https://elifesciences.org/articles/58227

7. Pawlowski C, Rincon-Hekling J, et al. Cerebral venous sinus thrombosis (CVST) is not significantly linked to COVID-19 vaccines or non-COVID vaccines in a large multi-state US health system. medRxiv. 2021, April 23. https://www.medrxiv.org/content/10.1101/2021.04.20.21255806v1

8. Frieden TR. Evidence for Health Decision Making —Beyond Randomized, Controlled Trials. N Engl. J Med.2017;377:465-475.

Can AI Reinvent Radiation Therapy for Cancer Patients?

John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, wrote this article.

Of all the advances in health care artificial intelligence (AI), medical imaging is probably the most remarkable success story. Two prominent examples come to mind: Machine learning has helped improve the screening and diagnosis of retinal disease and is making inroads in skin cancer detection. Given these developments, it’s not surprising to find researchers and clinicians developing the digital tools to improve radiotherapy, which combines imaging technology with high doses of ionizing radiation, delivered through a device called a linear accelerator.

Radiotherapy is one of the most common cancer treatments, used to treat more than half of cancers, yet this labor-intensive expertise is in short supply. ¹  The digital tools can meet unmet patient needs for the treatment and increase the accuracy of the delivered therapy.  

To fully appreciate the impact that AI-enhanced algorithms have on radiotherapy, it helps first to understand how the equipment and technology used to deliver radiation to a patient’s tumor functions. Ionizing radiation achieves its purpose by disrupting cellular DNA, which prevents cancer cells from growing and dividing, which in turn causes solid tumors to shrink in size. Unfortunately, the same radiation that disrupts tumor growth can also have a detrimental effect on nearby healthy tissue, resulting in various of complications.

To minimize this risk, computerized programs are employed to outline all the anatomical structures closest to the tumor to be irradiated so that the electron beam will more precisely target the tumor and spare the healthy tissues — a procedure called contouring. But there is significant disagreement among providers on how to perform the procedure. Diana Lin, with the Department of Radiation Oncology, along with several of her colleagues, point out that such variation is common and “can affect the resulting plan quality and patient outcomes.” ² Their systematic review also found that variations in target volume delineation was responsible for greater treatment toxicity and decreased survival. The medical literature also reveals that major deviations in target delineation occur in up to 13% of radiation therapy plans.

Although computer programs are available to help reduce inconsistencies and improve contouring, these digital tools are far from perfect. Chris Beltran, Ph.D., chair of the Division of Medical Physics at Mayo Clinic, Florida, points out that the relevant organs and tumors “are critical inputs for the computer models that are currently used to generate radiation dose plans. If organs are not properly identified, the radiation plan may not protect these critical structures or adequately treat the tumor.” While this computational modeling reduces the risk to healthy tissue, machine learning is now being investigated to make contouring more accurate.

Mayo Clinic and Google Health recently announced a joint initiative focusing on research into applying AI to radiation therapy planning. Radiation therapy experts from Mayo Clinic, including radiation oncologists, medical physicists, dosimetrists and service design, are collaborating with Google Health’s experts in applying AI to medical imaging. In this first stage of the initiative, Mayo Clinic and Google Health teams are using deidentified data to develop and validate an algorithm to automate the contouring of healthy tissue and organs from tumors and develop adaptive dosage and treatment plans for patients undergoing radiation therapy for cancers in the head and neck area. The goal of the IRB-approved project is to develop an algorithm that will improve the quality of radiation plans and patient outcomes while reducing treatment planning times and improving the efficiency of radiotherapy practice.

Because the head and neck contain several sensitive organs that are in close proximity to one another, the Mayo Clinic/Google project began its investigation in this area of the body. “Radiation oncologists today painstakingly draw lines around sensitive organs like eyes, salivary glands and the spinal cord to make sure radiation beams avoid these areas. And while this works well, it takes a really long time to get it exactly right,” says Cían Hughes, M.B., Ch.B., informatics lead at Google Health. “We see huge potential in using AI to augment parts of the contouring workflow, and hope that this work will ultimately enable a better patient experience and help patients get the treatment they need sooner.”

The potentially revolutionary impact of this new initiative becomes obvious when one considers the fact that virtually all linear accelerators are equipped with an open-source API, which means it may be possible for hospitals around the world to use this new technology to dramatically improve the radiological contouring and making these treatments available to underserved patient populations.

Reference

1. Thomadsen B. The shortage of radiotherapy physicists. J Am Coll Radiol. 2004 Apr;1(4):280-2. doi: 10.1016/j.jacr.2003.12.036. PMID: 17411581.

2. Lin D., Lapen, K, Sherer MV et al. A Systematic Review of Contouring Guidelines in Radiation Oncology: Analysis of Frequency, Methodology, and Delivery of Consensus Recommendations. Int J Radiology Oncol. 2020; 107:828-835.

A Heart Held Humble Levels and Lights the Way

Observations on executive training & executive rest

John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, wrote this article.

Humility is not a very popular word among business and health care executives. Often considered a sign of weakness, this personality trait is not often applauded in executive training programs or boardrooms. A revealing piece in Harvard Business Review sums up the problem in its title: “If Humility is So Important, Why Are Leaders So Arrogant?” The article goes on to discuss the push among HR consulting firms and psychology experts to develop the H Factor, a combination of honesty and humility. Despite this celebration of humility, “it flies in the face of daily headlines in the Wall Street Journal and the realities of our business and political cultures,” says the HBR article.

Several management experts have tried to explain this paradox. Edgar Shein from MIT Sloan School of Management posits that the prevailing mindset about managers is that life is a competition and being a successful leader is all about getting results at all costs, which in turn requires telling others what to do. There’s little room for humility and gentleness in that formula for success.

Which brings us to the blog’s title: a heart held humble levels and lights the way. It’s a quote from Along the Road, a song by Dan Fogelberg. It suggests that informed humility accomplishes two goals: It levels us, i.e., it provides balance in making decisions, and it lights up the path as we move forward to accomplish our mission. At the Mayo Clinic Platform, the pursuit of balance and enlightenment is accompanied by complete transparency about our goals, dreams, and fears. That certainly requires humility.

At some health care organizations, executive coaching is stigmatized. If a leader falters or is assigned to a role beyond competency (the Peter Principle), a coach is assigned. We have a different notion. If we're charged with leading a new team on a new journey with new rules — the COVID new normal — we must embrace the best of what collaborators and partners have to offer. We think bringing on an external coach as a sounding board during the next six months of great change will be empowering to us all. The idea is that we'll meet with the coach twice a month to present our strategy, structure, staffing, and process ideas to benchmark against the experience of high performing organizations and teams. It's likely we'll receive feedback and inspiration that exceeds our own life experiences.  At the same time, we'll understand more about how we can improve efficiency, communications, and decision-making. The process will be very personal, and we’ll have to grow as people and leaders. Our life experience has shaped our personalities and our approach to problems, and although it has served us well in the past, we’ll need to focus on how to we change to lead the team through the challenges ahead. 

Our life experiences have taught us to take accountability for every situation. As we build a scalable platform team, it will be more important to orchestrate and delegate, replacing individual efforts with repeatable processes. Building a sustainable organization that scales from dozens of projects to hundreds requires leadership evolution. Coaching can help with such polishing, especially when working at an accelerated pace. Such coaching is not only seldomly disclosed, but also rarely documented. However, we’ll keep diaries of what we learn each month and how it changes our behavior.  We'll share that broadly. Some may suggest that this exposes our vulnerabilities — that’s a good thing! 

Along the road to informed humility, we also recognize the need for rest. Despite what many executives imagine, the human body and mind are not perpetual motion machines. The workaholic CEO may be admired in much of corporate America, but as health professionals, we know better. The evidence demonstrating the detrimental effects of overwork on the brain and immune system is overwhelming; it would be irresponsible for us to ignore it. The American Institute of Stress calculates that 77% of Americans “regularly experience physical symptoms caused by stress.” The problem is so prevalent in society, there is even a medical specialty devoted to it: psychoneuroimmunology, which rests upon one simple truth: Thoughts have physiological consequences. And ignoring this truth may not have immediate repercussions for executives, but its insidious effects eventually take their toll. Solutions abound: Stress management techniques like mindfulness meditation, walks in the woods, crossword puzzles, long, hot baths, music — each of us responds to different modalities.

After all these decades, Fogelberg’s lyrics still offer sage advice to executives who want to inspire others and serve as role models:

Along the road
Your steps may tumble
Your thoughts may start to stray
But through it all a heart held humble
Levels and lights your way