Can a new test be trusted?
#test #trusted Welcome to InNewCL, here is the new story we have for you today:
Click Me To View Restricted Videos
February 17, 2023 – In January 2022, Anthony Arenz, a 51-year-old resident of Mesa, AZ, breathed a sigh of relief.
The blood test known as the Galleri test, which checks for 50 types of cancer, had found no positive signs.
It would be welcome news for anyone, but especially for a firefighter whose job puts them at a 9% higher risk of developing cancer and 14% higher risk of dying from it than the average person. The Mesa unit had already lost two soldiers to cancer in the past 3 years. Both were more than a decade younger than Arenz.
But when the city of Mesa offered more free exams — including a full-body MRI for firefighters over 50 — Arenz initially shrugged. With a negative Galleri test in hand, he didn’t want to spend any more time thinking about it.
Still, he felt creeping guilt for skipping a test that many of his fallen peers had not been offered. He tried to ease his anxiety with research. A look at the company’s website did not reassure him. According to Grail Bio, makers of the Galleri test, a “No Cancer Signal Detected” result does not rule out cancer.
Arenz booked his free MRI.
The results left him heavy: Stage I kidney cancer. The Galleri test had missed it.
Arenz received his free Galleri test as part of a cancer screening program funded by the City of Mesa at Scottsdale’s Vincere Cancer Center. Led by radiation oncologist and Vincere co-owner Vershalee Shukla, MD, the program screens first responders in more than 10 Arizona cities at no cost to them.
Vincere started using Galleri shortly after the consumer trial launched in June 2021. Since then, the first responder program has become the largest commercial user of the test in North America.
But Galleri’s ability to detect cancer and, perhaps more importantly, the consequences of false results have come under close scrutiny since the test started. Galleri, which has not yet been approved by the FDA, is so new that few know what erroneous results look like in real life and how often they can occur.
“Good” but “not ready yet”
After running the test on about 2,000 first responders, Shukla can offer some insight into the real-world value of the test in a high-risk population.
“Cancer screening is a very complicated subject,” she says. And “to be honest, the tests are good, but not ready yet [for wider use].”
Arenz wasn’t the only firefighter to get a surprise after a Galleri test.
In nearby Phoenix, 51-year-old Mike Curtis knew his cancer risk was high, but he wasn’t as concerned. Curtis had been involved in fires since he was 17, and his father, also a firefighter, had died of cancer at the age of 58.
He had used every Vincere Cancer Center free screening service since the program began in late 2018 — long before Shukla started using Galleri in 2021. His last lung scan was clear. But he took the Galleri test just to stay alert.
His result came as a shock. The test detected signs of cancer.
Curtis decided not to tell anyone, not even his wife. He would bear the bad news alone until he was sure.
However, Shukla immediately doubted the blood test results. She rushed to conduct several follow-up examinations. A week and scans of Curtis’ abdomen and pelvis later, her suspicions were confirmed. The Galleri test was wrong, Curtis didn’t have cancer.
The price of his peace of mind: a major overhaul with a $4,000 price tag. Luckily, the screening program covered the bill.
Overall, in just over 18 months of using the blood test, Shukla has encountered just one other false-positive result out of about 2,000 Galleri results.
She also discovered two positive signals for cancer with Galleri, which were confirmed by follow-up tests. One was chordoma, a rare type of bone cancer, and the other was squamous cell carcinoma of the head and neck. Galleri caught both of them remarkably early, in time for treatment.
But for Shukla, false negatives were particularly “terrible.” Arenz was just one of 28 types of cancer missed on blood tests. And with 500 negative tests yet to be validated, the 28 false negatives may be an underestimate.
In her experience, the binary test result — a simple positive or negative cancer signal — is an oversimplification of risk, she says. There “is a misperception that you may or may not have cancer” when the test itself is not definitive.
Grail Senior Medical Director Whitney Jones, MD, agrees that the test is not intended to be used as a standalone method in cancer screening. Galleri’s goal is to “complement other shows, not replace them,” he says.
Based on an analysis of the Galleri data and Shukla’s experience, the specificity of the test was over 99%. This means that the test successfully minimizes false alarms.
But the sensitivity of the test was much lower. Shukla’s data on first responders showed a sensitivity of 6.7%. That means the test misses about 93 out of 100 cancers. According to the latest data from Grail in more than 6,300 people over the age of 50, the sensitivity of the test was 29%.
Specificity and sensitivity are metrics used to qualify a test and demonstrate confidence in its ability to detect the target disease. A test with high specificity can correctly identify patients without the disease in question, while a test with high sensitivity can correctly identify patients with the disease. But there are tradeoffs between sensitivity and specificity. One value is increased at the expense of the other.
It’s normal for a cancer screening test to prioritize specificity, says Dr. Aparna Parikh, an oncologist at Massachusetts General Hospital Cancer Center in Boston. In a test like Galleri, to be used with other screening tests, “we see at least good specificity, which is important because we don’t want false positives where the downstream patient impact can be high. “
Overall, Jones says, Grail Bio’s goal is to develop a test sensitive enough to detect the most dangerous types of cancer without flooding the healthcare system with false positives. In addition, he says, sensitivity varies by cancer type and tends to be lower in cancers for which other available screening tests already exist, as well as early-stage diseases.
But the Galleri sensitivity readings are “a bit scary,” says Ji-Hyun Lee, public health physician and director of the Division of Quantitative Sciences at the University of Florida Health Cancer Center. Lee, who is not affiliated with Grail, reviewed the company’s publicly available data and Shukla’s data for WebMD.
While there is no definitive threshold for sensitivity, failure rates of 93% and 71% “provide little confidence in the [accuracy of the] testing,” says Lee.
However, positive and negative predictive values are more clinically relevant measures of a screening test. These numbers provide information about how likely a patient’s results are to be true and how concerned they should be about a positive result and how much they should trust a negative one.
In first responders, Shukla found that only half of the positive Galleri tests were confirmed cancer cases. And an analysis of Grail’s data found that just 38% of positive Galleri tests – 35 out of 92 people – had a validated cancer diagnosis.
In the general population, when only 38% of positive Galleri results actually have cancer, the test “isn’t very useful in making a decision for the patient or providers,” Lee says.
Galleri can also be a costly prospect for patients, regardless of the outcome, says Electra Paskett, PhD, an epidemiologist and cancer screening expert at Ohio State University. A positive Galleri test requires a cascade of follow-up diagnostic tests with no promise of payer coverage. For a negative result, Galleri recommends that the patient be re-examined in a year, at an annual cost of $950 plus any follow-up examinations when Galleri picks up anything.
“If a vendor wants to offer the Galleri test, I think all of these things need to be made clear,” says Paskett.
After a negative Galleri test, Arenz’s cancer didn’t slip through the cracks because he received other advanced imaging for free. But whether all physicians will make such efforts to support the Galleri findings, even in negative patients, is unknown.
A negative result can give patients “a huge false sense of security,” Shukla says. And when a test is positive, the workup isn’t easy, she says. Chasing after the cancer, especially when it’s not really there, can be nerve-wracking and expensive.
So the question is: why do the galleries test at all when they require so much validation?
According to Parikh, a high-risk group like firefighters is an ideal use case for galleri and other liquid biopsy tests. But, she says, if it were more widespread in the general population, she would be “cautious about the system’s ability to administer this test en masse.”
According to Shukla, it’s less about the results she’s getting today and more about making the test more effective for her patients in the future. First responders need a test like this that can quickly identify multiple types of cancer, she says. But to improve it, Grail needs more data in this high-risk population. That’s what she’s all about.
Curtis has no regrets about the Galleri test. The emotional cost of thinking he had cancer for a few days wasn’t a big price, he thought. It’s part of cancer screening.
But he says it would have been a much more distressing experience if he had been financially responsible for the workup or if he hadn’t had Shukla to manage his case from start to finish.
Because it was free, Arenz Galleri has no regrets either. But he is urging his colleagues to check the site, do some research and get more screening.
“In any medical center that’s just doing this one test, you just have to be careful,” Shukla says. “It’s not that easy.”
