A reality check on antibody testing: How do we race forward thoughtfully?
WASHINGTON, DC (ABC News) — As U.S. government officials begin debating the right steps in easing restrictions due to the novel coronavirus, antibody tests have been touted as the key to returning to normal.
This particular type of test can detect whether or not a person has developed antibodies against COVID-19, which would reveal whether a person was infected and recovered — even if they did not exhibit symptoms. Officials hope to use antibody testing to gain more clarity on the spread and deadliness of the virus. Others hope to determine the number of individuals that potentially have some level of immunity, toying with the idea of issuing “immunity passports” for people who test positive.
Some experts now warn that these promises are premature. The World Health Organization cautioned against instituting “immunity passports,” saying there is insufficient evidence that people who have recovered from COVID-19 are protected from a second infection.
Amid the haste to develop antibody testing, we may be setting ourselves up for disaster.
“Testing is not a panacea. Testing is a tool and no test is perfect. What people are looking for does not exist,” said Dr. Alan Wells, executive vice chairman of the section of Laboratory Medicine at University of Pittsburgh Medicine.
In an effort to rapidly expand access to antibody testing, the Food and Drug Administration has allowed companies to develop and distribute COVID-19 antibody tests so long as they provide the agency proof they have determined the test is accurate — a process called “validation” — and disclose on the packaging that their test has not been FDA authorized. So far only seven COVID-19 antibody tests have been issued an emergency authorization from the FDA.
As government officials pushed for the importance of antibody testing, biotechnology companies followed suit, accelerating the development and distribution of COVID-19 antibody tests with little oversight or transparency. With the FDA opening the gates to a flood of unreviewed antibody tests, more than 100 companies, many based in China, are ramping up distribution in the U.S. and experts are sounding the alarm.
“The FDA is highly remised in their handling of the antibody testing,” said Wells. “Early on they have been accused of squelching PCR (polymerase chain reaction) test development by being too tight in controlling and now the pendulum has swung too far the other way.”
The federal guidance that does exist confuses things even more, according to Wells, who added, “The FDA doesn’t even say what validation is.” Different tests “are coming out with different amounts of data, different levels, huge gaps and frankly some even pretty unbelievable results.”
Many of these antibody tests are similar to pregnancy tests, but requiring a prick of blood to rapidly detect whether or not the individual has antibodies present by showing simple colored lines.
“There’s a tradeoff between speed and accuracy, and some of these tests are claiming speed and accuracy at levels we haven’t seen before,” he added.
Rapid diagnostic tests such as these hold the potential for quick answers that may help health care providers make efficient decisions regarding treatment during moments when time is more precious than ever. But some have mistakenly utilized these rapid antibody tests as diagnostic tools, when results provide very limited information.
“The important consideration with antibody tests is that many people take up to two weeks or more to develop the antibodies in response to having the infection so they could actually turn out to be negative on the antibody tests, but actually have had the infection,” Dr. Mike Ryan, executive director of the WHO’s Emergencies Program, said in a recent briefing. If the health care system does not carefully introduce validated rapid antibody tests in a larger, well laid out comprehensive plan, he warned that may lead to “unnecessary confusion.”
Government reaction
The FDA responded to the growing confusion over COVID-19 antibody testing, warning health care providers to “be aware of their limitations.” In the same statement, the agency said it is bolstering their surveillance of these tests, working with the National Institutes of Health and Centers for Disease Control and Prevention on a validation project to help identify the most promising antibody tests. But this timeline remains unclear and may not respond quickly enough to the growing number of antibody tests already saturating the market.
Senior officials from the FDA told ABC News they recognize there is a large ecosystem of antibody testing and providers are hungry to use them and in response want to provide much-needed clarity. The FDA said through its collaboration with the NIH and CDC it will physically validate the tests rather than review paperwork received by companies applying for the emergency use authorization. Generally the tests it is reviewing have been volunteered by the companies.
But some members of the government believe these efforts fall short. In a new Democratic staff memo, Illinois Rep. Raja Krishnamoorthi, the chairman of the Subcommittee on Economic and Consumer Policy, wrote, “Companies are ignoring requests from HHS to voluntarily submit their tests for validation.” And later added that the lack of clarity and series of “unclear clarifications” puts the “public’s health at risk by allowing potentially fraudulent tests to spread unchecked.”
In response to this recent staff memo FDA Commissioner Stephen Hahn said at the White House briefing on Friday, “We provide flexibility.” Hahn said the FDA, with the CDC and NIH will work to provide “as much information as we possibly can” about these tests’ efficacy.
The FDA, according to the memo, also told the committee that it has not yet taken formal action against companies falsely marketing tests as FDA approved and for at-home use.
Overpromise and underdeliver
Some scientists, however, say that beyond companies knowingly scamming consumers, even the antibody tests manufactured by larger companies with fairly reputable histories overpromise and underdeliver.
“There is just so much variety in these assays out there, and we too have seen some tests to have a very high level of false positive results using samples that were collected prior to the outbreak,” said Eliza Theel, Ph.D., director of Mayo Clinic Infectious Diseases Serology laboratory testing.
Another major issue experts cite is the lack of standardized validation protocols. “You can cherry pick what your controls are,” said Wells.
Validating tests with the right samples is essential in order to minimize the potential of false positives and negatives.
“False positive means that the antibody reaction detected an antibody, but from some other coronavirus or some other related infection,” according to Dave Koch, Ph.D., director of clinical chemistry, toxicology and point-of-care testing at Grady Memorial Hospital in Atlanta. “There is also possibility of a false negative: I actually have the virus but the antibody hasn’t shown up yet or hasn’t gotten to the detection limits in the bloodstream yet.”
The currently available antibody tests have a 87% to 93% range in sensitivity and 95% to 100% range of specificity, according to a recently published Johns Hopkins report. Sensitivity and specificity refer to the accuracy of the test in terms of ability to correctly identify positive and negative results. In general specificity and sensitivity are inversely related, decreasing the threshold for positive results may increase the likelihood of returning false positives, and vice versa.
Dr. James Baker, an immunologist at University of Michigan who specializes in diagnostic laboratory immunology and is currently evaluating the accuracy of multiple COVID-19 antibody tests in order to help distributors submit EUA packages to the FDA, said that no test is perfect, however, an acceptable test would demonstrate mid-90% sensitivity and over 90% specificity for the particular antibody in question. Although these numbers appear pretty high when officials start mass testing that could result in thousands of people receiving incorrect results.
Baker believes the distribution of faulty tests is immoral: “When I see things like this I find it very disconcerting; people should not be doing this. It’s not appropriate.”
Concerns over false positive and negative tests were voiced by Dr. Demetre Daskalakis, deputy commissioner of the New York City Department of Health, within hours of New York Gov. Andrew Cuomo releasing the results of a statewide antibody sampling study this week. The results estimated that almost 14% of the New York state residents tested positive for COVID-19 antibodies. New York City had the highest estimated positive results at approximately 21%. Daskalakis warned of potentially misleading estimates due to “significant voids” in antibody testing.
Florian Krammer, a professor of vaccinology, who has spearheaded Mount Sinai Hospital’s COVID-19 antibody testing initiative and convalescent plasma clinical trials, echoed these sentiments. In a recent tweet, Krammer wrote, “But a 20% plus infection rate seems too high for NYC due to a number of reasons. I would think 6-8%, maybe 10% are closer to the truth.” Like Daskalakis, he had questions regarding specificity and sensitivity of these tests.
Maria Van Kerkhove, the technical lead on the pandemic for the WHO, cautioned against reading too much into these results in a recent briefing. “We need to put these into context over a large number of studies across multiple countries,” she said.
Interpreting results
Beyond the actual reliability of these tests there remain major gaps in our understanding of antibodies, which can affect the way we interpret the results. Experts still don’t know for certain when the antibodies will appear and whether or not they will reach detectable levels. Moreover, they are still not certain whether or not the presence of antibodies indicates any form of protection from future infection.
Even with laboratory-based antibody tests, which experts say are more reliable and quantify the level of antibodies present unlike rapid assays, questions regarding how to clinically interpret the data persist.
“What we do in the laboratory is provide objective evidence, but the antibody — even though it will get measured properly — we just don’t know what that means in a clinic sense,” said Koch.
Major research groups are stepping up. Foundation for Innovative New Diagnostics, a global nonprofit research organization, is collaborating with the WHO to develop a standardized protocol for evaluating COVID-19 tests. They hope to collaborate with other federal bodies and intend to openly share their data.
Dr. Rangarajan Sampath, FIND’s chief scientific officer, said he was “shocked” to see large governments, such as Spain, prematurely buying thousands of antibody tests without any independent evaluations. Only after purchasing the tests from the Chinese manufacturer and running their own evaluations did they learn the tests were approximately 30% accurate, according to Caixin Global reporting.
He hopes that FIND’s works provides a centralized resource of much-needed information that would provide insight on antibody test performance globally.
Sampath admitted that it takes time to set up a standardized quality control program for antibody tests for a novel virus like COVID-19. Scientists need more data and that data is simply not available yet.
“It will happen, but it doesn’t happen overnight.” he said. Sampath advised “not to classify antibody tests as good or bad. There’s more in understanding the context in which they are used and how we think of using them. Understand the product in front of you and know how to interpret the results.”
Leave a Reply