In March of 2019, scientists reported a somber discovery.
After compiling data taken from all over the world, the researchers found that killer fungi known as chytrid had caused declines in at least 501 species of amphibian. Worse still, 90 of the species affected had been wiped out entirely—driven extinct or to population levels so low that scientists can no longer find any trace that they still exist. Chytrid was even described as the “most destructive pathogen” to biodiversity ever.
But today, another group of scientists has called those findings into question. In what’s known in the industry as a technical comment, also published by Science, the researchers argue that there are many gaps in the study’s data set. What’s more, when they tried to reproduce the study’s results using the given data, they could not.
“When this paper first came out, I was super excited to read it,” says Max Lambert, a conservation biologist at the University of California, Berkeley and the lead author of the comment. “In population biology, it’s extraordinarily hard to monitor population statuses, let alone detect a trend of a decline, and then to know the exact cause. So, I was actually pretty stoked that they had that sort of rich data for that many species.”
But when he opened up the data, Lambert says he was baffled to find blank cells pretty much everywhere he looked. To try to understand what he was seeing, he asked colleagues and other experts to take a look, and within minutes, “everyone had found dozens and dozens of missing data and problematic data,” he says.
“It just seemed like it was rampant everywhere,” says Lambert.
The scientific process
It’s not unusual for scientists to disagree with each other. Debate and criticism are healthy parts of the scientific process. And in that spirit, Science has allowed the paper’s original authors to provide a technical response to the comment, also published today.
Technical comments and responses are published when reviewers believe further discussion would be valuable for the community, a spokesperson for Science wrote in an emailed statement. They “do not represent a statement of error from the journal,” it says.
“We’re very clear in our original paper. Many of these species have multiple causes of decline,” says Ben Scheele, a conservation biologist at the Australian National University and lead author of the paper in question. “And so we never say that chytrid is the sole cause of decline for those 501 species. We just say chytrid is implicated in the decline of those species. And we have, I think, it's 454 references. So there is a huge body of information that we cite.”
Also at issue though, is the quality of the sources Scheele and his coauthors cite. In many cases, the only evidence for the decline or disappearance of a species of frog or salamander is the testimony of an expert. And while expert opinion is clearly integral to scientific understanding, it’s not as rigorous as, say, a detailed study that tests a hypothesis.
Scheele chalks up the reliance on expert opinion to the fact that scientists are very much playing catch-up with the chytrid fungus. “We’re looking at declines described from the 1970s, and ‘80s, and through to the 1990s,” he says.
For instance, the strain of fungi that primarily infects frogs, now known as Batrachochytrium dendrobatidis or Bd, wasn’t even described until 1998. What’s more, he believes his paper’s findings offer “a conservative estimate.” In other words, he says it’s likely many more amphibians are imperiled even than those he documented. (Learn more about the amphibians at risk of extinction because of human impact.)
The disagreement highlights a problem in the way science is published, one that is not often talked about, says Jonathan Kolby, a National Geographic Explorer and technical consultant for the Convention on International Trade in Endangered Species of Wild Fauna and Flora (CITES), the treaty that regulates cross-border wildlife trade. Kolby is also one of Scheele’s co-authors.
That’s the fact that he’s personally witnessed many declines, but doesn’t always have the long-term population surveys to back up these observations. To his knowledge, there are currently no publishing outlets interested in these “non-data heavy accounts,” despite their obvious value.
“For example, there is one man who legally lives in the core zone of the rainforest at my field site in Cusuco National Park in Honduras,” wrote Kolby in an email. “He’s lived there his whole life and is now in his 80s. He told us that the climate and the species in this forest have changed a lot in his lifetime, and many of the once common frogs have disappeared. Albeit imperfect data, these kinds of important observations are not being recorded in the amphibian literature in a uniform fashion.”
Why it matters
To be clear, Lambert and his coauthors aren’t trying to say chytrid fungus, which eats away at amphibians’ skin, is a non-issue.
“It definitely is a big, bad thing,” says Lambert. (Read about the ripple effects of chytrid on tropical snakes.)
But how big and how bad are questions that are critical to the conservation of frogs and salamander species that remain, argue the commenters. Simply put, if funding and effort go into fighting chytrid for an amphibian that would be better served by focusing on things like habitat loss, then the misdiagnosis might actually stifle conservation efforts.
“It’s kind of like the COVID-19 case we have going on,” says Lambert. “If you go to the doctor, and the doctor says, ‘Oh, you’ve got flu,’ but you really have COVID-19, you're going to be in trouble.”
For her part, comment coauthor Priya Nanjappa says Scheele, et. al.’s published response feels like they’re doubling down rather than addressing the concerns about data quality.
“Bottom line for me is that if you're the only ones who can reproduce your results, then there's a problem with your methods, your data, or your explanations thereof,” says Nanjappa, who is director of operations at Conservation Science Partners, a nonprofit conservation organization.
“Science is under fire right now in the world, especially here in the U.S.,” says Nanjappa. “And when we can't even look at our own work and be able to accept certain flaws or at least opportunities for improvement, then we're not doing science any favors.”
‘The sad state of our knowledge’
Joyce Longcore, a mycologist who has studied the chytrid fungus extensively, says she’s worked with and has respect for people on both sides of the table.
“The use of concrete, replicable evidence in science is important but perhaps not straightforward when dealing with a pandemic that was at its height before the cause was identified,” says Longcore, who was part of neither the study nor comment.
James Gibbs, a conservation biologist at the State University of New York College of Environmental Science and Forestry, says he’s weighed in on controversies like this before in the amphibian world. When debate turns to formal comments and responses, “there are merits on both sides,” he says.
For instance, Gibbs called the original study “well-analyzed and framed,” but the uncertainty in some of their data is also evident. The real problem? There are plenty of well-established ways Scheele and his coauthors could have better acknowledged that uncertainty and more meaningfully built it into their estimations.
In a followup email, Scheele and his coauthors contend they did make an effort to address that uncertainty. And so the debate over methods goes on.
“This interchange reflects to me two groups of scientists trying to deal with the sad state of our knowledge about the status of biodiversity,” says Gibbs. “We all feel the need to try understand what is transpiring with the amphibian decline crisis, but there are real limits to what we know and what we can say.”
Editor's note: This story was updated on March 20, 2020, to include a new comment from Ben Scheele about the way uncertainty was built into the original research.