Today, npj Natural Hazards, a journal in the Nature family of journals, officially published my new paper, “Scientific Integrity and U.S. “Billion Dollar Disasters.” [emphasis, links added]
The paper shows — irrefutably in my view — that the “billion-dollar disaster” tabulation of the National Oceanic and Atmospheric Administration (NOAA), fails to meet the agency’s standards for information quality and scientific integrity.1
For reasons I describe in detail in the paper, the “billion-dollar disaster” tabulation is not suitable as a “database” (scare quotes — it is not data by any standard) for the detection and attribution of trends in extreme weather.
Similarly, the tabulation is not suitable for identifying the consequences of changes or variability in climate on the costs of disasters. The dataset has been widely misused in science, by the media, and in policy.
It is, in a word, misinformation.2
Here is how the paper starts:
In the late 1990s, the U.S. National Oceanic and Atmospheric Administration (NOAA) began publishing a tally of weather and climate disasters that each resulted in more than $1 billion in damage, noting that the time series had become “one of our more popular web pages”1. Originally, the data was reported in current-year U.S. dollars. In 2011, following criticism that the dataset was misleading, NOAA modified its methods to adjusted historical losses to constant-year dollars by accounting for inflation.
By 2023, the billion-dollar disaster time series had become a fixture in NOAA’s public outreach, was highlighted by the U.S. government’s U.S. Global Change Research Program (USGCRP) as a “climate change indicator,” was a cited as evidence in support of a “key message” of the Fifth U.S. National Climate Assessment showing that “extreme events are becoming more frequent and severe.”
The time series is often cited in policy settings as evidence of the effects of human-caused climate change to increase the frequency and intensity of extreme weather events and associated economic damage, including in federal agencies, Congress and by the U.S. President. In addition to being widely cited in justifications of policy, as of March, 2024, NOAA’s billion-dollar dataset has been cited in almost 1,000 articles according to Google Scholar.
NOAA’s “billion-dollar disaster” tabulation began as a simplistic but clever way to market NOAA and to attract the attention of reporters with a clickbaity listicle.
At some point along the way, the “billion-dollar disaster” list was somehow transformed into “data” used in peer-reviewed research, an official indicator of human-caused climate change featured by the U.S. National Climate Assessment, and used by the administration of President Joe Biden to justify a wide range of regulations and policy.
It is a remarkable story of how science can get off track and how misinformation can exist in plain sight, just like the emperor’s new clothes.
Here are some examples of such misinformation in plain sight that I cite in my paper:
- The NOAA official responsible for overseeing the dataset claimed that the dataset showed: “Climate change is supercharging many of these extremes that can lead to billion-dollar disasters.”
- At the press conference where the 2022 dataset was released, the NOAA Administrator claimed that the dataset indicated that, “Climate change is creating more and more intense extreme events that cause significant damage.”
- In 2021 the U.S. Department of Treasury identified increasing billion-dollar disasters as evidence of the effects of climate change on financial risks.
- The Fifth U.S. National Climate Assessment cited the NOAA dataset as evidence that “Climate change is not just a problem for future generations, it’s a problem today,” and claimed that the dataset, in part, demonstrated “the increasing frequency and severity of extreme events” due in part to “human-caused climate change.”
- In 2023, President Biden attributed weather- and climate-related disaster costs in the U.S. in 2022 to climate change, citing the NOAA dataset: “[C]limate change-related extreme weather events still pose a rapidly intensifying threat – one that costs the U.S. at least $150 billion each year … This year set a record for the number of climate disasters that cost the United States over $1 billion. The United States now experiences a billion-dollar disaster approximately every three weeks on average, compared to once every four months during the 1980s.”
The failures of the “billion-dollar disaster” dataset provide a high-profile test of NOAA — and the broader scientific community, the major media, and the Biden Administration — to correct course when science gets off track. I conclude the paper with some specific recommendations:
Identifying the reasons why NOAA’s billion-dollar disaster dataset has departed so significantly from the agency’s own standards of scientific integrity goes well beyond the scope of this paper. However, the steps necessary to bring the dataset back into conformance with NOAA’s information quality criteria are straightforward:
- Publish all data, including all versions of the dataset;
- Document and publish baseline loss estimates and their provenance;
- Clearly describe all methodologies employed to adjust baseline data;
- Document every change made to the dataset, give each successive version of the dataset a unique name, and publish all versions of the data;
- Maintain all historical versions of the dataset in a publicly accessible archive;
- Subject the methods and results to annual peer review by experts, including economists and others with subject matter expertise, who are independent of NOAA. Make the peer review reports public;
- Align NOAA’s practices with federal government policies for disseminating statistical information that are applied to other agencies;
- Align claims with IPCC methods and standards for any claims of detection and attribution, or justify why the claims are at odds with those of the IPCC.
I enjoyed working in a NOAA cooperative institute at the University of Colorado Boulder for 16 years. I have many friends, colleagues, and collaborators in the agency.
NOAA does excellent work — across both research and operations — and every minute of every day contributes positively to making the nation safer and furthering its economy. That makes the agency’s failures on the “billion-dollar disasters” so remarkable and troubling.
The Honest Broker is written by climate expert Roger Pielke Jr and is reader-supported. If you value what you have read here, please consider subscribing and supporting the work that goes into it.
Roger Pielke Jr. has been a professor at the University of Colorado since 2001. Previously, he was a staff scientist in the Environmental and Societal Impacts Group of the National Center for Atmospheric Research. He has degrees in mathematics, public policy, and political science, and is the author of numerous books. (Amazon).
Read rest at The Honest Broker
