NOAA’s ‘Billion-Dollar Disasters’ Report: An Egregious Failure Of Scientific Integrity

On Tuesday, the U.S. National Oceanic and Atmospheric Administration (NOAA) will release with great fanfare the year-end update of its “billion-dollar disaster” tally.

If the past is prologue, NOAA will vigorously promote the dataset in collaboration with environmental NGOs, reporters on the climate beat will uncritically parrot and amplify NOAA’s claims, and before long, the dataset will find itself cited in the peer-reviewed literature, identified by the U.S. government as a key indicator of human-caused climate change, and perhaps even cited by the U.S. president in support of the claim that all U.S. disaster costs are attributable to climate change. [emphasis, links added]

What you will not see is any scientific critique, evaluation, or independent peer review of the dataset. 1

That is, until now.

Today, I am very happy to share with you a new preprint of a paper that I have just submitted to the new Nature journal, npj Natural Hazards.

My paper, invited by the journal’s editors, is titled, Scientific Integrity and U.S. “Billion-Dollar Disasters.”

Here is the abstract:

For more than two decades, the U.S. National Oceanic and Atmospheric Administration (NOAA) has popularized a count of weather-related disasters in the United States that it estimates have exceeded one billion dollars (inflation adjusted) in each calendar year starting in 1980.

The dataset is widely cited and applied in research, assessment and invoked to justify policy in federal agencies, Congress and by the U.S. President. This paper performs an evaluation of the dataset under criteria of procedure and substance defined under NOAA’s Information Quality and Scientific Integrity policies.

The evaluation finds that the “[billion-dollar] disaster” dataset falls comprehensively short of meeting these criteria. Thus, public claims promoted by NOAA associated with the dataset and its significance are flawed and misleading. Specifically, NOAA incorrectly claims that for some types of extreme weather, the dataset demonstrates detection and attribution of changes on climate timescales.

Similarly flawed are NOAA’s claims that increasing annual counts of [billion-dollar] disasters are in part a consequence of human caused climate change. NOAA’s claims to have achieved detection and attribution are not supported by any scientific analysis that it has performed. Given the importance and influence of the dataset in science and policy, NOAA should act quickly to address this scientific integrity shortfall.

The full submission will soon appear at Nature’s preprint platform, Research Square, and meantime I have provided a link to the PDF. 2

My paper is about scientific integrity, and not about the reality or importance of climate change — as I write:

The point here is not to call into question the reality or importance of human-caused climate change – it is real, and it is important.

Just because climate change is important does not mean that scientific claims related to climate change get a free pass or are somehow insulated from scrutiny.

On the contrary, it is because of the importance of climate change in policy and politics that scientific claims related to climate change should adhere to the highest scientific standards.

Each of us has a responsibility to ensure that happens. Far too often, that is not the case.

You can explore the details of the evaluation in the paper, and here is the bottom line:

The NOAA [billion-dollar] disaster dataset comprehensively falls short of NOAA’s guidelines for scientific integrity. The shortfalls documented here are neither small nor subtle. They represent a significant departure from NOAA’s long-term history of scientific integrity and excellence, which has saved countless lives and facilitated the nation’s economy. A course correction is in order.

I conclude the paper with recommendations for that course correction. This is not rocket science — these recommendations could be implemented easily today or tomorrow.

Here is how the paper ends:

NOAA is a crucially important agency that sits at the intersection of science, policy and politics. It has a long and distinguished history of providing weather, climate, water, ocean and other data to the nation. These data have saved countless lives, supported the economy and supported significant scientific research. The agency is far too important to allow the shortfalls in scientific integrity documented in this paper to persist. The [billion-dollar] disaster dataset is an egregious failure of scientific integrity. However, science and policy are both self-correcting.

You can download a PDF of the submission here. Note that it has not yet gone through peer review. I welcome and invite scrutiny, critique, and comment. Let’s “steel man” this thing.

The Honest Broker is written by climate expert Roger Pielke Jr and is reader-supported. If you value what you have read here, please consider subscribing and supporting the work that goes into it.

Read rest at The Honest Broker