Polar Bear Estimations Have Margin Of Errors Too High For Conservation Status

Large margins of error in polar bear population estimates means the conservation status threshold of a 30% decline (real or predicted) used by the US Endangered Species Act and the IUCN Red List is probably not valid for this species.

Several recent subpopulation estimates have shown an increase between one estimate and another of greater than 30% yet deemed not to be statistically significant due to large margins of error.

How can such estimates be used to assess whether population numbers have declined enough to warrant IUCN Red List or ESA protection?

What do polar bear population numbers mean for conservation status, if anything?

Virtually all recent population size estimates for polar bear subpopulations have such enormous margins of error (aka confidence intervals) that even a 42% increase in one population count (details below) wasn’t enough to be statistically significant (Aars et al. 2017).

That means the ESA and Red List definitions of ‘threatened’ with or ‘vulnerable’ to extinction  — based on the likelihood of a population decline of 30% or more over the next three generations — are using a criterion that’s not statistically valid for polar bears.

I discussed this issue of wide margins of error in subpopulation counts a few years ago (in relation to Eastern Beaufort bears and the effects of thick spring ice on population numbers, especially for 2006) but the problem has become more widespread and the implications more problematic, as discussed in my recent State of the Polar Bear Report 2017 (Crockford 2018).

Ultimately, large margins of error for subpopulation estimates threaten to undermine conservation status assessment — or at least they should.

In addition, some consensus polar bear specialists are selectively choosing to ignore statistical significance and methodological differences in population estimates to suit their own agenda, and by doing so, are giving the public misinformation.

Two examples highlight the problems that large margins of error present: the last two Svalbard population survey estimates and the Western Hudson Bay population survey estimates since 1987 (made more complicated by methodological differences).

WESTERN BARENTS SEA (SVALBARD)

The Svalbard portion of the Barents Sea subpopulation was surveyed in 2015 and initially reported to have increased by 42% over the count done in 2004 (“Polar bears make a surprise comeback”).

The just-published paper reporting the results confirms that a 42% increase in abundance indeed occurred in the western Barents Sea (Aars et al. 2017, Table 3): 685 bears were counted in 2004 compared to 973 bears in 2015.

However, due to the large uncertainty (confidence intervals) in the estimates involved, even a 42% increase was not statistically significant (I had assumed, when the initial results were released, that it would be, and perhaps Jon Aars did too, but it turned out not to be so).

This point was strongly emphasized by the PBSG in their recent status update for the Barents Sea: “Because of the overlapping confidence intervals, it cannot be concluded that the BS subpopulation has grown.”

The authors of the Svalbard survey (Aars et al. 2017), however, had this to say in their abstract:

“There is no evidence that the fast reduction of sea-ice habitat in the area has yet led to a reduction in population size. The carrying capacity is likely reduced significantly, but recovery from earlier depletion up to 1973 may still be ongoing.”

So, it wasn’t a decline but it might have been an increase. Not a very satisfying result.

The obvious question is this: if it had been a 42% decline, would alarm bells of a population doomed to extinction have been rung regardless of the ‘statistically insignificant’ caveat? See below for a hint.

WESTERN HUDSON BAY

As recently as 2 November 2017, biologist Nick Lunn was interviewed by the CBC and for the news program The National.  He stated outright, without qualification, that Western Hudson Bay polar bear numbers have dropped from about 1,200 (in 1987) to about 800 now, claiming a 33% decline.

However, it is not scientifically appropriate to compare these figures because they were based on different types of surveys conducted over different portions of the region (they are also statistically insignificant).

Lunn should know better because the published reports make it clear these numbers are not comparable.

The official Western Hudson Bay (WH) estimate accepted by the PBSG in 2014, and by the IUCN in 2015, is 1,030 bears (range 754-1,406). Environment Canada considered the subpopulation ‘likely stable’ in 2014, an assessment upheld by the 2016 survey.

Because even the 2011 and 2016 Western Hudson Bay aerial surveys used somewhat different methods, the only population size numbers (subsets of each total) that can be compared are these:

2011 (949, range 618-1280)

2016 (842, range 562-1121)

The slight apparent decline over 5 years (11%) was not statistically significant (Dyck et al. 2017, pg. 3, 37). Numbers may have dropped a bit but we can’t say for sure.

Similarly, when differences in methodology and statistical significance are taken into account, there is no evidence to suggest the estimate for 2016 is different from the 2011 estimate of 1,030 bears (Stapleton et al. 2014), which was not statistically different from the estimate of 935 (range 794-1076) calculated in 2004 (Regehr et al. 2007).

The PBSG still cites the estimate of 1,030 for WH and has not yet assessed the 2016 survey results generated by Dyck et al. (2017).

Yet, polar bear specialists continue to tell the media that the WH Bay subpopulation has declined to a worrying level.

Note that a similar but opposite problem arose for a recent “genetic capture-recapture” study in the Baffin Bay, which generated a 36% increase over the previous survey (SWG 2017).

Because the methods used between the two studies were deemed to be different, the apparent increase has been called into question (SWG 2017) and the PBSG emphasize that no trend can be concluded based on this data, although it is clear the expected decline due to overhunting did not occur and the population is likely stable.

DISCUSSION

My recent paper used a global population estimate that did not have confidence intervals calculated (Crockford 2017; Crockford and Geist 2018, see also Crockford 2018).

I determined that a modest increase in the global population size of about 16% had taken place between 2005 and 2015, when new surveys for the western Barents Sea, Western Hudson Bay, and the Baffin Bay are taken into account.

However, I noted this increase might not be statistically significant and that the reason for calculating it was to point out the major decline that had been predicted to occur back in 2007 had not happened.

In other words, I acknowledged I was not addressing confidence intervals and that the increase might not be significant: I didn’t simply pretend the issues didn’t exist.

I can appreciate that large error ranges and different methodologies are frustrating for biologists involved. To be presented with population estimate data that appear to show a pattern (with numbers either increasing or decreasing) only to find that the mathematical constraints imposed by the generation of those numbers prevent you from saying so is very aggravating.

But if a 42% increase (or decrease) in a population’s abundance is not statistically significant (or is based on incomparable figures due to survey design), what’s the point in generating the estimates in the first place?

If they are used properly, population estimates that come with such caveats are not useful for either management purposes or tracking of trends over time because you can’t really say if numbers have actually gone up or down to a meaningful degree.

Polar bear biologist Andrew Derocher apparently agrees:

Aerial surveys ==> large confidence intervals & we’d need > 50% decline for statistical significance. Not precautionary management to wait https://t.co/cMw61Rwg71

— Andrew Derocher (@AEDerocher) September 14, 2017

However, if they are used improperly (i.e., by failing to mention the statistical significance issue and/or the differences in methodology), these numbers cease to be scientific.

Note in the tweet below, Derocher insists that an 18% decline is “not slight” even though it is much less than the statistically insignificant 42% Barents Sea increase:

Western Hudson Bay #polarbear population dropped from 1030 bears to 842 over 2011-2016. 18% decline. Not “slight”. https://t.co/AoVSqWSQjw pic.twitter.com/tmlgYlZ3a1

— Andrew Derocher (@AEDerocher) September 14, 2017

There is clearly a problem here that won’t be easily resolved.

It may turn out to be the case that in order for there to be a statistically significant decline, polar bear numbers would have to drop by 67% (as they were predicted to do back in 2007).

In other words, two-thirds of the world’s bears might have to disappear before a statistically significant decline would be registered — and that clearly is not the point of surveying populations so that they can be managed responsibly.

I don’t have a solution to offer but I do know this: biologists and journalists all need to be as transparent and honest as possible when dealing with these numbers.

It’s not ethical to deliberately misrepresent the data to the public in order to make your case, as Nick Lunn and Andrew Derocher have been doing recently with Western Hudson Bay numbers.

In contrast, I have raised the issue of an apparent increase in global polar bear numbers to emphasize that the predicted 67% decline in numbers has not occurred.

Journalists have a responsibility, when presented with population numbers during interviews, to ask “is that statistically significant?” and “were the estimates from previous years generated the same way?”

If they have been doing this, they haven’t been reporting it, as far as I’ve seen.

That said, the Barents Sea study was a big eye-opener for me and it suggests some variation of the Aars et al. (2017) caveat may always be necessary:

“There is no evidence that the fast reduction of sea-ice habitat in the area has yet led to a reduction in population size. The carrying capacity is likely reduced significantly, but recovery from earlier depletion up to 1973 may still be ongoing.”

Read rest at Polar Bear Science