Walsh et al. 2016: the uncertainties

(en español aquí)

Last year I published a post to present the new Walsh arctic sea ice dataset (Walsh et al. 2016, A database for depicting Arctic sea ice variations back to 1850. Geogr Rev, 107: 89–107. doi:10.1111/j.1931-0846.2016.12195.x) The data and the documentation describing this dataset are available at NSIDC (G10010).

One year ago, I posted the following graph, where we can see Arctic sea ice extent in March, when the ice is near its annual maximum, and in September, when ice extent reaches its yearly minimum, since 1850 to 2013, according to the new Walsh dataset:

The main conclusions of the paper are that recent sea ice extent is the smallest on record, and that the current downward trend has no precedent in duration or scales since 1850.

Without calling into question these conclusions, I pointed out that the reliability of the record is much lower before 1935: «Prior to 1935, there are not September direct observations at all, and the extent numbers before that year rely almost completely on interpolation. (Likewise the extent numbers during 1935-1952 have larger uncertainties than the values from 1953 onwards).»

The new Walsh dataset has resolved many of the problems of the former Walsh dataset (Walsh, 1978; Walsh & Johnson, 1978; Walsh & Chapman, 2001). The incorporation of AARI data for the Russian sector has been one of the keys for this improvement. The current Walsh dataset presents a much higher correlation with Arctic surface air temperature than the former one.

However, the new dataset still presents some weak points and uncertainties. In this post I will discuss some of the uncertainties of the new Walsh Arctic sea ice extent dataset:

  1. Uncertainties caused by the areas without actual observations and by the algorithm used to fill these areas. 

Walsh et al. use a variety of data sources for the pre satellite era, with different geographic and temporal coverages (the most relevant data sources are  AARI, DMI and ACSYS). For every month of the dataset, Walsh et al. put together all the available observations from the different data sources. The areas without observations are filled with an analog algorithm (*).

From 1953 onwards there are continuous and reliable observations for almost the whole Arctic. Hence, the analog algorithm is not neccesary and therefore it hasn’t any effect on the results.

However, before 1953 the data are sparser, and every year there are several areas without actual observations. For instance, the following maps show the available observations for August and September 1941 (red=ice, blue=water, grey= no data):

Walsh et al. 2016 arctic sea ice actual observations 1941 august september

Most of the Arctic  without any observation. Thus, it must be filled by the analog algorithm. Obviously, the areas without data and their infilling lead to a higher uncertainty on the results.

However, despite the dificulties and uncertainties, overall it seems that during 1933-1952 the available observations are enough to get reasonably reliable results.

Nevertheless, the actual observations are clearly sparser before 1933, due to the fact that one of the most important data sources (AARI) is not available anymore. And it gets even worse before 1900, when DMI data also disappear.

For instance, in the following maps we can see the available actual observations for August and September 1879 and 1850:

Walsh et al. 2016 arctic sea ice actual observations 1850 august september

Almost the whole Arctic without data. Therefore, the analog algorithm must be used in order to fill the vast areas without obsevations. Obviously, the results of this infilling have huge uncertainties. Likewise, the lack of actual observations and the reliance on the infilling algorithm could explain the scarce variability of September Arctic sea ice extent between 1850 and 1930 in the first graph of this post. Thus, the low variability of the earlier record could be an artifact caused by the sparseness of data and the use of the analog algorithm.


2. Uncertainties caused by the use of Kelly grids (white coloured areas at DMI maps) 

One of the primary data sources before 1957 for the new Walsh dataset are the Kelly grids (Kelly, 1979:  An Arctic sea ice dataset 1901-1956, Glaciological Data Series; 5, p.101  pdf ) based on DMI charts .

For a given month, the DMI charts provide actual observations of sea ice (coloured in red) and a white coloured area described as «ice supposed but no information at hand».

Kelly digitized the white coloured areas, as if they were actual observations. And Walsh et al. 2016 still use the white coloured areas as a valid data source.

Obviously, the use of estimates or guesses that are not based on actual observations leads to higher uncertainties on the results. In addition, when there are alternative sources, it is clear that the white coloured areas are often plainly wrong. For example, the following figure shows a comparison between the DMI chart for August 1952 and the available AARI charts for the same month (the latter ones are based on actual observations):

Walsh et al. 2016 DMI vs AARI August 1952(click for a larger image)

We can see the open water (blue) along the Siberian seas, whereas according to the DMI chart the whole Siberian Arctic is white. Therefore, the white coloured area should be taken as an area without data, instead of an ice covered area.

In fact, Walsh et al. 2016 are already using AARI charts for August 1952, so the white coloured area has already been corrected. However, what happens in the regions where there is not an alternative data source? And, what happens before 1933, when AARI charts are not available anymore? The answer is that Walsh et al. take the white areas as ice covered.

The actual observations displayed at DMI charts (coloured in red) are really useful. However, the white coloured areas are clearly areas without data, so I don’t think they should be taken as ice covered. This only adds uncertainty to the dataset, and diminishes its consistency.

In addition, it seems that Walsh et al. have misplaced by a month the Kelly grids. That is, in August they are using the white coloured areas from July charts, in July they are using the white areas from June charts, etc.

At the figure below, we have to focus on Hudson Strait, between Baffin Island and the Labrador Peninsula (I have circled the Strait). Walsh’s chart (bottom) for August 1935 shows the Strait ice covered. This resemblances the July DMI chart (top left) instead of the August one (top right):


Walsh et al. 2016 arctic sea ice misplaced Kelly grids DMI August 1935(click for a larger image)

If we look at the whole Baffin Bay Region, we can see the resemblance with July instead of August too.

I have checked the new Walsh dataset and it seems that this misplacement happens every year when this data source is used (1901-1956).

This error contributes to more uncertainties (although, as I have stated above, even if the white coloured areas were used at the right month, they are not a reliable nor a consistent data source).

In conclusion, Walsh et al. have done a great work putting together many different data sources, and they have resolved many of the problems of the former Walsh dataset. From 1933 onwards, their September results are reasonably reliable. Nevertheless, between 1900 and 1932 their results have much higher uncertainties, due to the lack of actual observations and to the use of unreliable data sources (Kelly grids).  And, before 1900, their results should be taken with great caution. Simply, before that year there are not enough observations to get Arctic wide reliable results.


(*) https://nsidc.org/data/docs/noaa/g10010-sea-ice-1850-onward/G10010_SIBT1850.pdf

“Stated succinctly, the process is as follows:

1.For each grid point p with no data in calendar month m and year y , and no data in the surrounding 2 months, areas with existing data in m-y are compared with calendar month m of all years 1900-2000 to select the best analogs. 

2.If the three best analog years do not have data at point p, the search is repeated by limiting the analog candidates to 1953-2000.

3. If Step 2 does not produce three analogs with data at point p , then fewer than three analogs are used.

4. Point p is “filled in” with the average concentration of the (up to) 3 analog fields.”

Esta entrada fue publicada en Arctic sea ice before satellites, Artículos científicos, Ártico, banquisa antes de los satelites, Reconstrucciones climáticas y etiquetada , , , , . Guarda el enlace permanente.

8 respuestas a Walsh et al. 2016: the uncertainties

  1. Pingback: Walsh et al. 2016 : las incertidumbres | Banquisa en el Ártico: el blog del hielo marino

  2. Walsh et al. 2016 acknowledge some of these uncertainties. They state:

    «The trends and variability in the regional time series (as well as in the pan-Arctic time series of Figure 8) clearly contain uncertainties to which several factors contribute. First, there are uncertainties in each of the original sources, and these uncertainties vary among the sources. The uncertainties include underestimates of summer ice concentrations derived from the satellite sources in recent decades, the use of categories of ice concentration in some sources (Danish Meteorological Institute, U.S. Naval Oceanographic Office), and the reconstruction of ice edges based on ship reports on from different dates within a particular month (e.g., the synthesis of whaling ship reports). With the exception of the passive microwave concentrations (Ivanova et al. 2014), the uncertainties in individual sources have been quantified by the providers of the data. Second, additional uncertainties are introduced by the data fusion process, particularly by (1) the superposition of concentration gradients upon the ice-edge information and (2) the in-filling of missing data by the analog and temporal interpolation procedures described earlier. The latter two types of uncertainty are greatest in the earlier part of the period, especially 1850-1900, when the input data were relatively sparse. An evaluation of these uncertainties can be performed by either (1) retaining only the ice-edge information or (2) withholding all available information for particular months with substantial information available. Such an evaluation will be undertaken in the coming months. Nevertheless, greater confidence must clearly be assigned to the variability and trends in the more recent decades than in the earlier portions of the time series discussed here.»


  3. Another source of uncertainty is the likely lack of consistency between the passive microwave satellite record and the earlier data (*). Walsh et al. are aware of this and, in order to address it, they are planning to present an alternative time series with different values for the satellite era (I guess that they could use NIC charts, that usually present higher values than passive microwave ones).

    In 2016 we published a paper that presented a new time series of September Arctic sea ice extent from 1935 to 2014.

    The graph below shows a comparison between our work (red line) and the new Walsh dataset (blue line) since 1935:

    Our numbers before 1979 run always somewhat lower than those of Walsh et al. This is probably due to the fact that Walsh et al. have not adjusted the pre-satellite values, whereas we adjusted them. And, what happens if we adjust Walsh’s data? If we apply the same adjustment used by Meier et al. 2012 and by our paper, and extending it back to 1935, this is the result:

    Walsh’s data become even closer to ours.


    (*) Walsh and Chapman 2001: “The solid curve NIC data for 1978^94) shows larger ice-covered areas than does the time series utilizing passive-microwave SMMR/SSMI) data for the same period. The differences range from approximately 5% in winter (Fig.1a) to 15-20% in summer (Fig.1b).The differences, which arise from the smaller ice concentrations in the passive-microwave dataset, are largest in regions and time periods when melt creates a wet surface.”

    Rayner et al. 2003: “ For example, satellite-borne passive microwave retrievals of sea ice concentration are not consistent with historical charts based on in situ observations, aerial reconnaissance and infrared satellite images. (…) A bigger problem, however, is that thin ice is not identified as such by the microwave retrievals: instead it is returned as a mixture of thick ice and open water [Emery et al., 1994]. Also, ponds resulting from summer melting on top of the ice often cause the microwave instrument to return a 10–30% lower than actual concentration of sea ice [Comiso and Kwok, 1996]: this particularly affects the Arctic in summer”

    Meier et al. 2012: “ However, they are not consistent with the passive microwave satellite record and any quantitative estimates of trends or variability across the 1978–1979 boundary are limited by uncertainties resulting from the inconsistent data sources.”

    Agnew and Howell, 2003: “Compared with the Canadian regional ice charts, the NASA Team algorithm underestimates the total ice‐covered area by 20.4% to 33.5% during ice melt in the summer and by 7.6% to 43.5% during ice growth in the late fall. (…) Compared with the U.S. National Ice Center hemispheric chart series, the average underestimation is 18.6% in summer.”


  4. Gridded Monthly Sea Ice Extent and Concentration, 1850 Onward, Version 2

    Minor changes. Uncertainties (and errors) remain.

    Error: during 1901-1956 Kelly grids are still misplaced by a month . That is, in August they are using July charts, in July they are using June charts, etc.

    Walsh et al. V2, August 1936. We can see the ice at Baffin Bay:

    DMI chart (the source where Kelly grids are derived from), August 1936. Baffin Bay is almost ice free:

    DMI chart, July 1936: sea ice at Baffin Bay:

    Walsh’s August map looks like DMI for July instead of August.

    If we look at Baffin Bay, we can clearly see the resemblance with July instead of August too.

    These charts are just an example, because this misplacement seems to happen from May to August, every year from 1901 to 1956.

    I hope they will correct it for a V3 version.

    (Although I think that Kelly grids are not a consistent nor a reliable data source.)

  5. https://nsidc.org/data/G10010
    «Note: A user let us know that the Kelly source fields appear to include other data sources that are temporally displaced by one month. For example, the August 1936 Kelly source appears to have included chart information from the July DMI charts or yearbooks. We suggest you delay downloading the V2 data until we are able to investigate this problem more throughly. When the issue is resolved we will remove this warning.»


  6. In addition, these are some more examples of the misplacement, focusing on Baffin Bay:

    Looking at Walsh’s map for August 1938…

    …sea ice at Baffin Bay looks like DMI for July …

    …instead of August:

    Looking at Walsh’s map for August 1939…

    …sea ice at Baffin Bay resembles DMI for July…

    …instead of August…


    Moreover, looking at Walsh’s map for July 1939…

    …sea ice at Labrador coast looks like DMI for June…

    …instead of July:

    Those are just some examples, there are more of them (summers of 1937, 1936, 1935, 1934, 1933… 1901).

  7. The error warning (https://web.archive.org/web/20190914151836/https://nsidc.org/data/G10010) at https://nsidc.org/data/G10010 has been removed. However, they have not corrected anything yet…

  8. Finally, NSIDC published a news announcement covering this issue:


    “NOAA@NSIDC would like to make users aware of a data update and an error notice for the NOAA/NSIDC Gridded Monthly Sea Ice Extent and Concentration, 1850 Onward data set.
    Error notice:
    A user has made us aware that the ice cover derived from the Danish Meteorological Institute Yearbooks may have been incorporated incorrectly into the Kelly grid source as it was used in G10010. The Kelly grids may have used the ice map from the subsequent month, rather than the intended month. The Kelly grids were used over April through August or September of 1901-1956. We are investigating the problem, and plan to correct the misalignment. We will inform registered users of the outcome. This apparent misapplication of the Kelly grid source is present in both V1 and V2.”

    A similar error warning was sent to registered users of this dataset too, and published in the User Guide: https://nsidc.org/data/g10010/

    Let’s hope they are correcting this error in a next update.

Deja una respuesta

Introduce tus datos o haz clic en un icono para iniciar sesión:

Logo de WordPress.com

Estás comentando usando tu cuenta de WordPress.com. Salir /  Cambiar )

Imagen de Twitter

Estás comentando usando tu cuenta de Twitter. Salir /  Cambiar )

Foto de Facebook

Estás comentando usando tu cuenta de Facebook. Salir /  Cambiar )

Conectando a %s