µ u v - copernicus...discoveries in various archives and museums in norway and finland have re...

ECMWF COPERNICUS REPORT

Copernicus Climate Change Service

State of data rescue assessments

Issued by: University of Bern / Stefan Brönnimmann

Date: 03/01/2018

Ref: C3S_DC3S311a_Lot1.3.1_2017_State of data rescue assessments_v2.docx

Official ref. number service contract: 2017/C3S_3S311a_Lot1_UBERN/SC1

This document has been produced in the context of the Copernicus Climate Change Service (C3S).

The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts,

operator of C3S on behalf of the European Union (Delegation Agreement signed on 11/11/2014). All information in this

document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.

The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission

and the European Centre for Medium-Range Weather Forecasts has no liability in respect of this document, which is merely

representing the authors view.


C3S_DC3S311a_Lot1.3.1_2017_State of data rescue assessments_v2.docx Page 3 of 25

Contributors

MET OFFICE 1. Paul van der Linden

2. Rob Allan

UNIVERSITY OF BERN 1. Stefan Brönnimmann

2. Yuri Brugnara

UNIVERSITY ROVIRA I VIRGILI 1. Manola Brunet

UNIVERSITY OF COLORADO 1. Gilbert Compo

IEDRO 1. Richard Crouthamel

UNIVERSITY OF EAST ANGLIA 1. Phil Jones

2. Clive Wilkinson

MÉTÉO-FRANCE 1. Sylvie Jourdain

UNIVERSITY OF GIESSEN 1. Jürg Luterbacher

KNMI 1. Peter Siegmund

UNIVERSITY OF LISBON 1. Maria Antonia Valente



Table of Contents

1. Introduction 5

2. State of data rescue assessments 6



1. Introduction

This document describes the current state of data rescue assessments, as prepared for the journal

Geoscience Data.



2. State of data rescue assessments

A roadmap to climate data rescue services (C3S Data Rescue Service: Deliverable

1.3.1-2017)

As prepared for Geoscience Data Journal

Abstract. The scientific environment surrounding climate change adaptation, mitigation and risk

management is evolving rapidly. Quantitative approaches such as risk mapping or impact modelling rely on

past meteorological data with daily or sub-daily resolution, a large fraction of which have never been

digitised. Here we argue that climate data rescue must be seen as a continuous, coordinated long-term

effort. Technical developments (e.g., data assimilation), new scientific questions (e.g., process understanding

of extreme events) and new social (e.g., risk assessment, health) or economic services (e.g., new renewable

energy sources, agriculture, tourism, etc.) render data valuable that hitherto were not considered so. This

continuous effort is currently undertaken by projects of various sizes, structure, funding and staffing, as well

as by dedicated programs within many national weather services. These activities are often not sufficiently

coordinated at international level and will benefit considerably from the data rescue services being

established within the Copernicus Climate Change Service (C3S) (https://climate.copernicus.eu/). In this

paper, we discuss the state of climate data rescue, the requirements for data rescue services, plus the

challenges, and opportunities presented to them.

1 Introduction

The past few years have seen considerable progress in our scientific understanding of climate change and

related consequences (IPCC, 2013). Adaptation and climate risk management are rapidly developing fields,

that increasingly use numerical models and rely on quantitative high resolution, high-quality and continuous

data sources. With this, comes the need for more and longer data series than are currently available. In fact,

the amount of information that could be made available, but only exists in hard copy form, is substantial. As

a consequence of the growing need for data to support climate change research, climate data rescue has

seen a marked revival over the past years. This resurgence has been a result of climate data rescue being

pursued as a community effort, whose success depends not only on the individual groups, projects and

initiatives involved, but on maintaining and growing a coherent and strong community to lead it forward.



In the following we would like to posit three premises that are discussed in the remainder of the paper.

(i) Climate data rescue should not be approached as a ’one-off’ task but seen as a continuous activity. The

increased demand for historical weather data is at least partly due to new techniques, which have permitted

the reconstruction of past global- to regional-scale weather patterns through the assimilation of such

observations in ways that were not previously possible. Data assimilation approaches allow making use of

only sparse input data to obtain useful new data products (Compo et al., 2006, 2011). Several historical

dynamical weather reconstructions or reanalyses have been produced in the last few years (Compo et al.,

2011; Poli et al., 2016, Laloyaux et al., 2017), demonstrating that large-scale weather reconstruction is

generally possible and can be improved by more observations from additional data rescue efforts. These long

historical reanalyses primarily rely on sub-daily surface pressure observations, which traditionally have

received less weight or interest by historical climatologists. This shows that new techniques (data

assimilation), new scientific questions (e.g., process understanding of extreme events) and new social (e.g.,

risk assessment) or economic issues (e.g., new renewable energy sources) render data valuable that hitherto

were not considered so (e.g., barograph strips). This requires going back to original archives and data

repositories, for the recovery, imaging and digitising of their relevant historical records as well as quality

control. Thus, considering meteorological and climate data rescue as a continuous long term effort that

changes with new techniques and new demands require an environment that can sustain continuity.

(ii) Although it is a global, continuous effort, it has been argued that data rescue is best performed by many

small, individual projects (see Brunet and Jones, 2011). Each of these projects would establish its own links to

archives (e.g., Veale et al., 2017), libraries, but also science and end users. At the same time, through close

collaborations among these groups, the networks are shared where possible. Also, many of the projects have

a local connection, which is important as information is often local-to-regional scale. The experience of the

last 10 or so years has also shown that a big, sustainable, multinational data rescue project funded by major

international bodies, such as the World Meteorological Organization (WMO), Global Framework for Climate

Services (GFCS), Global Climate Observing System (GCOS), Group on Earth Observations (GEO), Joint

Technical Commission for Oceanography and Marine Meteorology (JCOMM), and the like, is not likely to

happen. Nevertheless, what has been demonstrated by the Atmospheric Circulation Reconstructions over

the Earth (ACRE) initiative (Allan et al., 2011a; Allan et al., 2011b; Allan et al., 2016) is that an international

umbrella linking together various local, regional and global, projects, programs, institutions and organizations

into a coherent community, effectively ‘pooling’ its resources, personnel and funds, with various regional

data rescue foci under it (e.g. Williamson et al., 2015; Williamson et al., 2017a; Williamson et al., 2017b), is a



viable role model for ongoing data rescue activities. Climate data rescue therefore requires maintaining an

active, vibrant community.

(iii) To sustain such a community effort requires various undertakings at different levels. While at the level of

the individual research groups, serving aids such as best practice guides, metadatabases, and software tools

can save efforts and lead to better coherency, there is also the level of community at large, of national

weather services and funding bodies and of international organizations. Topics including scientific merit,

ownership and data policies concern all levels from the WMO down to the individual researcher.

In this paper, representing Deliverable 1.3.1-2017 of the Copernicus Climate Change Service 311a Lot1 for

Collection and Processing of In Situ Observations - Data Rescue (C3S Data Rescue Service), we provide a

roadmap for meteorological and climate data rescue services that addresses these three premises. A

roadmap for the storage and management of the meteorological data over land has been outlined elsewhere

(Thorne et al., 2017). For marine data, details of the latest release of the ongoing international marine

meteorological data repository, the International Comprehensive Ocean-Atmosphere DataSet (ICOADS)

version 3, are provided by Freeman et al. (2017).

We first discuss how climate data rescue currently works (an annex to this paper, which will be updated

annually, provides an inventory of global activities). Then we discuss the three premises specified above,

from which follow requirements for climate data rescue services, embedded within a coherent, ‘end-to-end’

IT infrastructure encompassing an international data portal, data registry, tools and techniques, that are now

being established by the C3S Data Rescue Service (DRS).

2 Climate Data Rescue

Climate data rescue mostly concerns data that pre-date the age of electronic data acquisition. In developing

countries, weather observations are still performed manually and require digitisation. Although important

terrestrial surface and upper air data series have been digitised and processed (e.g., Alcoforado et al., 2012;

Camuffo and Bertolin, 2012; Camuffo et al., 2013; Domínguez-Castro et al., 2014; Brunet et al., 2014;

Slonosky, 2014; Ashcroft et al., 2014; Stickler et al., 2014; to give only very few examples) a huge fraction has

still not yet been digitised. This also applies to the historical marine observations from exploration, naval,

postal, merchant and passenger ships.

Overall, the fraction of yet to be digitised data is difficult to quantify. Data sources include information

contained in the charts of thermographs, barographs, and the like (Fig. 1), while the great bulk of digitised



climate data, be it from marine or terrestrial sources, has been keyed from hard copy readings in one

tabulated form or another. Sources of historical weather observations from ship log books have almost

exclusively focused on the holdings from the major European colonial powers (United Kingdom, The

Netherlands, Spain, Germany - and are far from exhausted) and the United States. However, recent

discoveries in various archives and museums in Norway and Finland have recovered some 53K images of log

books, with at least another 10K log books from the latter country yet to be imaged. Archives in countries

such as Sweden and Denmark have yet to be ‘touched’, and those in France and Portugal are in need of

greater attention. Also a wide range of repositories, such as the Ottoman and Venetian archives, are as yet

little known to climate scientists (e.g., Luterbacher et al. 2012). It is very important to note that many of

these logs cover exploration and commercial/mercantile voyages into the ‘data sparse’ oceans of the South

Hemisphere middle to higher latitudes (see Figure 1 in Freeman et al., 2017). The above examples only begin

to illustrate the huge amount of non-digitised climate information that can be found in archives around the

world.

Most developed countries already have the infrastructure (organization of the archives, formation of the

personnel, etc.) to efficiently carry out a national data rescue program, although they rarely allocate enough

resources for a full-time implementation. In developing countries, the WMO and other organizations have

been funding numerous programs in recent years to reorganize archives, buy digitisation hardware and

software, and train the personnel, avoiding the loss of many documents to improper storage. WMO also

publishes guidelines for best practices in the digitisation of climate data (WMO, 2016) and supervises an

International Data Rescue (I-DARE) Portal (https://idare-portal.org) where actual and potential data rescue

activities can be reported (Siegmund, 2014).

https://idare-portal.org/



Fig. 1. Barograph strip chart from a French station. Strip charts are a precious but often neglected source of

high-resolution observations. Image credit: Météo-France.

2.1 Current status of data rescue

Who is doing the climate data rescue work presently? One of the tasks of the C3S Data Rescue Service is to

answer this question, in order to improve coordination between different projects and avoid repetition of

work. We launched an online survey in late 2017, targeted at the coordinators of data rescue projects, an

exercise that will be repeated every year to compile an inventory of activities as completely as possible. The

respondents were representative of all six WMO regions, although the majority (60%) were from Europe.

International meetings (including ACRE workshops), the scientific literature and the web were additional

sources of information for the inventory of data rescue activities in the year 2017, which are given in the

Appendix and will be used to update the WMO I-DARE portal and the C3S DRS portal.

Data rescue is usually approached on three different time scales: i) efforts of short duration (weeks or

months), where the data to be rescued is determined by the specific needs of one-off activities (such as an

undergraduate thesis or a research article) and carried out by a single person or research group; ii) projects

with limited duration, usually 1-3 years, that are funded by regional, national or international institutions

(such as the European Commission) to achieve pre-determined goals that are usually tightly linked with

research activities involving different institutes; iii) long-term endeavours, typically carried out by public

national/regional weather services or through voluntary work (e.g., citizen science) and international

‘grassroots’ data community initiatives (e.g., ACRE).

Projects and one-off activities can set priorities that better suit the needs of climate research and leave clear

traces of what is done by publishing reports or journal articles; moreover, in most cases the rescued data are

immediately made public without restrictions. However, their short durations mean the loss of specific

know-how that will cost time to be acquired by a new project. Long-term endeavours allow a more efficient

and thorough data rescue, but they can be hampered by lack of funds or personnel. Moreover, in the case of

national or regional weather services, it is sometimes difficult to obtain detailed information on what has

been done and the data may be subjected to limitations in their distribution. In fact, there exists a certain

amount of data that have already been digitised years or even decades ago, but which remains inaccessible

to most of the scientific community. In some cases data owners may not even be aware that their data can



be useful to others, or they do not know any platform where they can share them. It is not uncommon for

rescued data or metadata to be stored on private computers until they are eventually forgotten or lost

(particularly for one-off activities). This not only applies to keyed data. For instance, during the WMO DARE1

program in the 1990s, nearly 2 million pages of historical surface hydro-meteorological data from 48 African

countries was imaged and placed on nearly 100,000 microfiche sheets. These sheets, originally held by the

Belgian National Meteorological Service, were turned over to the African Centre of Meteorological

Application for Development (ACMAD; located in Niamey, Niger) in the mid-2000s to be held in safe keeping

until funding was available to digitize these imaged data. Unfortunately, although placing imagery onto

microfiche and microfilm was state-of-the-art technology at the time, scientists soon realized that unless

those microfiche were kept in environmentally controlled storage areas, the images would soon fade. For

more than a decade, these microfiche sheets have been exposed to temperatures approaching 40 degrees

Celsius. Although a digitisation project was eventually funded by the US Agency for International

Development in 2013, this allowed the scanning of only a small fraction of the microfiche sheets. At this

point in time, neither ACMAD nor the WMO is any closer to rescuing and digitising these historic surface data

than when the DARE1 program started. This example also shows how data rescue can be negatively affected

by the postponement of funding, as paper and film degrade over time. An issue with this project is that many

of the African countries destroyed the original paper records, as they were told that the information was

now safe.

Nevertheless, several data rescue activities have delivered, and continue to deliver, data to the scientific

community. A new focus for the data rescue community has seen the linking together of historical

climatologists with impact researchers, (e.g., Allan et al., 2011a; Allan et al., 2011b; Allan et al., 2016;

Williamson et al., 2015; Williamson et al., 2017a; Williamson et al., 2017b), at the centre of which is the

previously mentioned international ACRE initiative (Fig. 2). ACRE undertakes and facilitates historical global

surface terrestrial and marine weather data recovery, imaging and digitisation, feeding these data into open-

access international repositories responsible for such material, seeing that they provide the best quality and

quantity of surface weather observations for assimilation into all reanalyses, and ensuring that reanalyses

outputs are freely available and feed seamlessly into the climate science, climate applications and services,

impacts, risks and extremes communities.



Fig. 2. The interplay between the international ACRE initiative (including the locations of its various annual

international workshops), linked data rescue projects, and C3S DRS regional data rescue activities.

2.2 What is rescued?

Most data rescue activities are focused on data from surface stations on land (Fig. 3, top). This is the only

category that can provide long (centennial) records of the same geographical location and has, therefore,

obvious value in climate change research. Marine data rescue is important to improve reanalyses and can

provide critical observations for remote regions or where land data are insufficient (tropics, southern

hemisphere, polar regions), particularly in early colonial times. Reanalyses also benefit from upper-air data

(Stickler et al., 2014), which is also instrumental in understanding climate change (e.g., Thorne et al., 2011;

Seidel et al., 2011). In the polar regions, observations involving ice surfaces and extents are also essential

(e.g., Turner and Comiso, 2017).

Temperature, precipitation and pressure are the three core variables in climate data rescue (Fig. 3, middle).

While temperature and precipitation are often directly analysed, pressure is usually assimilated into



reanalyses or used for other types of weather reconstruction (e.g., Luterbacher et al., 2002; Schwander et al.,

2017). Other variables, previously considered “minor”, are now also being rescued regularly. Wind and

radiation, in particular, have received increasingly high attention over the last decade in response to the

observed “global stilling” (Vautard et al., 2010) and “global brightening” (Wild, 2012) phenomena. These

variables are also particularly important to the emerging renewable energy sector (see e.g. C3S ECEM and

C4ENERGY, http://climate.copernicus.eu/ecem-european-climatic-energy-mixes,

http://climate.copernicus.eu/clim4energy-service-providing-climate-change-indicators-tailored-energy-

sector).

Awareness of the importance of the single observations is widespread in the scientific community, therefore

data rescue is no longer limited to daily or monthly means (Fig. 3, bottom). One third of the respondents to

the survey were dealing with hourly or finer resolutions (e.g., thermographs).

The survey also included an open question on what assistance is needed. Nearly all of the respondents asked

for resources for data keying. In fact, many data rescue projects can only afford the imaging of documents

and leave the transcription of the data to future initiatives. It is easy to see why citizen science and

volunteers play an increasingly important role for climate data rescue.

http://climate.copernicus.eu/ecem-european-climatic-energy-mixeshttp://climate.copernicus.eu/clim4energy-service-providing-climate-change-indicators-tailored-energy-sectorhttp://climate.copernicus.eu/clim4energy-service-providing-climate-change-indicators-tailored-energy-sector



Fig. 3. Survey results (based on 30 responses).

3 Premises on climate data rescue

(i) Climate Data Rescue as a Continuous Effort



Climate data have been rescued and compiled in many efforts over the years. Already early 19th century

scientists such as Alexander von Humboldt, Wilhelm Brandes, Heinrich Dove, Matthew Maury, James Pollard

Espy and others tried to compile global meteorological data sets (e.g., Dove, 1844). Their goals were to

establish global climatologies of temperature, weather conditions over the oceans and air pressure. Later,

scientists such as Felix Exner and Sir Gilbert Walker compiled global data sets in order to study typical

variations on a seasonal and multi-seasonal scales. In the 20th century, data were compiled to study climate

trends, fluctuations and changes.

While these efforts built typically on previous work, this is not always the case. For instance, none of these

past applications required sub-daily surface pressure data, which today is the most important input for long

reanalyses (Compo et al., 2006). Similarly, in the past, monthly or seasonal values could be used to respond

to most research questions, while nowadays extreme events have a greater priority, implying a need for daily

or even hourly values. Therefore, archive work is still necessary to provide the input. Perhaps future long

reanalysis efforts will make use of wind data over land, cloud motion, or rainfall data. In addition to

anticipating the future needs, we should establish an environment in which returning to the archive is made

easy. Likewise, some variables may contain important information on other properties of interest, such as

aerosol information that can possibly be derived from sunshine duration records (Sanchez-Romero et al.,

2014). The numerous spectroscopic data (solar and stellar spectra) that were obtained in the early 20th

century might eventually also be used for atmospheric science, once suitable methods have been developed,

but digitising the data now would be extremely costly. Thus, technical changes (new methods) render data

valuable that were not hitherto seen as so.

These technical changes are sometimes related to scientific questions, for which historical information is

desired (e.g., the role of aerosols or clouds, for the examples above). Of course, scientific questions (such as

improving process understanding of extreme events) also steer the demand for data products. The scientific

demand is also related closely to societal demands. The interest in new renewable energy sources might also

trigger interest in wind and cloud cover information and in sunshine data records, which may not have been

digitised in previous efforts. Going back to the archives or to the imaged data sheets will often be necessary,

as not all future needs can be anticipated. Thus, climate data rescue must be seen as a continuous long term

effort that changes with new techniques and new demands.

(ii) A Distributed Approach to Climate Data Rescue



Even if demand and techniques change, would it not make sense to raise funds (millions) and do it “once and

for all”? Unlike launching a satellite of installing a super-computer, data rescue is labour intensive. It requires

training many persons, some of whom would have to come from other fields and whose knowledge would

be lost afterwards. Large masses of data can be rescued in citizen science projects, especially if they are well

defined, such as oldweather.org (Fig. 4). This is suitable for some of the sources that are well structured.

However, many other sources require dedicated scientific work and are not suitable for such efforts.

More importantly, even if the digitising aspect (keying, OCR) is done efficiently, the preceding (imaging,

preparing for online data rescue) as well as the following steps (controlling, documenting, formatting, etc.)

are typically more time consuming and require expertise that is beyond most the scope of citizen scientists.

Data rescue comprises a much wider range of task skills than scanning millions of pages and typing numbers

from sheets.

Online digital archives offered by an increasing number of institutions, from the local science societies to the

large libraries facilitate the access to manuscript climate data, and the trend towards digital humanities will

further allow exploiting synergies (use of data bases etc.). These are places and institutions where “once and

for all” efforts are undertaken.

Our inventory shows that data rescue projects have very different sizes and targets; they are driven by

specific data needs. Therefore, weather and climate data rescue is a distributed approach, which however

makes use of citizen science approaches, imaged library holdings and data bases of digital humanities. Not

only is climate data rescue dealing with spatial data, which often requires knowledge on the location, access

to a local archive, but also the expertise and knowledge is distributed.



Fig. 4. Screenshot of oldweather.org (new Whaling Chapter). Citizen science projects such as oldweather.org

have provided millions of valuable ship data, but cannot “do the job”. Only together with research projects,

efforts of national weather services and projects supported by development agencies and foundations can

the body of climate data be rescued.

(iii) Efforts must target projects, the scientific community, and science administration

For a distributed approach to work, it must not only be beneficial to society, but also attractive for scientists.

For individual research project to engage in weather and climate data rescue, the threshold must be low. For

the community it is important that the individual projects are guided towards delivering data in a common

format to data repositories. The C3S Data Rescue Service plays an important role in both aspects, as is

outlined in the following Section.

Other factors that are important to make data rescue attractive for scientists, but are beyond the control of

C3S DRS, concern the community at large, national weather services, funding bodies and international

organizations. Here, the C3S Data Rescue Service linkages to ACRE and its associations with WMO, GCOS,

JCOMM and the like will need to come into play.



Science requires an open, traceable data policy. During the past ten to twenty years, a lot has been achieved

towards this aim. Data sets can obtain digital object identifiers and funding bodies are more willing to fund

data rescue projects than in years past. In the last few years, several data journals have appeared that cater

for the needs of the data rescue (and other) communities. Most of them require the data to be deposited at

an approved public repository, with a digital object identifier. All of them are Open Access. Efforts on

weather and climate data rescue are still often published in regular meteorological and climatological

journals, which however mostly do not have a specific data policy. The fact that data journals have impact

factors that are similar or even higher than that of climate journals can be taken as an indication that the

merit of data rescue is higher than a decade ago.

Data policies do, of course, not only concern the journal, but national policies as well as WMO

recommendations. At the Seventeenth World Meteorological Congress in Geneva 2015, the WMO adopted

its policy for the international exchange of climate data and products to support the implementation of the

GFCS as Resolution 60. It states that GFCS relevant data and products from specified sources (such as World

Data Centres) are made accessible among members on a free and unrestricted basis.

4 Climate Data Rescue Services

The functioning of the C3S Data Rescue Service from a user perspective is shown in Fig. 5. On every step from

the idea of a project to data submission, C3S DRS provides support, ranging from data bases, to software

user support, community building, best practice reports, and capacity building. The first steps are data bases

with existing projects (I-DARE) and metadata. Here, metadatabases, where crucial information can be

retrieved (existing metadata on the series, links to archive holdings, contact persons, in the best cases links

to already imaged data sheets). The annual overview of data rescue activities is another example; it might

help to trigger or better embed activities.

Once digitising is undertaken, several tools may facilitate the work surrounding the digitising process.

Researchers should find answers to specific technical questions, e.g. how to digitize thermograph, barograph

or pluviograph charts. Software tools that can help in every phase of digitisation, from file naming of the

images to the quality control of data and metadata, will be provided on a free license. Projects targeting

working with volunteers will find outreach material helpful. Moreover, an online facility for data keying will

be developed. Equally important are capacity building workshops, since data rescue projects often target

developing countries. Following best practice guidelines may save time and efforts and at the same time

might contribute to increasing consistency across projects.



In addition to documents, the C3S Data Rescue Service in conjunction with ACRE provides a network of

interested scientists. Projects can then easily be embedded within an international community, with

communication (blogs, newsletters), workshops, session at important meetings etc. Most important in this

context are links to archives, museums and libraries, interdisciplinary collaborations, and links to end users in

order to anticipate future needs. In this way, climate data rescue services can make it attractive for a project

to engage more widely, and the project in turn enlarges the expertise of the community, and builds

knowledge and ‘confidence’ in weather and climate disciplines and their wider value to society.

Apart from the political side of data policies, the scientific side and practical aspects of data management

need consideration. Climate data rescue services need to be closely linked with data holdings (e.g., Menne et

al., 2012; Rennie et al., 2014; Cram et al., 2015; Freeman et al., 2017) and global repositories within C3S

(Thorne et al., 2017) and offer these links as services to projects. In that respect, the C3S Data Rescue Service

should channel data rescue projects towards common data holdings.

Fig. 5. Schematic that summarizes all phases of a data rescue project, together with the assistance that will

be provided by the C3S Data Rescue Service (red background).



Conclusions

Historical weather and climate data are relevant for climate sciences as well as for society in the context of

climate change adaptation and mitigation, and risk assessment. Large amounts of meteorological data that

could be helpful have not yet been digitised. To promote data rescue, climate data rescue services are now

being developed under C3S. In this paper we outline a roadmap towards successful climate data rescue

services based on three premises, from which follow requirements for such a service. We posit that data

rescue must be seen as a continuous, long-term activity. Changes in the demand (e.g., for analysing past

extreme events with respect to future changes in extremes) and changes in the technical possibilities (e.g.,

use surface pressure to obtain a complete description of the atmospheric state at high temporal and spatial

scale) lead to a re-valuation of historical data and thus the potential value of rescued data. Such changes will

also occur in the future: Demand will change (e.g., focus of solar and wind data for planning renewable

energy generation, or stronger focus on climate impact data) and technical possibilities will change (e.g.,

assimilation of further variables). We also postulate that a distributed approach is most suitable to the

problem of rescuing climate data, and hence the task will be shared by many research projects and efforts

from weather services and other institutions. Society is thus best served with an active data rescue

community that provides expertise on procedures, infrastructure, tools, and metadatabases. Furthermore, to

maintain this community, ensuring scientific merit and an open data policy will make it attractive for

research projects to engage in the process. Data rescue services operate as facilitators for individual projects

and collectively push towards these aims.

Over the last years, data rescue efforts have led to new data sets, and the importance of data rescue work

has been more and more appreciated. These efforts must be sustained, and climate data rescue services

under the C3S Data Rescue Service will contribute.

Acknowledgements. The work was founded by the EC via Copernicus Climate Change. In addition to being

supported by funding from the EU Copernicus Climate Change Service (C3S), Prof. Rob Allan also

acknowledges the University of Southern Queensland, Toowoomba, Australia and the Centre for Maritime

Historical Studies, University of Exeter, UK where he is an Adjunct and Honorary Professor respectively.



References:

Alcoforado MJ, Vaquero JM, Trigo RM, Taborda JP. 2012. Early Portuguese meteorological measurements

(18th century). Climate of the Past 8: 353–371, doi: 10.5194/cp-8-353-2012.

Allan R, Brohan P, Compo GP, Stone R, Luterbacher J, Brönnimann S. 2011a. The international Atmospheric

Circulation Reconstructions over the Earth (ACRE) initiative. Bulletin of the American Meteorological Society

92: 1421–1425.

Allan R, Compo G, Carton J. 2011b. Recovery of global surface weather observations for historical reanalyses

and international users. Eos, Transactions, American Geophysical Union 92: 154,

doi:10.1029/2011EO180008.

Allan R, Endfield G, Damodaran V, Adamson G, Hannaford M, Carroll F, Macdonald N, Groom N, Jones J,

Williamson F, Hendy E. 2016. Toward integrated historical climate research: the example of Atmospheric

Circulation Reconstructions over the Earth. Wiley Interdisciplinary Reviews: Climate Change 7: 164–174.

Ashcroft L, Gergis J, Karoly DJ. 2014. A historical climate dataset for southeastern Australia, 1788–1859.

Geoscience Data Journal 1: 158–178, doi:10.1002/gdj3.19

Brunet M, Jones P. 2011. Data rescue initiatives: bringing historical climate data into the 21st century.

Climate Research 47: 29–40, doi:10.3354/cr00960.

Brunet M, Gilabert A, Jones P, Efthymiadis D. 2014. A historical surface climate dataset from station

observations in Mediterranean North Africa and Middle East areas. Geoscience Data Journal 1: 121–128,

doi:10.1002/gdj3.12.

Camuffo D, Bertolin C. 2012. The earliest temperature observations in the world: the Medici Network (1654–

1670). Climatic Change 111: 335–363.

Camuffo D, Bertolin C, Diodato N, Cocheo C, Barriendos M, Dominguez-Castro F, Garnier E, Alcoforado MJ,

Nunes MF. 2013. Western Mediterranean precipitation over the last 300 years from instrumental

observations. Climatic Change 117: 85–101.

Compo GP, Whitaker JS, Sardeshmukh PD. 2006. Feasibility of a 100-year reanalysis using only surface

pressure data. Bulletin of the American Meteorological Society 87: 175–190.

Compo GP, Whitaker JS, Sardeshmukh PD, Matsui N, Allan RJ, Yin X, Gleason BE, Vose RS, Rutledge G,

Bessemoulin P, Brönnimann S, Brunet M, Crouthamel RI, Grant AN, Groisman PY, Jones PD, Kruk M, Kruger

AC, Marshall GJ, Maugeri M, Mok HY, Nordli Ø, Ross TF, Trigo RM, Wang X, Woodruff SD, Worley SJ. 2011.

The Twentieth Century Reanalysis Project. Quarterly Journal of the Royal Meteorological Society 137: 1–28.



Cram TA, Compo GP, Yin X, Allan RJ, McColl C, Vose RS, Whitaker JS, Matsui N, Ashcroft L, Auchmann R,

Bessemoulin P, Brandsma T, Brohan P, Brunet M, Comeaux J, Crouthamel R, Gleason BE, Groisman PY,

Hersbach H, Jones PD, Jonsson T, Jourdain S, Kelly G, Knapp KR, Kruger A, Kubota H, Lentini G, Lorrey A, Lott

N, Lubker SJ, Luterbacher J, Marshall GJ, Maugeri M, Mock CJ, Mok HY, Nordli O, Rodwell MJ, Ross TF,

Schuster D, Srnec L, Valente MA, Vizi Z, Wang XL, Westcott N, Woollen JS, Worley SJ. 2015. The International

Surface Pressure Databank version 2. Geoscience Data Journal 2: 31–46, doi: 10.1002/gdj3.25.

Domínguez-Castro F, Vaquero JM, Rodrigo FS, Farrona AMM, Gallego MC, García-Herrera R, Barriendos M,

Sanchez-Lorenzo A. 2014. Early Spanish meteorological records (1780–1850). International Journal of

Climatology 34: 593–603.

Dove, HW. 1844. Über die nicht periodischen Änderungen der Temperaturverteilung auf der Oberfläche der

Erde im Zeitraume von 1782 bis 1842. Reimer: Berlin, Germany.

Freeman E, Woodruff SD, Worley SJ, Lubker SJ, Kent EC, Angel WE, Berry DI, Brohan P, Eastman R, Gates L,

Gloeden W, Ji Z, Lawrimore J, Rayner NA, Rosenhagen G, Smith SR. 2017. ICOADS Release 3.0: a major

update to the historical marine climate record. International Journal of Climatology 37: 2211–2232.

IPCC. 2013. Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth

Assessment Report of the Intergovernmental Panel on Climate Change, Stocker TF, Qin D, Plattner G-K,

Tignor M, Allen SK, Boschung J, Nauels A, Xia Y, Bex V, Midgley PM (eds.). Cambridge University Press:

Cambridge, UK and New York, NY, USA; 1535 pp.

Laloyaux L, De Boisseson E, Dahlgren P. 2017. CERA-20C: An Earth system approach to climate reanalysis.

ECMWF Newsletter 150: 25–30.

Luterbacher J, Xoplaki E, Dietrich D, Rickli R, Jacobeit J, Beck C, Gyalistras D, Schmutz C, Wanner H. 2002.

Reconstruction of sea level pressure fields over the Eastern North Atlantic and Europe back to 1500. Climate

Dynamics 18: 545–561.

Luterbacher J, García-Herrera R, Akcer-On S, Allan R, Alvarez-Castro MC, Benito G, Booth J, Büntgen U,

Cagatay N, Colombaroli D, Davis B, Esper J, Felis T, Fleitmann D, Frank D, Gallego D, Garcia-Bustamante E,

Glaser R, González-Rouco JF, Goosse H, Kiefer T, Macklin MG, Manning S, Montagna P, Newman L, Power

MJ, Rath V, Ribera P, Riemann D, Roberts N, Sicre M, Silenzi S, Tinner W, Valero-Garces B, van der Schrier

G, Tzedakis C, Vannière B, Vogt S, Wanner H, Werner JP, Willett G, Williams MH, Xoplaki E, Zerefos CS,

Zorita E. 2012. A review of 2000 years of paleoclimatic evidence in the Mediterranean. In: The Climate of

the Mediterranean region: from the past to the future, Lionello P (Ed.). Elsevier: Amsterdam, The

Netherlands; 87–185.



Menne MJ, Durre I, Vose RS, Gleason BE, Houston TG. 2012. An overview of the global historical climatology

network-daily database. Journal of Atmospheric and Oceanic Technology 29: 897–910.

Poli P, Hersbach H, Dee DP, Berrisford P, Simmons AJ, Vitart F, Laloyaux P, Tan DG, Peubey C, Thépaut JN,

Trémolet Y. 2016. ERA-20C: An atmospheric reanalysis of the twentieth century. Journal of Climate 29:

4083–4097.

Rennie JJ, Lawrimore JH, Gleason B, Thorne PW, Morice CM, Menne MJ, Williams CN, Gambi de Almeida W,

Christy J, Flannery M, Ishihara K, Kamiguchi K, Klein-Tank AMG, Mhanda A, Lister D, Razuvaev V, Renom M,

Rusticucci M, Tandy J, Worley S, Venema V, Angel W, Brunet M, Dattore B, Diamond H, Lazzara M, Le Blancq

F, Luterbacher J, Mächel H, Revadekar J, Vose R, Yin X. 2014. The international surface temperature

initiative global land surface databank: monthly temperature data release description and methods,

Geoscience Data Journal 1: 75–102, doi:10.1002/gdj3.8.

Sanchez-Romero A, Sanchez-Lorenzo A, Calbó J, González JA, Azorin-Molina C. 2014. The signal of aerosol-

induced changes in sunshine duration records: A review of the evidence, Journal of Geophysical Research

119: 4657–4673.

Schwander M, Brönnimann S, Delaygue G, Rohrer M, Auchmann R, Brugnara Y. 2017. Reconstruction of

Central European daily weather types back to 1763. International Journal of Climatology 37: 30–44.

Seidel DJ, Gillett NP, Lanzante JR, Shine KP, Thorne PW. 2011. Stratospheric temperature trends: Our

evolving understanding. Wiley Interdisciplinary Reviews: Climate Change 2: 592–616.

Siegmund P. 2014. ‘White paper on an International Data Rescue Portal (I-DARE)’. 16pp. WMO: Geneva,

Switzerland.

Slonosky V. 2014. Historical climate observations in Canada: 18th and 19th century daily temperature from

the St. Lawrence Valley, Quebec. Geoscience Data Journal 1: 103–120, doi:10.1002/gdj3.11.

Stickler A, Brönnimann S, Valente MA, Bethke J, Sterin A, Jourdain S, Roucaute E, Vasquez MV, Reyes DA,

Allan R, Dee D. 2014. ERA-CLIM: Historical Surface and Upper-Air Data for Future Reanalyses. Bulletin of the

American Meteorological Society 95: 1419–1430.

Thorne PW, Lanzante JR, Peterson TC, Seidel DJ, Shine KP. 2011. Tropospheric temperature trends: history of

an ongoing controversy. Wiley Interdisciplinary Reviews: Climate Change 2: 66–88.

Thorne PW, Allan RJ, Ashcroft L, Brohan P, Dunn RJH, Menne MJ, Pearce PR, Picas J, Willett KM, Benoy M,

Brönnimann S, Canziani PO, Coll J, Crouthamel R, Compo GP, Cuppett D, Curley M, Duffy C, Gillespie I,

Guijarro J, Jourdain S, Kent EC, Kubota H, Legg TP, Li Q, Matsumoto J, Murphy C, Rayner NA, Rennie JJ,

Rustemeier E, Slivinski LC, Slonosky V, Squintu A, Tinz B, Valente MA, Walsh S, Wang XL, Westcott N, Wood

K, Woodruff SD, Worley SJ. 2017. Towards an integrated set of surface meteorological observations for



climate science and applications. Bulletin of the American Meteorological Society (early online), doi:

10.1175/BAMS-D-16-0165.1.

Turner J, Comiso J. 2017. Solve Antarctica’s sea-ice puzzle. Nature 547: 275–277.

Vautard R, Cattiaux J, Yiou P, Thépaut JN, Ciais P. 2010. Northern Hemisphere atmospheric stilling partly

attributed to an increase in surface roughness. Nature Geoscience 3: 756–761.

Veale L, Endfield G, Davies S, Macdonald N, Naylor S, Royer M-J, Bowen J, Tyler-Jones R, Jones C. 2017.

Dealing with the deluge of historical weather data: the example of the TEMPEST database. Geo: Geography

and Environment 4: e00039, doi: 10.1002/geo2.39.

Wild M. 2012. Enlightening global dimming and brightening. Bulletin of the American Meteorological Society

93: 27–37.

Williamson F, Allan R, Switzer AD, Chan JCL, Wasson RJ, D'Arrigo R, Gartner R. 2015. New Directions in

Hydro-Climatic Histories: Observational Data Recovery, Proxy Records and the Atmospheric Circulation

Reconstructions over the Earth (ACRE) initiative in Southeast Asia. Geoscience Letters 2: 2,

doi:10.1186/s40562-015-0018-z.

Williamson F, Ren G, Allan R. 2017a. The Atmospheric Circulation Reconstructions over the Earth (ACRE)

initiative ACRE China workshop: Recovery, digitisation, and analysis of pre-mid-twentieth century climate

observational data in East Asia workshop on 23–24 August, Beijing, China. Earth and Space Science 4: 40–

43.

Williamson F, Allan R, Ren G, Lee TC, Lui WH, Kubota H, Matsumoto J, Luterbacher L, Wilkinson C, Wood K.

2017b. Collating Historic Weather Observations for the East Asian Region: Challenges, Solutions, and

Reanalyses. Advances in Atmospheric Sciences (Special CSSP China Issue), in press.

WMO. 2016. Guidelines on Best Practices for Climate Data Rescue. WMO-No. 1182, 30pp. WMO: Geneva,

Switzerland.


climate.copernicus.eu copernicus.eu ecmwf.int

ECMWF - Shinfield Park, Reading RG2 9AX, UK

Contact: [email protected]

1. Introduction2. State of data rescue assessments

µ u v - copernicus...discoveries in various archives and museums in norway and finland have re...

Documents