Fire Fire Everywhere and Lots of Data to Save

This year’s Climate and Society class is out in the field (or lab or office) completing a summer internship or thesis. They’ll be documenting their experiences one blog post at a time. Read on to see what they’re up to.

By Ipsita Kumar, Climate and Society 2014

Imagine your computer crashing and losing all your data. Sounds painful, right? Now imagine that happening to weather data for an entire country. That’s what happened in Jamaica more than 20 years ago when a fire sent decades of data up in smoke, a loss that researchers are still trying to deal with.

The year was 1992 and the Meteorological Services of Jamaica (Met henceforth) had just completed a 30-year analysis of their weather and climate data covering 1961-1990. Scientists commonly use 30-year periods as “climate normals” to understand seasonal and longer-term climate shifts.

On May 23 at 2:00 a.m., the Kingston fire department got a call that the Met had caught on fire. Though firefighters were dispatched, everything was lost.

Effects of the current drought in Jamaica (Source: Ipsita Kaur).

Scientists immediately started trying to find data in other locations. They contacted agencies they sent data to or had working relationships with. Many sent back what they had but some didn’t. That gave scientists a piecemeal collection of data of specific indicators or time periods. Scientists dug deeper and recovered data from burnt papers. They currently have unexplored microfiche tapes, with data dating back to the 1900s. All this means there are still large gaps.

Moving on another 5 years later, the Met faced another problem. The software used to store their current data crashed. They haven’t recovered the lost data yet, though the loss was dampened this time because they still had data on paper.

What happens to research when you lose all your data? A lot and none of it good.

The Met is using monthly data prior to 1992 to calculate and create drought and other forecasts, and longer-term predictions, but they’re plagued by problems with reliability. They are also asked by policymakers and practitioners to provide guidance on the implications of changing weather and climatic patterns on development. This has been a major challenge given that they lost years of daily data. But the main issue currently is calculating the 30-year mean.

Effects of the current drought in Jamaica (Source: Ipsita Kaur)

There’s some data available prior to 1992, but it’s not as good as after 1992. They’re using a number of techniques to try and fill in gaps including advanced computer modelling and satellites, which have records from late 1970s. They’re also conduct stringent quality checks and going through numerous steps to ensure data reliability.

One lesson learned from the whole ordeal is to not put all your eggs in one basket and the Met has put mechanisms in place to avoid it. They store data in three locations, maintain a repository of who they provided data to and what data they provide, photographs of data and its storage in hard drives. They are still looking to explore data archives safeguarded from fire and other threats and of course, the world of cloud. Two major challenges they face are financing and manpower. Nevertheless, one can learn a lot from what they have done to avoid loss of data and to move forward with their analysis with incomplete data.