Glenn Cumiskey is Digital Preservation Resource Manager at the British Museum
The British Museum faces many challenges in its plans to better manage its data. For example, the capacity of the British Museum storage area network (SAN) has for some years been unable to meet the growing demands placed on it by the voluminous growth of data created by Museum staff in pursuit of their day-to-day activities.
Data growth is driven by many factors. As technologies reduce in cost but increase in the size and complexity of the assets they generate, the number of assets created and the storage footprint of these will naturally and potentially exponentially increase.
There are however some predictions around data growth that we can make, based on research into global data growth:
- By 2025 data volumes will have grown tenfold from 2016-2017 levels (see Fig 1.)
- Scientific imaging (e.g. MRI scanning) is a significant driver of data growth in certain sectors
- Unstructured audio-visual data is a significant driver of data growth
Fig 1. Data growth globally to 2025. Source: https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf
There is no reason to suspect that the data growth profile of the Museum is an exception to the above statements.
In response to this growth, staff have adopted storage solutions that are cheap, available and meet their short-term requirements of larger storage capacity that is affordable, coupled with fast access to data stored there. In practise this often means the use of external USB hard drives. Within the Museum the volume of data stored on these devices outstrip similar data stored on our SAN by an order of magnitude.
These short-term solutions tend to grow organically into longer-term solutions that are inevitably viewed by staff as personal, project, research and departmental archives. This has resulted in the growth of behavioural patterns and shadow IT environments that are highly risky, and vulnerable to catastrophic data loss.
Data loss of this nature risks the loss of the significant investment used to create it while also endangering operational efficiencies, project delivery, reuse potential, and reputational standing.
In response to this, and as part of the Museum’s growing digital preservation solution, we are proposing a tiered approach to data management that takes as a first step resilient, affordable, scalable storage but that grows to a fully blown digital preservation service for assets deemed to be of sufficient value.
In the coming months I would like to outline our approach and demonstrate that even national institutions are constrained by some quite challenging economic and cultural factors.
The value of this series of articles I hope will be to demonstrate an approach that can be adopted by many cultural institutions that feel overwhelmed by the situation they find themselves in. While not ideal, it is a ‘good enough’ approach that I believe fits the profile of the data we create and collect, and that chimes nicely with the adage ‘don’t let the best be the enemy of the good’.
Until next time.
Glenn Cumiskey
Read more...