Ray Moore is an Archives and Records Officer with the City of Edinburgh Council and member of the DPC Advocacy and Community Engagement Sub-Committee.
Back in spring (2025) the City of Edinburgh Council made a successful application to the UK National Archives, through its Archives Revealed program to secure and catalogue the digital legacy of the Transport Initiatives Edinburgh Ltd, latterly known as TIE Ltd (TIE). Active between 2002 and 2011, TIE functioned as an arm’s length external organisation (ALEO) for the City of Edinburgh Council and was responsible for the development of the transport infrastructure within the Edinburgh and surrounding areas. The company gained some notoriety, both nationally and internationally, due to the handling of the Edinburgh Trams development, with claims of mismanagement, contractual disputes, litigation, delays, and financial overspend. Such was the extent of these concerns that the project became the subject of a non-statutory public inquiry in 2014 (this reported in 2023). The records associated with TIE therefore represent an important, if controversial, part of Edinburgh’s recent history, but also have considerable academic interest as important exemplar of ALEO’s and their role in how local government managed infrastructural development at the start of the century.
Working with project partners from the Digital Preservation Coalition (DPC), the Business Archives Council of Scotland (BACS), and the Centre of Business History in Scotland (CBHS) at the University of Glasgow we are working to document and catalogue the collection, but also to produce transferable methodologies for the appraisal and cataloguing of large digital datasets, and to create appropriate metadata (at scale) to support both preservation and access to the collection.
A problem shared, is a problem halved.
Our principal focus in the project are those records stored within a Microsoft SharePoint instance, which amounts to some 7 million discreet files, some 5TB of digital data, and covering some 80 different formats. The scale of the ‘problem’ quickly focuses the attention. Indeed, sometimes it feels unsurmountable when we add in other forms of networked storage, legacy systems, external hard drives, portable media, tape backups, etc. As if our ‘issues’ were not enough we’ve also had to grapple with poorly documented data transfers from external systems into MS SharePoint, successive version migrations, and at least one, almost catastrophic, hardware failure. We are more fortunate than many, however, in that at least one diligent data officer has spent some (significant) time in organising the data into a form that supports ongoing access. These are ‘real-world’ challenges that we seldom acknowledge, and rarely openly discuss.
Our hopes for the project are ambitious, and we recognise that this a learning experience not simply for us, but also for the local government sector generally. A sector that has, traditionally, struggled to deal with its digital legacy. It can be no coincidence that ‘records of local government’ have been listed as a ‘Critical Endangered’ species within the DPC’s Bit List, and remained a ‘non-mover’ since 2019. We hope that by publicising our experiences that others might learn from our outcome’s, challenges, and, importantly, from our mistakes.
Struggling at the First Hurdle.
What perhaps we did not adequately legislate for were that our problems would begin almost on our first day; that what we had conceived of as a relatively simple task of securing the data would translate into many weeks of complex negotiation, protracted timetables, adapting workflows, and changing our plans. Like many organisations our IT is outsourced, consequently any non-standard request requires extensive form-filling, meetings, and negotiation, and whilst we had legislated for this in our plan, we perhaps underestimated the associated timeframes and financial costs of this element of the project. Our experience is not unique, and it is easy to see how colleagues within the local government sector have struggled to make significant headway in digital preservation and the provision of access to those collections. Thanks to some fantastic advocacy work in the City of Edinburgh Council we are fortunate to have an information management team who recognise the importance of digital preservation, and a Council that is sympathetic to its goals and objectives. Indeed, we are one of the few local authorities in the UK to be members of the Digital Preservation Coalition and to represent our community, but there remain problems to overcome.
We’re continuing our work to catalogue and improve access to this digital collection in accordance with the its Archives Revealed program and we’ll keep you all posted on our progress. Even if it isn’t all roses.
We’d certainly be interested in hearing more from colleagues who have experience of preserving large company archives, of wrangling data from MS SharePoint, and those who might want to form a support group to share experiences in the local government sector. We’ll be working in conjunction with the DPC to develop a workshop which will provide some support and guidance on dealing with large business archives in 2026 so keep your eyes peeled for that one.
