• You will need to gather a lot of information from colleagues in order to build a complete picture of the challenge and move towards an appropriate solution. The lists of questions included over the next steps will provide a useful starting point.

• Equally important is the information you will share with colleagues about digital preservation. Ensuring that everyone understands why this work needs to take place will also help you to make progress. Having your high level overview or ‘elevator pitch’ (prepared in the step above) to hand will help! See also the DPC’s Executive Guide on Digital Preservation which may be helpful in crafting more generic messages about the importance of digital preservation.

In her talk at the EDRMS Preservation Briefing Day, Emma Yan from the University of Glasgow stresses the importance of gathering the right people together at an early stage of scoping this preservation work - ‘EDRMS preservation at University of Glasgow – where do we begin?’, a presentation at Unbroken records: A briefing day on Digital Preservation and EDRMS, 20th May 2021 (DPC members login to view the recording).

EDRMS preservation at University of Glasgow – where do we begin?   - Emma Yan (University of Glasgow)

Kyle Browness from Library and Archives Canada gave a talk at the DPC’s Connecting the Bits unconference event in June 2020 entitled ‘The challenges and lessons of processing records from an EDRMS‘. He talked about how the right team (and skillset) was assembled to work on the challenge (DPC members login to view the recording).

• Who set up the system?

• Who provides ongoing support?

• Who administers the system?

• Who can make changes to the system?

• Who can use the system?

• Who takes decisions about records in the system?

• Who has current responsibility or ownership for the records?

• Who will have responsibility for the records in the future?

• Who do you need to persuade to support or sponsor your preservation work?

Once you have identified who you need to involve, take some time to consider when you will involve them and to what extent. Do you just need to have a quick chat with them or are they key stakeholders that you will need to keep in touch with on a regular basis? Establish a plan for how you will keep them up to date with the work you are doing. You may find it helpful to establish a formal project with scheduled meetings or a forum to report to, ensuring you have a regular platform to ask questions, report on progress and keep people informed.

Before making progress with any digital preservation challenge it is essential to gain an overview of the problem you are trying to solve - what exactly is it that you are trying to preserve and why? An initial high level scoping exercise should be carried out to give you an overview of the record keeping system that you are dealing with, the types of record that it contains, the value of the content and the important characteristics that must be preserved. Some of this will be obvious to you, but it is worth documenting what you are doing and why at a high level. Further stages of this process will take you deep down into the details of the challenge and it may be helpful to be able to step back and remind yourself of the high level task that you set out to achieve. Having this information at your fingertips will also be handy when talking to colleagues and could even form the basis of a quick ‘elevator pitch’ that describes what you are doing and why.

The following high level questions can be used to help articulate what you are trying to achieve:

• What - what needs to be preserved? Note that within a record keeping system, there may be a mix of records and not all will need to be preserved for the long term.

• Where - where are the records stored currently? Where do they need to go?

• Why - why does this work need to be carried out? What is the value of the content? What are the implications if the records are lost?

• When - when does this work need to be carried out? Is there a date that the system will be decommissioned or become unsupported?

• Who - who owns the record keeping system? Is it an internal system or one that you are accepting from an external donor or depositor? Who needs to access the records (both now and in the future)? What will they use the records for?

• How - how will users find the records they need? How will they access those records?

Introduction

Preserving content from a record keeping system is a complex and challenging process involving many stakeholders. Below is a series of steps that you will need to work through in order to carry out the task. Note that these steps are presented as a guide only and do not necessarily define a linear structure. Do work through them in an order that makes sense to you. There is inevitably some overlap between some of the initial steps which are focused on fact finding. These steps are designed to help you understand the challenge before moving towards a solution and in reality, some of these will run in parallel as you locate the right people to answer your questions and begin your investigations.

It should be noted that this is a complex process typically involving multiple stakeholders and different technologies. It may take a period of several years to move through this process towards a satisfactory preservation outcome. Use the points below to help you start to map out the process and put in place a plan to work through the issues. It is likely that some of the steps will need to be revisited more than once as the landscape changes. Records Management and digital preservation are ongoing tasks which are best implemented as part of an ongoing cycle of work.

• Understanding the problem - get a high level overview of the problem you are trying to solve.

• Gathering the right team - establish which people, both within and outside of your organization,  you will need to talk to to move forward.

• Understanding the technical infrastructure - understand the system in which the records are currently stored and how it is maintained and supported.

• Understanding the records and the processes which created them - understand how the records have been created, managed and stored.

• Understanding the digital objects - understand the individual files or digital objects to be preserved.

• Understanding the metadata - understand the metadata that is associated with the records

• Assessing the risks - understand the risks of leaving records within the record keeping system.

• Selecting a preservation approach - decide on which preservation approach to take.

• Making the case - make the case to carry out preservation actions.

• Taking action - carry out the planned preservation work.

• Processing the lessons learned - consider how to share and act on any lessons learned.

Some of the key challenges relating to the preservation of records from an EDRMS or other record keeping system are articulated below:

No one-size-fits-all solution

There isn’t a single simple approach to the preservation of records. If you are looking for a quick fix you are unlikely to find it in this resource! There are many record keeping systems available and numerous ways that institutions can configure and use a system. There are also numerous different approaches to digital preservation. The fact that different organizations tend to have different systems, different records management policies and different priorities leads to variations in how the preservation challenge will be tackled. 

Preservation at scale

Within a single record keeping system an organization may have tens of thousands of records and a proportion of these may need to be preserved for the long term. Some organizations will have more than one record keeping system they need to turn their attention to (perhaps including legacy systems). Any other challenges associated with preservation are magnified when attempting operations such as this at scale and this can be daunting.

Understanding how the system works and is used

Like so many digital preservation tasks, it is hard to move forward without really understanding what you are trying to preserve. It is important to work with relevant stakeholders to understand the record keeping system and how it is used. As is evidenced by this resource, a substantial amount of background work is required before preservation action can happen. Much of this resource therefore consists of lists of questions that you will need to ask in order to find out more. 

Knowing which records to keep

A records retention schedule is invaluable for establishing which records need to be kept for the long term, who is responsible for them and what actions should be taken on them (retain/transfer/destroy). In an ideal world these retention periods are built into or mapped to the record keeping system in use, allowing reports and queries to be created to inform the preservation process, but this will not always be the case. In some situations a more manual process will be required to establish how long records need to be retained, and therefore which require long-term preservation. In addition, system configuration can result in issues relating to records accessibility, for example security settings within the record keeping system may mean some content is difficult or impossible to access, while some business systems may overwrite or delete metadata or audit files after a set period of time. 

Knowing which version to keep

A record keeping system may keep multiple versions of the same record but the retention schedule may not go into this level of detail regarding which version/s to preserve. This situation can be further complicated when an organization keeps records in a mixture of analogue and digital formats. In some cases, the master version of a record may be a document that has since been printed out and physically signed, in which case, the original digital record may no longer be important. Decisions made on which version to keep should take into account how the record keeping system handles versions, what procedures are in place for creating versions and how record creators behave. Records and their versions need to be understood in context. For some organizations it may be logical to implement a policy to retain only the final version of a record. In other situations, users of the system may be able to flag up other versions that need to be preserved. It should be noted that the context of the organization and the records is important here - draft versions can be important legal or historical documents and should be assessed carefully.

Assessing the content - file formats

It can be a challenge to get an overview of exactly what is in a record keeping system. Unfortunately it is often not possible to get an accurate report on file formats. File format identification within a record keeping system such as an EDRMS typically looks only at MIME type or file extension and it is not always possible to run DROID or other file format identification tools over the content until it has been extracted out of the system. For planning purposes, a digital archivist may prefer to have that information at an earlier stage of the process. 

Metadata - too much or too little? 

The record keeping system may store lots of metadata, often in accordance with jurisdictional or international standards. This is clearly a good thing from a digital preservation perspective, but also brings challenges. It may not be straightforward to understand the different metadata fields and how and why they have been created or generated, or to establish which are useful to retain within the digital archive. Conversely, some systems may have very little metadata, or metadata that is challenging to access and/or extract.

Technical challenges

Depending on the preservation approach selected you may need to explore available options for export and/or transfer of records from the system in which they reside and establish a workflow or process. It may be possible to set up an automated transfer to your digital archive but this is technically complex and dependent on many factors including the availability of APIs, compatibility of systems and skills available. Emulation is an alternative preservation approach but comes with its own technical complexities.

User expectations

Providing access to records after they have been preserved is another challenge. If users have had these records easily accessible within their current record keeping system they may have high expectations for the same level of access once the records are in the archive. 

An ongoing challenge

Unless the record keeping system is a legacy system, it is likely that selection of records and transfer to the digital archive will be an activity that will be repeated over time. There are particular challenges associated with this. You will need to consider frequency of transfer, what happens to those records that have been transferred (will they exist in both locations or just the digital archive?), how you manage new versions of records, and how you ensure you don’t transfer the same record twice.

In his talk at the EDRMS Preservation Briefing Day, Jamie Dawes-Hughes from the Welsh Government provided a short case study highlighting some of the challenges encountered when attempting to export records from an EDRMS for submission to The National Archives UK - ‘Preserving the Welsh Government’s digital records’, a presentation at Unbroken records: A briefing day on Digital Preservation and EDRMS, 20th May 2021 (DPC members login to view the recording).

Preserving the Welsh Government’s digital records  - Jamie Dawes-Hughes (Welsh Government)

A record can be defined as "information created, received, and maintained as evidence and as an asset by an organization or person, in pursuit of legal obligations or in the transaction of business". (ISO 15489-1:2016) 

The term ‘record’ is commonly used in government, business, and archival settings while other domains often make use of other terms. Note that a recently published British Standard BS 10025:2021 (Code of practice for the management of records) has been written in a way which enables the reader to use whichever terms work best for them and their organisation to describe what they produce, receive and accumulate – whether that is 'data', 'documents', 'information' or 'records' (or a combination of these and other related terms). 

To ensure that records can be properly understood once removed from an EDRMS it is necessary to capture not only the information communicated as their content (often recorded in binary formats), but also the context of the transactions that created them and their relationships with other records, as well as the structure used to present information to users.   

For the sake of consistency this document uses the term ‘record’ to refer to the combination of  content, contextual and structural information held within a record keeping system identified as needing to be preserved. 

A record typically comprises:

• record content, for example:

  • Minutes from a meeting recorded in a Microsoft Word document

  • A technical report published as a PDF

  • A presentation recorded as a digital video

• record context, for example:

  • Who created a record?

  • Which business process created the record?

  • How does the record relate to other records or information?

• record structure, for example:

  • Which software created the record?

  • In what format was the record created?

  • How was it presented to users?

Contextual and structural information about records are usually recorded as metadata although it can often be difficult to capture from systems lacking sophisticated export functionality. Record metadata, may include:

• Descriptive (e.g. title, description, creator, creation date)

• Rights (e.g permissions, intellectual property)

• Technical (e.g. format, filename, size)

• Administrative (e.g. retention period, disposition date, audit logs)

To be trusted, a record must have the following characteristics:

• Authenticity: The record is what it purports to be, was created or sent by who purports to have done so, and was created or sent when it purports to have been.

• Reliability: The contents of the record can be trusted as a full and accurate representation of the facts to which they attest. 

• Integrity: The record is complete and unaltered. Any authorised additions or annotations are explicitly indicated and traceable.

• Usability: The record can be located, retrieved and presented in a timely manner. It should be linked to any related records including groupings or aggregations of records.

In many cases, an organization must ensure these characteristics are maintained for as long as the record exists. This brings particular challenges where the lifespan of the record is longer than that of the system in which it was stored.

Some records can be disposed of after a few years have passed but others have much longer retention periods and some must be retained indefinitely as archives. How these records (and their characteristics) can be effectively captured and preserved for the long term is a challenge faced by many organisations.

Scope

The EDMRS Preservation Toolkit relates to the preservation of digital records that are held in record keeping systems such as Electronic Document and Records Management Systems (EDRMS). 

For the purposes of this resource, record keeping systems are defined as the manual or automated applications, policies and processes implemented to capture, organize, and categorize records. Record keeping systems support the management, access, retrieval, use, and disposition of records. They include both EDRMS and document-centric collaboration platforms such as Sharepoint, Office365 and Google Drive. There are many other systems in use within organizations that store records, for example, systems used to store Financial or Human Resources records. This resource hasn’t been created with these systems in mind, but it is anticipated that some of the advice may be more broadly applicable.

The advice within this guide applies equally to current systems in use, legacy systems that are falling out of use, and systems that are being procured for the future. Whichever stage of this lifecycle your document or records management system is in, if it contains records that need to be maintained for the long term, there are preservation considerations that should be addressed.

A record keeping system tends to facilitate the storage and use of current records. Though many of the features of a record keeping system overlap with those you might expect to find in a digital preservation system, they typically don’t support all the functionality that you would expect to find within a digital archive (for example, active monitoring of checksums and the migration of files into new formats for preservation and/or access).

This resource discusses several different approaches to the challenges of preserving records from a record keeping system.

1. 1. Transfer records to a digital archive

   2. Emulate the system or application that holds the records

   3. Leave in current system and manage ‘in situ’

   4. Migrate records to a new record keeping system

Generic advice on how to approach the challenge of preserving records and make a decision on preservation approach is included within this toolkit but detailed instructions on how to carry out the preservation actions on a particular system are out of scope.

A presentation by James Doig from the National Archives of Australia at the DPC’s EDRMS Briefing Day in May 2021 gives a good introduction and overview to the topic of the preservation of record keeping systems, particularly with regard to government record keeping and the challenges we face (Members login to view recording).

Preservation Across the Digital Continuum: The National Archives of Australia’s Experience With EDRMS Preservation (transcript available here) - James Doig (National Archives of Australia) 

Audience

Many different organizations can be charged with the task of preserving records. The advice within this resource is intended to be broad enough to be applicable to any type or size of organization faced with this challenge whether they are a national archive, a business archive or a local record office and whether they are responsible for preserving records from a wide range of external sources or their own internal record keeping system/s.

The challenge of preserving records is a global one. The issues and solutions are broadly the same across geographic borders but it should be noted that different record keeping practices (and in some cases terminology) may be in place. 

This resource is aimed at:

• Those involved in managing records

• Those involved in preserving records

• Those involved in developing relevant policies and procedures

• Those managing and overseeing relevant projects and services

• Those involved in supporting relevant systems and software

• Those involved in procuring relevant systems and software or developing business requirements

The toolkit assumes some knowledge of basic digital preservation concepts. The Digital Preservation Handbook provides a good introduction to digital preservation and a helpful glossary of terms.

This toolkit provides advice and guidance on the preservation of records held within an Electronic Document and Records Management Systems (EDRMS) or other record keeping system. As there are numerous different technologies in use and ways of working, there is no one-size-fits-all approach to this challenge. This guide is therefore not intended to be prescriptive, but instead provides general information and guidance on the areas that should be considered in order to define an effective approach to this challenge.  This toolkit will: flag up some of the key challenges that you may face signpost some of the things you should consider provide you with the questions that you need to ask within your own organisation map-out a process for moving forward provide examples of how others have tackled the challenge The toolkit is illustrated with case studies, demonstrating real-life applications of the guidance and principles, and signposts other useful sources of information to help inform your work. Please contact us if you have a relevant case study you would be happy to share.

The EDRMS Preservation Toolkit was created by the DPC EDRMS Preservation Task Force who came together in 2020-2021 to share their knowledge and experience on this topic. Further information on the task force and how this resource was created can be found here.

The EDRMS Preservation Task Force was established by the DPC as a result of a digital preservation project with the Nuclear Decommissioning Authority and our thanks go to them for supporting this work.

Computational access is a term that is mentioned with increasing frequency by those in the digital preservation community. Many practitioners are aware it might be helpful to them (and indeed to their users) but do not have an understanding of what exactly it entails, how it is best applied and perhaps most importantly, where to start. This guide has been created to provide an introduction to this topic and to help the community move forward in applying computational access techniques.

This guide was created collaboratively in 2022 by Leontien Talboom of University College London alongside the DPC and a panel of community experts. Further information about the creation of this guide can be found here.

Our thanks go to the Software Sustainability Institute who funded elements of this work.

Previous versions of DPC Rapid Assessment Model are available below:

Version 1

Version 1 (2019) - The DPC Rapid Assessment Model - the full model and worksheet

The first version of DPC RAM was officially launched in September 2019 in the Lightning Talks session of the iPRES conference in Amsterdam and with the following blog post: Introducing the DPC RAM

Version 1 translations:

• Français  - Grille d'évaluation rapide (French)
• 日本語  - デジタル保存連合 ラピッド・アセスメントモデル (Japanese)
• Português - Modelo de avaliação rápida (Portuguese)

 Version 2

Version 2 (2021) - The DPC Rapid Assessment Model - the full model and worksheet

Version 2 of RAM was released in March 2021. A summary of some of the changes that were made can be found in the following blog post: DPC RAM (version 2) - what has changed and why?

Version 2 translations:

• 简体中文 - 数字保存联盟快速评估模型（Simplified Chinese)

• 繁體中文 - 數位保存聯盟快速評估模型 (Traditional Chinese)

• Nederlands - Het Rapid Assessment Model (Dutch)

• Espanol - Modelo de Evaluación Rápida (Spanish) /  DPC RAM hoja de trabajo

• Français  - Grille d'évaluation rapide (French)

• Italiano - Il modello di valutazione rapida (Italian)

• Portuguesa - Modelo de Avaliação Rápida (Portuguese)

• 日本 - デジタル保存連合 ラピッド・アセスメントモデル (Japanese) 

• Türkçe - Sayısal Koruma Koalisyonu Hızlı Değerlendirme Modeli ve Analiz Cetveli  (Turkish)

The following tools provide detailed guidance on advancing or refreshing digital preservation capabilities within an organisation. Please note that elements of some of these resources are available to members only, so please log in to access them.

	DPC Rapid Assessment Model Use this to benchmark your digital preservation maturity and inform planning and prioritisation of future preservation activities
	Digital Preservation Policy Toolkit Use this to help develop a new or revised digital preservation policy
	Executive Guide on Digital Preservation Use this to help advocate and communicate effectively with senior managment. Also see Make The Case for Digital Preservation in Your Organisation
	Digital Preservation Business Case Toolkit Use this to discover additional resources to enhance your procurement experience
	Digital Preservation Procurement Toolkit Use this to help deliver a successful procurement of third party digital preservation services or infrastructure
	EDRMS Preservation Toolkit Use this to get guidance on the areas that should be considered in order to define an effective approach to this challenge.
	Computational Access: A beginner's guide  Use this to gain an understanding of computational access techniques and approaches and to find out how to get started
	Digital Preservation Documentation: a guide Use this for advice on how to create and manage high quality documentation.
	Digital Asset Register Toolkit Use this for practical guidance on how to build, maintain, and use a Digital Asset Register (DAR)
	Community Archives Digital Preservation Toolkit Use this if you are a Community Group with digital content or collections you would like to preserve for long term use and access.