DPC

Electronic hospital and medical records

   Endangered large

Personal medical records and records of hospital treatment are increasingly, if not uniformly born digital. By implication those records should be retained trough the life-time of the patient and perhaps longer as required for intergenerational study; and yet there is little evidence of the medical profession participating in the digital preservation community.

Group: Sensitive Data

Trend: New Entry

Consensus Decision

Added to List: 2017

Last update: 2017

Previous category: Of Concern

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Medical scans; records of treatment and care plans; health advice and notifications;

‘Critically Endangered’ in the Presence of Aggravating Conditions

Loss of context; loss of authenticity or integrity; poor storage; lack of understanding; churn of staff; significant volumes of data; significant diversity of data; ill-informed records management; poorly developed transfer and integrity checking; poorly developed migration or normalizations specifications; longstanding protocols or procedures that apply unsuitable paper processes to digital materials; encryption

‘Vulnerable’ in the Presence of Good Practice

Well managed data infrastructure; preservation enabled at the point of creation; carefully managed authenticity; use of persistent identifiers; well managed records management processes; application of records management standards; recognition of preservation requirements at highest levels; strategic investment in digital preservation; preservation roadmap; participation in digital preservation community.

2019 Review

Medical and hospital records were introduced to the BitList in 2017 though at that time there was limited capacity to address the topic and it was simply published as ‘of concern’. It would have been reviewed in 2019 in any case but was independently received as a submission to the open nomination process. This topic remains large and ill defined. It is useful to draw attention to the scale of the digital preservation challenges which arise in hospitals and in the medical profession, but it would be useful in future years to split this entry into more discrete elements which would be more impactful.

Additional Jury Comments

Increasing sensitivity and awareness of data protection requirements could act inadvertently as a barrier to lifecycle data management. It is striking how little evidence there is of the health technology companies participating in the global digital preservation community.

Read More

Email

   Endangered large

Documents, correspondence and other records created in the course of contractual dealings between individuals and agencies, especially where the subjects are of long duration and may be subject to legal scrutiny at undefined points in the distant future.

Group: Formats

Trend: No change

Consensus Decision

Added to List: 2017

Last update: 2017

Previous category: Of Concern

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools or services within this group would have a global impact.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Email in all its forms, including individual messages, threads of conversation, mailboxes, email servers and file attachments.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Conflicting and unmanaged IPR; use of personal accounts for professional work and vice versa; proliferation and duplication of attachments; email not recognized as a record; absent, unworkable or inconsistent records management; dependence on free cloud-based services; lack of migration path; lack of preservation planning; perverse incentives to delete; encryption.

Vulnerable in the Presence of Good Practice

Application of appraisal and selection tools; timely transfer to preservation facility or archive; commitment to transparency; preservation policy; working preservation plan; clear migration path; widespread recognition of email as a record.

2019 Review

This entry was made in 2017 which the Jury did not have capacity to assess. It has been assessed this time in part to highlight significant developments since then including the recommendations of the Email Preservation Taskforce and the development of the ePADD software. Email presents many kinds of preservation challenge, from scale through core technologies, attachments, privacy and intellectual property rights. Because this entry intersects with many others, the presence of email within these other entries should be considered an aggravating condition.

Additional Jury Comments

Email is hugely important as it has been so pervasive as a communication mechanism for society. Some methods used and responsibility adopted for collecting at business and public body level (again will differ globally) but this will be a fraction of the communities that use it and few will be set up for the long-term care of this data.

See also:

·        Prom, C (2019) Preserving Email (2nd Edition) DPC Technology Watch Report 19-01 online at: http://dx.doi.org/10.7207/twr19-01 (from 7th November)

·        Murray K and Prom, C (2018) The Future of Email Archives: A Report from the Task Force on Technical Approaches for Email Archives, CLIR, online at: https://clir.wordpress.clir.org/wp-content/uploads/sites/6/2018/08/CLIR-pub175.pdf

Read More

Legacy Media Art

   Endangered large

Media art in storage or not otherwise displayed but where the artists or technicians are available to support installation

Group: Media Art

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Media art in storage

‘Critically Endangered’ in the Presence of Aggravating Conditions

Lack of documentation to enable maintenance; lack of clarity with respect to intellectual property; complex interdependencies on specific hardware, software or operating systems; lack of capacity in the gallery or workshop; lack of strategic investment; complex external dependencies; loss of institutional memory resulting from staff churn; poor working relationship between the gallery and artist/workshop; lack of conservation assessment.

Vulnerable in the Presence of Good Practice

Strong documentation; clarity of preservation path and ensuing responsibilities; proven preservation plan; capacity of workshop to support re-installation; capacity of gallery to conserve; capacity of gallery to re-install; retention of institutional memory including archives of correspondence between gallery and artist/workshop; strong and continuing working relationship between the gallery and artist/workshop; regular conservation assessment.

2019 Review

Media Art was introduced in 2017, though with particular reference to historical media art which was categorised as ‘critically endangered’. The jury introduced this entry to ensure greater specificity in its recommendation. It is intended to represent works held in galleries but no longer displayed, but where there is a continuing working relationship between the gallery and the artist or workshop and reasonable expectation that support for preservation could still be obtained when required.

Additional Jury Comments

This entry attempts to capture a point in the lifecycle of media art where preservation risks are increasing but not yet critical. There is a risk that preservation issues will not become apparent until the piece is brought out of storage when being considered for loan or exhibition - often on timescales that make it too late to address preservation concerns effectively. Galleries should be aware that the range of data/formats/hardware/software embedded in media art can be wide and vary at different speeds.

Read More

Massively Multiple Online Gaming Platforms and Experiences

   Endangered large

Massively Multiple Online Gaming is an evolving, transient but significant cultural. Gameplay is referenced here particularly as means of participation, along with social media and in-game interaction between players. Video streaming of game content, the means of viewing but not participating in game play, has a septate entry.

Group: Gaming

Trend: no change

Consensus Decision

Added to List: new entry

Last update: new entry

Previously: Critically Endangered

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

Loss seems likely: by the time tools or techniques have been developed the material will likely have been lost.

Examples

There are numerous examples for this entry, so the example of Fortnite, a 2017 free to play online game will serve to illustrate. As Fortnite is freely accessible to play, the game is at the forefront of internet culture with over 250 million registered users as of March 2019. In addition, Fortnite's parent company, Epic Games, provides access to Unreal Engine, a software platform consisting of integrated tools for game developers, with instructional guides, game simulations and learning tools. Unreal Engine was first released in 1998 and is currently in its 4th release. At risk also is this software, its associated tools and code which is key digital evidence of MMOG evolution, learning, creativity and expression across multiple platforms (PC, console, mobile).

‘Critically Endangered’ in the Presence of Aggravating Conditions

Controversies around IPR; lack of offline backup; changing business model of providers; limited recognition of value of game play; over dependence on goodwill of ad-hoc community; lack of preservation know-how at service providers; dependency on bespoke hardware or interfaces.

‘Vulnerable’ in the Presence of Good Practice

Well documented code; IPR supportive of preservation; large and committed user community

2019 Review

This is a subset of an entry made in 2017 for ‘Gaming’ which the Jury has split into four discrete entries. By creating this entry for Game Play, the Jury encourages greater consideration of the technical complexities which arise from preservation of software environments as well as the cultural and historic value which these games are likely to acquire. It appears that this entry is categorised as a lower risk than in 2017, but this is a function of splitting that entry.

Additional Jury Comments

The difficulty of saving game play nicely encapsulates why video recordings of (online) gameplay is important: we are never going to be able to recreate the experience of playing something like World of Warcraft or Fortnite at their peak (or at their inception, or at any other point in time). They will never have the same configuration of subscribers, to say nothing of the innumerable changes made to the software over the years, which have significantly altered how the game works and looks. Loss is inevitable, and it’s already happened. The social and cultural aspects of play are incredibly important and on-screen recording is the most robust way to capture that.

Read More

Master Digital Music and Sound Recordings

   Endangered large

Master recordings of music and other performance from which retail products are derived, typically in multiple tracks and uncompressed high-resolution sound quality

Group: Sound and Vision

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a small effort to address losses in this group, requiring the application of proven preservation tools or techniques.

Examples

Master recordings owned by music industry

‘Critically Endangered’ in the Presence of Aggravating Conditions

Single point of failure; storage on old or degrading media; lack of ongoing investment in changing preservation requirements; lack of capability; poor documentation; dependence on small staff

‘Vulnerable’ in the Presence of Good Practice

High quality storage; meticulous and consistent replication; trusted repository; preservation requirement understood at executive level and funded accordingly; leadership in preservation community; expert staff

2019 Review

This entry has been split out of the previous entry for ‘Digital Music Production and Sharing’ though it has overlaps with other entries. It is a separate entry to emphasise the inherent and very great value of master recordings over and above those distributed, and the concomitant need for active preservation. The Jury were particularly aware of the case of the ‘Universal Fire’ of 2008 which has only recently been reported widely.

Additional Jury Comments

The archival practices of the studios are typically based on value - the recordings are assumed to be worth keeping. However, this means relatively low-value masters may not be transferred to new media in a timely way and could be lost. There is no comprehensive deposit scheme to address the long-tail of music production, and it is often unclear exactly where responsibility lies.

Read More

Orphaned Works

   Endangered large

Digital materials where copyright is uncertain, disputed or unknowable meaning that preservation actions are constrained or prevented.

Group: Orphaned Works

Trend: New Entry

Consensus Decision

Added to List: 2017

Last update: 2019

Previous category: Endangered

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Photographs, music recordings, literature. A specific example is elements of The National Disc of the BBC Domesday Project outside of legal deposit mandate where the copyright owner cannot be traced.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Lack of documentation; dependencies resulting from hardware, software or media; lack of use resulting in lack of priority; lack of strategic investment in digital preservation; workflows that inhibit preservation of content that has not been licensed; encryption; poor storage.

Vulnerable in the Presence of Good Practice

Preservation pathway enabled; proven preservation plan applied; active effort to resolve IPR issues; institutional willingness to take risks for preservation.

2019 Review

There is little evidence of any renewed effort to address the issue of orphaned works since 2017. Improvement to the baseline competence of the archival and library professions in their understanding of copyright and the skills to preserve contents, provides a narrow basis for optimism in some contexts but the scale of the challenge is likely to have grown just as quickly if not more so, and aggravating conditions become more prevalent too.

Additional Jury Comments

The Jury would encourage organizations to take a risk-based approach which would help them preserve collections. Copyright infringements are only likely to become a significant issue in the context of access, and in most cases the likelihood of any specific action is small. Preservation needs to be presented as a social good, one without which copyright holders would simply be unable to benefit from the property rights they seek to protect.

Read More

PDF other than PDF/A

   Endangered large

Documents presented in PDF (Portable Document Format) format (ISO 32000:1 and ISO 32000:2) and other data wrapped inside them, other than PDF/A but including all other variants and versions.

Group: Formats

Trend: No change

Consensus Decision

Added to List: 2017

Last update: 2018

Previous category: Of Concern

Imminence of Action

Action is recommended within five years, detailed assessment within three years.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world..

Effort to Preserve

It would require a small effort to address losses in this group, requiring the application of proven preservation tools or techniques.

Examples

PDF 1.1, 1.2, 1.3, 1.4 (excluding PDF/A as a subset), 1.5, 1.6, 1.7 and 2.0. PDF/X and PDF/E

‘Critically Endangered’ in the Presence of Aggravating Conditions

Loss of context; loss of authenticity or integrity; external dependencies; poor storage; lack of understanding; significant diversity of data; poorly developed digitization specifications; lack of integrity checking; poorly developed migration or normalizations specifications; lack of virus control; poor storage or replication; lack of validation at the point of creation; encryption.

‘Vulnerable’ in the Presence of Good Practice

Well managed data infrastructure; preservation planning; authenticity managed; use of persistent identifiers; reduction of dependencies; application of records management standards; recognition of preservation requirements beyond formats; strategic investment in digital preservation; preservation roadmap; participation in digital preservation community; format validation.

2019 Review

The judges have introduced this entry as a subset of a previous entry for ‘PDF’, emphasizing the different threats faced by different types of PDF. PDF/A explicitly reduces dependencies and thus curtails preservation risks for certain types of content: PDFs of other types do not. PDF and PDF/A have sometimes been misunderstood as a generic solution to digital preservation requirements. In the eyes of the judges it can only offer a preservation solution when embedded within a wider preservation infrastructure.

Additional Jury Comments

See also:

Fanning, B (2017) Preserving with PF/A (Second Edition), DPC Technology Watch Report 17-01 online at http://dx.doi.org/10.7207/twr17-01

Read More

Premium or institutional social media services

   Endangered large

Commercial social media services which are based on a costed subscription and contract.

Group: Social Media

Trend: no change

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Premium versions of Vimeo, Flickr, Yammer, Slack, Mircosoft Teams and others.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Unstable business models and pricing schedules from providers; lack of export functionality; unstable terms and conditions; lack of onsite copy of key media; lack of strategic plan for IT provision; confusion on IPR; asymmetrical contract arrangements; conflating preservation and access.

‘Vulnerable’ in the Presence of Good Practice

Offline back up for key media; fit to preservation and records management plan; strategic roadmap for adoption of social media;

2019 Review

The judges have introduced this entry as a subset of a previous entry, emphasizing the different threats faced by services that are ‘paid-for’ versus ‘free-at-the-point-of-use’. Both depend on the business model of the vendor and the terms and conditions which they impose. For this group, the business model and sustainability are more obvious and contracts may be enforceable more readily. Moreover, because these services have a slightly higher barrier to entry they may be favoured by agencies better able to respond to closure or loss.

Additional Jury Comments

Traditional web archiving can be employed where the user pays for a service but content is ultimately publicly available (such as Flickr) where the user may pay for a full account to be able to host more photos publicly. But much is unclear about how to preserve internal social media / closed networks that web archiving can't get to or existing tools don't cover. The growth in use of these products for communication and social networking leads us to think this is endangered rather than vulnerable. It’s possible that existing tools could be modified to tackle some of these closed networks but that is likely to require some investments, perhaps related to corporate records in some cases (thinking about internal Slacks, for instance), and more education about the importance of preserving this material and not trusting the publishing platforms to host the content forever.

Read More

Pre-Production TV and Movie Materials

   Endangered large

Digital records of the creative and production process for film and television, such as initial designs, screenplay and script, on set still photography, rushes or out-takes that are not included in the final production and therefore not available to on-air broadcast archives or film libraries

Group: Sound and Vision

Trend: New Entry

Consensus Decision

Added to List: 2017

Last update: 2019

Previous category: Of Concern

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

TV and Movie production archives in digital form; outputs of script management software; drafts of screenplay; continuity photography; costume design; set design; lighting and sound design.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Lack of custodial responsibility; confusion over intellectual property rights; lack of appraisal; lack of recognition of preservation at executive level;

‘Vulnerable’ in the Presence of Good Practice

Preservation responsibility understood and acted upon; preservation infrastructure and planning for key items; access and use of collections to inform subsequent productions

2019 Review

This entry was first proposed in 2017 and noted as being ‘of concern’, though the Jury did not have the capacity to assess the entry thoroughly. Additional expertise has been recruited to the Jury this year to help with the assessment.

Additional Jury Comments

These materials are not being collected in any coherent way and will be lost as they are not valued by production companies, the space needed to hold them is costly and the expertise needed to catalogue and collect them is lacking. In short, nobody really cares. This is not primarily a technical problem and advocacy is needed urgently.

Read More

Proceedings and Evidence in Court

   Endangered large

Digital materials presented in court as evidence or documents such as rulings and proceedings generated through legal proceedings

Group: Digital Legal Records

Trend: No change

Consensus Decision

Added to List: 2017

Last update: 2018

Previous category: Endangered

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Evidence submitted to courts of all kinds including text messages, photography, CCTV, email, 3d and 2d scanning, scientific reports and analyses, documents and websites; digital record of proceedings; digital records of rulings and all manner of quasi-judicial proceedings and tribunals.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Loss of context; loss of integrity; external dependencies; poor storage; lack of understanding; churn of staff; significant or diversity of data; poorly developed specifications; ill-informed records management; poorly developed transfer protocols; poorly developed migration or normalization; longstanding protocols or procedures that apply unsuitable paper processes to digital materials.

‘Vulnerable’ in the Presence of Good Practice

Well managed data infrastructure; preservation enabled at ingest; carefully managed authenticity; use of persistent identifiers; finding aids; well managed records management processes; recognition of preservation requirements at highest levels; strategic investment in digital preservation; preservation roadmap; participation in digital preservation community.

2019 Review

This entry is a subset of an entry made in 2017 for ‘Digital Legal Records and Evidence’ which the Jury has split into four more discrete entries. This category includes evidence that has been presented as evidence in court, as well as court proceedings. It recognizes that courts are not limited in the types of evidence that they can admit but that they have a responsibility to provide robust preservation that ensures the authenticity of their records and evidence.

Additional Jury Comments

Standard Records Management processes within designated agencies should be able to take care of the preservation of materials like this but given that evidence is likely to involve complex types of data, such agencies may not be equipped to deliver preservation effectively. It is surprising that courts are not more obvious in the digital preservation community, where solutions now exist.

More concrete examples would be welcome. It is the evidentiary value of submissions to court that may be lost, and therefore veracity of decision could be questioned. Evidence submitted in digital form is of a greater risk (e.g a video file submitted on a CD in the 90s) than records of the proceedings themselves (e.g. transcripts).

Read More

Scroll to top