Pre-Production TV and Movie Materials

   Endangered large

Digital records of the creative and production process for film and television, such as initial designs, screenplay and script, on set still photography, rushes or out-takes that are not included in the final production and therefore not available to on-air broadcast archives or film libraries

Group: Sound and Vision

Trend: New Entry

Consensus Decision

Added to List: 2017

Last update: 2019

Previous category: Of Concern

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

TV and Movie production archives in digital form; outputs of script management software; drafts of screenplay; continuity photography; costume design; set design; lighting and sound design.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Lack of custodial responsibility; confusion over intellectual property rights; lack of appraisal; lack of recognition of preservation at executive level;

‘Vulnerable’ in the Presence of Good Practice

Preservation responsibility understood and acted upon; preservation infrastructure and planning for key items; access and use of collections to inform subsequent productions

2019 Review

This entry was first proposed in 2017 and noted as being ‘of concern’, though the Jury did not have the capacity to assess the entry thoroughly. Additional expertise has been recruited to the Jury this year to help with the assessment.

Additional Jury Comments

These materials are not being collected in any coherent way and will be lost as they are not valued by production companies, the space needed to hold them is costly and the expertise needed to catalogue and collect them is lacking. In short, nobody really cares. This is not primarily a technical problem and advocacy is needed urgently.

Read More

Email

   Endangered large

Documents, correspondence and other records created in the course of contractual dealings between individuals and agencies, especially where the subjects are of long duration and may be subject to legal scrutiny at undefined points in the distant future.

Group: Formats

Trend: No change

Consensus Decision

Added to List: 2017

Last update: 2017

Previous category: Of Concern

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools or services within this group would have a global impact.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Email in all its forms, including individual messages, threads of conversation, mailboxes, email servers and file attachments.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Conflicting and unmanaged IPR; use of personal accounts for professional work and vice versa; proliferation and duplication of attachments; email not recognized as a record; absent, unworkable or inconsistent records management; dependence on free cloud-based services; lack of migration path; lack of preservation planning; perverse incentives to delete; encryption.

Vulnerable in the Presence of Good Practice

Application of appraisal and selection tools; timely transfer to preservation facility or archive; commitment to transparency; preservation policy; working preservation plan; clear migration path; widespread recognition of email as a record.

2019 Review

This entry was made in 2017 which the Jury did not have capacity to assess. It has been assessed this time in part to highlight significant developments since then including the recommendations of the Email Preservation Taskforce and the development of the ePADD software. Email presents many kinds of preservation challenge, from scale through core technologies, attachments, privacy and intellectual property rights. Because this entry intersects with many others, the presence of email within these other entries should be considered an aggravating condition.

Additional Jury Comments

Email is hugely important as it has been so pervasive as a communication mechanism for society. Some methods used and responsibility adopted for collecting at business and public body level (again will differ globally) but this will be a fraction of the communities that use it and few will be set up for the long-term care of this data.

See also:

·        Prom, C (2019) Preserving Email (2nd Edition) DPC Technology Watch Report 19-01 online at: http://dx.doi.org/10.7207/twr19-01 (from 7th November)

·        Murray K and Prom, C (2018) The Future of Email Archives: A Report from the Task Force on Technical Approaches for Email Archives, CLIR, online at: https://clir.wordpress.clir.org/wp-content/uploads/sites/6/2018/08/CLIR-pub175.pdf

Read More

Legacy Media Art

   Endangered large

Media art in storage or not otherwise displayed but where the artists or technicians are available to support installation

Group: Media Art

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Media art in storage

‘Critically Endangered’ in the Presence of Aggravating Conditions

Lack of documentation to enable maintenance; lack of clarity with respect to intellectual property; complex interdependencies on specific hardware, software or operating systems; lack of capacity in the gallery or workshop; lack of strategic investment; complex external dependencies; loss of institutional memory resulting from staff churn; poor working relationship between the gallery and artist/workshop; lack of conservation assessment.

Vulnerable in the Presence of Good Practice

Strong documentation; clarity of preservation path and ensuing responsibilities; proven preservation plan; capacity of workshop to support re-installation; capacity of gallery to conserve; capacity of gallery to re-install; retention of institutional memory including archives of correspondence between gallery and artist/workshop; strong and continuing working relationship between the gallery and artist/workshop; regular conservation assessment.

2019 Review

Media Art was introduced in 2017, though with particular reference to historical media art which was categorised as ‘critically endangered’. The jury introduced this entry to ensure greater specificity in its recommendation. It is intended to represent works held in galleries but no longer displayed, but where there is a continuing working relationship between the gallery and the artist or workshop and reasonable expectation that support for preservation could still be obtained when required.

Additional Jury Comments

This entry attempts to capture a point in the lifecycle of media art where preservation risks are increasing but not yet critical. There is a risk that preservation issues will not become apparent until the piece is brought out of storage when being considered for loan or exhibition - often on timescales that make it too late to address preservation concerns effectively. Galleries should be aware that the range of data/formats/hardware/software embedded in media art can be wide and vary at different speeds.

Read More

Massively Multiple Online Gaming Platforms and Experiences

   Endangered large

Massively Multiple Online Gaming is an evolving, transient but significant cultural. Gameplay is referenced here particularly as means of participation, along with social media and in-game interaction between players. Video streaming of game content, the means of viewing but not participating in game play, has a septate entry.

Group: Gaming

Trend: no change

Consensus Decision

Added to List: new entry

Last update: new entry

Previously: Critically Endangered

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

Loss seems likely: by the time tools or techniques have been developed the material will likely have been lost.

Examples

There are numerous examples for this entry, so the example of Fortnite, a 2017 free to play online game will serve to illustrate. As Fortnite is freely accessible to play, the game is at the forefront of internet culture with over 250 million registered users as of March 2019. In addition, Fortnite's parent company, Epic Games, provides access to Unreal Engine, a software platform consisting of integrated tools for game developers, with instructional guides, game simulations and learning tools. Unreal Engine was first released in 1998 and is currently in its 4th release. At risk also is this software, its associated tools and code which is key digital evidence of MMOG evolution, learning, creativity and expression across multiple platforms (PC, console, mobile).

‘Critically Endangered’ in the Presence of Aggravating Conditions

Controversies around IPR; lack of offline backup; changing business model of providers; limited recognition of value of game play; over dependence on goodwill of ad-hoc community; lack of preservation know-how at service providers; dependency on bespoke hardware or interfaces.

‘Vulnerable’ in the Presence of Good Practice

Well documented code; IPR supportive of preservation; large and committed user community

2019 Review

This is a subset of an entry made in 2017 for ‘Gaming’ which the Jury has split into four discrete entries. By creating this entry for Game Play, the Jury encourages greater consideration of the technical complexities which arise from preservation of software environments as well as the cultural and historic value which these games are likely to acquire. It appears that this entry is categorised as a lower risk than in 2017, but this is a function of splitting that entry.

Additional Jury Comments

The difficulty of saving game play nicely encapsulates why video recordings of (online) gameplay is important: we are never going to be able to recreate the experience of playing something like World of Warcraft or Fortnite at their peak (or at their inception, or at any other point in time). They will never have the same configuration of subscribers, to say nothing of the innumerable changes made to the software over the years, which have significantly altered how the game works and looks. Loss is inevitable, and it’s already happened. The social and cultural aspects of play are incredibly important and on-screen recording is the most robust way to capture that.

Read More

Master Digital Music and Sound Recordings

   Endangered large

Master recordings of music and other performance from which retail products are derived, typically in multiple tracks and uncompressed high-resolution sound quality

Group: Sound and Vision

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a small effort to address losses in this group, requiring the application of proven preservation tools or techniques.

Examples

Master recordings owned by music industry

‘Critically Endangered’ in the Presence of Aggravating Conditions

Single point of failure; storage on old or degrading media; lack of ongoing investment in changing preservation requirements; lack of capability; poor documentation; dependence on small staff

‘Vulnerable’ in the Presence of Good Practice

High quality storage; meticulous and consistent replication; trusted repository; preservation requirement understood at executive level and funded accordingly; leadership in preservation community; expert staff

2019 Review

This entry has been split out of the previous entry for ‘Digital Music Production and Sharing’ though it has overlaps with other entries. It is a separate entry to emphasise the inherent and very great value of master recordings over and above those distributed, and the concomitant need for active preservation. The Jury were particularly aware of the case of the ‘Universal Fire’ of 2008 which has only recently been reported widely.

Additional Jury Comments

The archival practices of the studios are typically based on value - the recordings are assumed to be worth keeping. However, this means relatively low-value masters may not be transferred to new media in a timely way and could be lost. There is no comprehensive deposit scheme to address the long-tail of music production, and it is often unclear exactly where responsibility lies.

Read More

Orphaned Works

   Endangered large

Digital materials where copyright is uncertain, disputed or unknowable meaning that preservation actions are constrained or prevented.

Group: Orphaned Works

Trend: New Entry

Consensus Decision

Added to List: 2017

Last update: 2019

Previous category: Endangered

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Photographs, music recordings, literature. A specific example is elements of The National Disc of the BBC Domesday Project outside of legal deposit mandate where the copyright owner cannot be traced.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Lack of documentation; dependencies resulting from hardware, software or media; lack of use resulting in lack of priority; lack of strategic investment in digital preservation; workflows that inhibit preservation of content that has not been licensed; encryption; poor storage.

Vulnerable in the Presence of Good Practice

Preservation pathway enabled; proven preservation plan applied; active effort to resolve IPR issues; institutional willingness to take risks for preservation.

2019 Review

There is little evidence of any renewed effort to address the issue of orphaned works since 2017. Improvement to the baseline competence of the archival and library professions in their understanding of copyright and the skills to preserve contents, provides a narrow basis for optimism in some contexts but the scale of the challenge is likely to have grown just as quickly if not more so, and aggravating conditions become more prevalent too.

Additional Jury Comments

The Jury would encourage organizations to take a risk-based approach which would help them preserve collections. Copyright infringements are only likely to become a significant issue in the context of access, and in most cases the likelihood of any specific action is small. Preservation needs to be presented as a social good, one without which copyright holders would simply be unable to benefit from the property rights they seek to protect.

Read More

PDF other than PDF/A

   Endangered large

Documents presented in PDF (Portable Document Format) format (ISO 32000:1 and ISO 32000:2) and other data wrapped inside them, other than PDF/A but including all other variants and versions.

Group: Formats

Trend: No change

Consensus Decision

Added to List: 2017

Last update: 2018

Previous category: Of Concern

Imminence of Action

Action is recommended within five years, detailed assessment within three years.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world..

Effort to Preserve

It would require a small effort to address losses in this group, requiring the application of proven preservation tools or techniques.

Examples

PDF 1.1, 1.2, 1.3, 1.4 (excluding PDF/A as a subset), 1.5, 1.6, 1.7 and 2.0. PDF/X and PDF/E

‘Critically Endangered’ in the Presence of Aggravating Conditions

Loss of context; loss of authenticity or integrity; external dependencies; poor storage; lack of understanding; significant diversity of data; poorly developed digitization specifications; lack of integrity checking; poorly developed migration or normalizations specifications; lack of virus control; poor storage or replication; lack of validation at the point of creation; encryption.

‘Vulnerable’ in the Presence of Good Practice

Well managed data infrastructure; preservation planning; authenticity managed; use of persistent identifiers; reduction of dependencies; application of records management standards; recognition of preservation requirements beyond formats; strategic investment in digital preservation; preservation roadmap; participation in digital preservation community; format validation.

2019 Review

The judges have introduced this entry as a subset of a previous entry for ‘PDF’, emphasizing the different threats faced by different types of PDF. PDF/A explicitly reduces dependencies and thus curtails preservation risks for certain types of content: PDFs of other types do not. PDF and PDF/A have sometimes been misunderstood as a generic solution to digital preservation requirements. In the eyes of the judges it can only offer a preservation solution when embedded within a wider preservation infrastructure.

Additional Jury Comments

See also:

Fanning, B (2017) Preserving with PF/A (Second Edition), DPC Technology Watch Report 17-01 online at http://dx.doi.org/10.7207/twr17-01

Read More

Premium or institutional social media services

   Endangered large

Commercial social media services which are based on a costed subscription and contract.

Group: Social Media

Trend: no change

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Premium versions of Vimeo, Flickr, Yammer, Slack, Mircosoft Teams and others.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Unstable business models and pricing schedules from providers; lack of export functionality; unstable terms and conditions; lack of onsite copy of key media; lack of strategic plan for IT provision; confusion on IPR; asymmetrical contract arrangements; conflating preservation and access.

‘Vulnerable’ in the Presence of Good Practice

Offline back up for key media; fit to preservation and records management plan; strategic roadmap for adoption of social media;

2019 Review

The judges have introduced this entry as a subset of a previous entry, emphasizing the different threats faced by services that are ‘paid-for’ versus ‘free-at-the-point-of-use’. Both depend on the business model of the vendor and the terms and conditions which they impose. For this group, the business model and sustainability are more obvious and contracts may be enforceable more readily. Moreover, because these services have a slightly higher barrier to entry they may be favoured by agencies better able to respond to closure or loss.

Additional Jury Comments

Traditional web archiving can be employed where the user pays for a service but content is ultimately publicly available (such as Flickr) where the user may pay for a full account to be able to host more photos publicly. But much is unclear about how to preserve internal social media / closed networks that web archiving can't get to or existing tools don't cover. The growth in use of these products for communication and social networking leads us to think this is endangered rather than vulnerable. It’s possible that existing tools could be modified to tackle some of these closed networks but that is likely to require some investments, perhaps related to corporate records in some cases (thinking about internal Slacks, for instance), and more education about the importance of preserving this material and not trusting the publishing platforms to host the content forever.

Read More

Semi-Published Research Data

   Endangered large

Data sets produced in the course of research and shared informally between researchers such as by posting to a website or portal but without preservation capability or commitment. Typically the data remains in the hands of the researchers who have the job of maintaining it.

Group: Research Outputs

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: New Entry

Previous category: New Entry

Imminence of Action

Action is recommended within five years, detailed assessment within three years.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to prevent losses in this group, such as the development of new preservation tools or techniques.

Examples

Departmental webservers; project wikis; GitHub repositories

‘Critically Endangered’ in the Presence of Aggravating Conditions

Originating researcher no longer active or changed research focus; staff on temporary contracts; dependence on single student or staff member; weak or fluid institutional commitment to subject matter; weak institutional commitment to data sharing; complicated or contested intellectual property; encryption; limited or dysfunctional data management planning.

Vulnerable in the Presence of Good Practice

Data in preparation for transfer to specialist repository; robust data management planning; documented and managed professionally;

2019 Review

Research data has been an entry in the BitList since 2017 so the Jury this year has tried to spell out the different complexities that arise in the preservation of research outputs more generally. This entry represents ‘self-help’ data sharing which is to be encouraged as a means to facilitate open science but shouldn’t be confused with long-term preservation.

Additional Jury Comments

Research data is complex and has specific requirements for documentation which may only be known to subject matter experts. However, data creators are not necessarily well placed to sustain data in the long term.

Read More

Published research data appended to journal articles

   Endangered large

Closed research data sets produced and documented in accordance with good practice and simply appended to a journal article or transferred to a repository which does not have sufficient subject-matter expertise or funding commitment to ensure reliable or ongoing preservation for the long term.

Group: Research Outputs

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within five years, detailed assessment within three years.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a small effort to prevent losses in this group, such as the deployment of proven preservation tools or techniques.

Examples

Data sets added to papers in repositories that are designed primarily for papers; electronic journals offering data sets without obvious preservation capacity; institutional repositories servicing highly complex scientific data sets with insufficient subject-matter expertise.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Unstable funding or revenues; poorly designed migration or normalization processes; poorly formed ingest and quality assurance procedures; rapid churn of staff; incoherent patterns of subject matter; lack of domain knowledge; no or very small numbers of users; weak or absent collecting policy; deposit to ensure minimal compliance with funder mandate; limited or dysfunctional data management planning.

‘Vulnerable’ in the Presence of Good Practice

Clear preservation planning; repository development roadmap; ability to transfer collections or share metadata with subject repositories or portals; strong user base; demonstrable re-use of data; clear collecting policy; data management planning early in data lifecycle.

2019 Review

Research data has been an entry in the BitList since 2017 so the Jury this year has tried to spell out the different complexities that arise in the preservation of research outputs more generally. This entry draws attention to services which take upon themselves commitments to preserve research data, but which may not be able to deliver those promises through lack of capability.

Additional Jury Comments

Research data is complex and has specific requirements for documentation which may only be known to subject matter experts. However well intended, it is risky for institutions to attempt to replicate that level of expertise across all the domains within the institution, and it can be hard for smaller publishers to make commitments to sustain data in the long term.

Read More

Scroll to top