Politically Sensitive Data

   Critically Endangered small

Digital content where the knowledge to preserve exists and there is no threat to obsolescence, but where political interests may be served by elimination, falsification or concealment.

Group: Political Data

Trend: No Change

Consensus Decision

Added to List: 2017

Last update: 2018

Previously: Critically Endangered

Imminence of Action

Action is recommended within twelve months, detailed assessment is now a priority

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to prevent losses in this group, such as the development of new preservation tools or techniques.

Examples

EG: Online News; Social media and web-based campaigning; Social media relating to 2016 UK/EU referendum; Promises made in Scottish independence referendum 2014; US Environmental Data; UK Public Finance Initiative (PFI) documents; Recordings of Leinster House; Politwoops; DeletedbyMPs; 4chan.org.

‘Practically Extinct’ in the Presence of Aggravating Conditions

Opaque terms and conditions that facilitate deletion or obfuscation; lack of access to web-harvesting; significant lobby interest; change of administration; data resides in single jurisdiction;

reputational risk to collecting institution

‘Endangered’ in the Presence of Good Practice

Robust political archives; robust preservation services for investigative journalists

2019 Review

The nature and extent of political campaigning online continues to become more apparent.  This has drawn attention to the manipulation of digital media but not explicitly the issue of deliberate deletion, alteration or concealment. GDPR provides a pretext for the disposal of records. The increased capability of archives to secure the content from outgoing governments and ministers is a source of encouragement, such as in Canada accusations that the incoming Liberal government had wiped the memory of the outgoing Conservative government were shown to be unfounded.  Nonetheless there is a pressing need for a deep and comprehensive assessment of the risks faced by politically sensitive data and the impact which such deletions have on the public good. That another year should have passed without such an assessment is a matter of grave concern, suggesting that the trend is towards significantly greater risk

Additional Jury Comments

Maybe it is not the duty of archives or libraries to preserve the falsification but to preserve the constituent pieces to allow researchers to infer elimination, falsification or concealment. Web harversting orgs will pick up web and possibly some social media. Will others pick up the rest?  Risk is high because some of the data will be in difficult to reach platforms. Some inevitable loss.

Is there a technological path to capturing and protecting this information before deletion or manipulation? This could well be the basis for crowd funding and/ or crowd implementation.

Read More

Old or non-current video games

   Critically Endangered small

Older video games designed and played on platforms and devices that are no longer supported, especially those with complex but obsolete rights management or system dependencies. This group includes also older editions of games still published in newer editions

Group: Gaming

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: New Entry

Previously: New Entry

Imminence of Action

Action is recommended within twelve months, detailed assessment is now a priority

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

Loss seems likely: by the time tools or techniques have been developed the material will likely have been lost.

Examples

Civilization; SimCity; Sonic the Hedgehog; EA Sports; Doom; Wolfenstein 3D; Lemmings

‘Practically Extinct’ in the Presence of Aggravating Conditions

Complex hardware dependencies or bespoke hardware; dependence on obsolete, low usage operating systems with no emulation pathway; complex intellectual property rights; use of older magnetic media; free distribution on magazines; loss of underlying code or gaming engine; limited or no commercial interest; dependency on remote servers that are closed.

‘Endangered’ in the Presence of Good Practice

Emulation pathway; source code; trusted repository; large user community;

2019 Review

This is a subset of an entry made in 2017 for ‘Gaming’ which the Jury has split into four discrete entries.  This entry for older games encourages greater consideration of the technical complexities which arise from preservation of software and hardware environments as well as the historic value which games have acquired.  There is an active specialist market for older games which enables, preservation but skews it to commercial interests. The meaning of ‘older’ is open to interpretation but certainly any version more than 10 years from release should be included here, even if and perhaps especially if there are more recent releases

Additional Jury Comments

Focussing on offline gaming here - it’s difficult to distinguish these from the related ‘Offline gaming’ entry but there are additional complexities in terms of systems dependencies that need to be considered. It is one thing to be able to get older software running on a different CPU architecture (emulation), but another thing entirely to recreate the peripherals (Wiimotes, plastic guitars, steering wheels, even bongo drums!) that are required to play the games as intended. There’s a sort of kinaesthetic issue: one could perhaps use the accelerometers in modern smartphones to simulate the functionality of a Wiimote, but it will feel different in the hand.

It might be possible to sub-divide between games that have a dependency on communication with servers that no longer exist (loss has already happened) and others where the dependencies are to do with old hardware/software where there is more potential to preserve the interaction.

It’s unclear who is responsible for preserving what and games may be falling between the cracks.

Read More

Media inside paper files

   Critically Endangered small

Media inside paper files occur in records since the 1980s and will continue to do so for many years.

Group: Portable Media

Trend: New Entry

Consensus Decision

Added to List:2019

Last update: 2019

Previously: New Entry

Imminence of Action

Action is recommended within three years, assessment within 12 months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a small effort to prevent losses in this group, such as the deployment of proven preservation tools or techniques.

Examples

Digital media mixed with paper files in records offices and filing cabinets of almost every kind of enterprise. 

‘Practically Extinct’ in the Presence of Aggravating Conditions

Unsustainable effort to assess; exotic or obsolete media; poor storage; lack of descriptive labelling;

‘Endangered’ in the Presence of Good Practice

Carefully labelled; managed programme of assessment and retrieval; robust media used

2019 Review

This is a new entry submitted through the open submission and validated by the Jury who report the significant amounts of digital media being transferred to archives fold into traditional files.  The Jury noted that it is relatively simple to preserve this material once identified using standard tools, but it can be an ‘unknown unknown’ and that assessment can seem overwhelming.

Additional Jury Comments

Highly dependent on who is looking after the portable formats.  There are good example, for example in libraries, where disks are stored at the back of books or front of magazines and can be processed at the point of acquisition. In archives, however dealing with bit-level preservation of external media (often on legacy formats) is largely an unquantified problem and so resource commitments will not be in place. So there is a method and tools but simply no time committed and no proper assessment either. In other agencies the issue will not have even have been considered and for them it will be much harder over time, with some inevitable loss.

Read More

Maritime Archaeological Archives

   Critically Endangered small

These are collection of digital records from maritime archaeological work including photographs, maps and plans, field notebooks, post-excavation finds analysis and other analytical records.

Group:  Museum Data

Trend: New Entry

Consensus Decision

Added to List: New Entry

Last update: New Entry

Previous category: New Entry

Imminence of Action

Action is recommended within 12 months, detailed assessment is now a priority

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to prevent losses in this group, such as the development of new preservation tools or techniques.

Examples

Records of excavations in marine environments which may fall outside the jurisdiction of terrestrial heritage services.

‘Practically Extinct’ in the Presence of Aggravating Conditions

Poor documentation; lack of preservation mandate; dependence on proprietary and non-standard data types

‘Endangered’ in the Presence of Good Practice

Preservation planning from the outset; subject specialist repository; user community

2019 Review

This is a new entry taken from the open submission process in 2019.  It is grouped with Museum data sets as archaeological archives typically make their way to museums, but it is also closely aligned to research data.

Additional Jury Comments

There are trusted custodians of this data such as ADS, DANS or the British Museum as well as in oceanographic research agencies, but perhaps hard to integrate good practice at an international scale. The real challenge therefore is in identifying and sustaining a custodian as other bodies have experience with this data. The proliferation of innovative data recording technologies also implies likely problems of format dependence and documentation.

Read More

Data posted to defunct or little-used social media platforms

   Critically Endangered small

Older or less widely used social media platforms to which content has been uploaded but for which no guarantees have been made about the longterm

Group:  Social Media

Trend: New Entry

Consensus Decision

Added to List: 2017

Last update: 2017

Previous category: Of Concern

Imminence of Action

Action is recommended within three years, assessment in twelve months.

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

Loss seems likely: by the time tools or techniques have been developed the material will likely have been lost.

Examples

BeBo, MySpace, Google Buzz and others

‘Practically Extinct’ in the Presence of Aggravating Conditions

Closure of platform; lack of offline equivalent; lack of export functionality; no preservation undertaking from service provider; unstable business plan from service provider.

‘Endangered’ in the Presence of Good Practice

Offline Replication; clear notice periods and alerts; committed ongoing maintenance of service

2019 Review

The judges have revived this entry from initial submission in 2017 that they were not able to assess at that time. It emphasizes the different threats faced by social media users.  Because these services are older, the need to act is more urgent than for others.

Additional Jury Comments

It is to be hoped that some of these have been archived via traditional web archiving and so the remnants of these sites can be found in bits and pieces in various web archives, but it may be too late to save some of the content that is likely already be lost. If some of this is still available there may be hope in trying to preserve but it may be difficult if the platforms aren't willing to share data or work with preservationists. ArchiveTeam has stepped in here too. There is undoubtedly a story here which could be used as a call for arms to raise awareness about the preservation of current social media platforms too

Need to decouple the content from the platform. Critically endangered if the availability of the content is based on non-public access therefore the best you can ever do is sample some content from people who are willing to provide access.

Read More

Digital Archives from Public Enquiries and Commissions

   Critically Endangered small

Data from public enquiries and reconciliation commissions which can be traumatic, politically uncomfortable and contested, typically comes in many different forms and formats.  It is not always possible to identify an archival authority to look after the resulting evidence and proceedings, and these need carefully managed to ensure their integrity and accessibility into the future. Data protection issues and cultural sensitivities only amplify the challenge to preservations.

Group: Digital Legal Records

Trend: No Change

Consensus Decision

Added to List: 2017

Last update: 2018

Previously: Critically Endangered

Imminence of Action

Action is recommended within twelve months, detailed assessment is now a priority

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to prevent losses in this group, such as the development of new preservation tools or techniques.

Examples

The Tunisian Truth and Dignity Commission to investigate human rights violations committed prior to 2012; enquiries into historical child abuse; Bloody Sunday Enquiry (Saville Inquiry); East Timor Tribunal;

‘Practically Extinct’ in the Presence of Aggravating Conditions

risk of falsification; fragile or obsolete media; dependence on proprietary formats or products; lack or loss of documentation; inaccessible to web harvesting technologies; lack of version control; lack of integrity checks or integrity records; poor chain of custody;

‘Endangered’ in the Presence of Good Practice

Strong sense of archival responsibility; carefully constructed rules around information privacy that retain robust and appropriate preservation capabilities

2019 Review

This entry was received in 2017 but has been split into a series of more discrete entries this year.  This entry represents a small group of very specific but also highly significant recommendations jury which each have their own challenges but which have sufficient similarities to be taken as a group.  The Jury noted that there is in fact considerable evidence of good practice emerging from some of the examples where clear archival responsibility has been the key to progress.

Additional Jury Comments

The submissions from which this entry is made seem like the tip of an iceberg.  Hard to see how to segment it but necessary to raise awareness.

Case files and correspondence are one thing. Retention of these should be clear, but may differ widely between jurisdictions and/or levels of government. If retention is not long term or permanent, risk of loss may not be so critical. Retention of 'unused' or 'potential' evidence is likely a different matter altogether. It may not even been considered a record, and certainly isn't a record of the court. Should it be returned to the suspect or accused? Are their rights being considered here - not just in terms of preservation, but also simply disposition? There are legal and ethical issues around this which need to be fleshed out in conjunction with assessing its preservation risk.

Read More

Digital archives of community groups

   Critically Endangered small

Digital materials including ephemera, correspondence and campaign materials created as a by-product of small scale or ad-hoc community action groups

Group: Community Archives

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within twelve months, detailed assessment is now a priority

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to prevent losses in this group, such as the development of new preservation tools or techniques.

Examples

Archives of smaller and ad-hoc political and campaigning organisations; environmental protests; sports clubs; smaller religious groups; amateur music or drama; fan groups

‘Practically Extinct’ in the Presence of Aggravating Conditions

Poor documentation; lack of replication; lack of continuity funding; lack of residual mechanism; dependence on small number of volunteers, lack of preservation mandate; lack of preservation thinking at the outset; conflation of backup with preservation; conflation of access and preservation; inaccessible to web archiving; dependence on social media providers; distrust of ‘official’ agencies.

‘Endangered’ in the Presence of Good Practice

Residual archive with residual funding able to receive and support collections; active user community; intellectual property managed to enable preservation.

2019 Review

The jury created this entry in 2019 as a subset of ‘Community Archives and Community-Generated Content’ which has been split into two to provide greater specificity in recommendations.

Additional Jury Comments

Typically born digital material is more at risk - community groups simple don't know about the risk of loss. Many are unaware of digital preservation terminology. It is the ad-hoc nature of these groups and projects which is of great concern.

Significant need to raise awareness and provide a "home” but also to do so with sufficient sensitivity so as to ensure community groups remain in control of their own material.

Read More

Digital Archives of music production

   Critically Endangered small

Digital materials created by musicians and fans as a by-product of performance or recording, not otherwise published or shared

Group: Sound and Vision

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: New Entry

Previous category: New Entry

Imminence of Action

Action is recommended within three years, assessment within 12 months

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to prevent losses in this group, such as the development of new preservation tools or techniques.

Examples

Pre-production notes; demo recordings; photography; correspondence.

‘Practically Extinct’ in the Presence of Aggravating Conditions

fragile or obsolete media for offline content; service provider preservation capability for online content; dependence on proprietary formats or products; lack or loss of documentation; uncertainty over intellectual property rights; lack of version control; lack of policy or mandate

‘Endangered’ in the Presence of Good Practice

Replication; clarity of intellectual property rights; preservation agency involved and capable of looking after content

2019 Review

This entry has been split out of the previous entry for ‘Digital Music Production and Sharing’ though it has overlaps with other entries including ‘Pre-production TV and Movie materials’.  A separate entry has been included to emphasise the inherent and value of the archival materials relating to the recording process over and above the recordings themselves.

Additional Jury Comments

Inevitable loss of existing data but it would require major effort to fix in terms of identifying organizations who are preserving this content. Not clear that this is being done already

Read More

Open source intelligence sources of current conflicts

   Critically Endangered small

Open source social media and web content which supports crowd-sourced investigation and fact-checking to verify or refute claims of state agencies and rebel groups in the context of current political or military conflict.

Group: Digital Legal Records

Trend: New Entry

Unanimous Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within twelve months, detailed assessment is now a priority

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

Loss seems likely: by the time tools or techniques have been developed the material will likely have been lost.

Examples

Social media sources relating to current conflicts, such as in Yemen or Syria.

‘Endangered’ in the Presence of Good Practice

Offline backup captured by journalist or investigating authority;

2019 Review

This is a new entry received through open submission in 2019 and subsequently split into three elements by the Jury, relating to current, recent and historic sources.  This entry relates in particular to materials relating to current and ongoing conflicts.  Social media companies have a policy to take down or suppress content that they consider to be propaganda for terrorist groups.  This has had the unintended consequence of deleting or supressing content that was being used in open source investigation or fact checking for journalistic or judicial purposes, and which may therefore be an impediment to refutation or prosecution.  However a new generation of cloud based services, such as Hunchly have emerged in the last few years which allow investigators to copy and stabilise content to private accounts in the process of investigating it: so the ethical requirements of social media companies and the integrity of investigation are both served. The Jury notes that the such content remains at risk, and the process of investigation is slower than algorithmic deletion.  Nonetheless there is a notable difference in the investigation of current conflicts than historic ones where evidence has been lost.

Additional Jury Comments

--

Read More

Web domains with no legal deposit

   Critically Endangered small

Web archiving is a recognized specialism within digital preservation, able to capture large quantities of material with routine and standards-based tools.  But there are significant issues of intellectual property rights associated with website capture and republication.  In many jurisdictions, but by no means all, those obstacles are overcome by regulations that enable a national library or other ‘legal deposit’ agency to copy and preserve content.  Where no such permission exists, there is a significant risk of loss.

Group: Web

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within 12 months, detailed assessment is now a priority

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

Loss seems likely: by the time tools or techniques have been developed the material will likely have been lost.

Examples

Domains registered without a country code; domains with a country code but weak or unenforceable legal deposit permission to harvest.

‘Practically Extinct’ in the Presence of Aggravating Conditions

Rapid churn of websites; lack of access to Internet Archive harvest; contentious content; encryption; digital rights management; non-standard content management

‘Endangered’ in the Presence of Good Practice

Permissive approach to Legal deposit;

2019 Review

This entry came about by Jury deliberation on entries to the open nomination process.  It is characterised by regulatory barriers rather than technical ones, though the pace of change in web technologies as well as the growth of web content mean that significant technical challenges still exist.  The Jury also notes that local conditions are also a significant factor.  For example, web sites often also fall under public records legislation or are important elements of corporate records: and so important parts of the web are harvested even when there is no explicit legal deposit legislation. Moreover the Jury particularly recognizes the work of the Internet Archive to capture and preserve content.  Even so there are significant gaps in web archiving and in too many cases it is regulation that is the barrier.

Additional Jury Comments

--

Read More

Scroll to top