Custom Online Databases

   Endangered large

Data collected, presented and disseminated in custom online databases that is not stored elsewhere, particularly data at risk when it is locked in the database because no export or harvest options are available.

Digital Species: Databases, Research Outputs, Web

New Entry

Consensus Decision

Imminence of Action

Action is recommended within three years, detailed assessment within one year

Significance of Loss

The loss of tools, data or services within this group would impact on different people and sectors.

Effort to Preserve | Inevitability

It would require a small effort to preserve materials in this group, requiring the application of proven tools and techniques.


Custom databases created project websites for research, citizen science.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Lack of open licences; lack of export options; lack of system maintenance; expired domain; lack of export functionality; lack of technical knowledge and skills; limited or dysfunctional data management planning; web capture challenges that means unlikely to be picked up by automatic crawlers.

‘Vulnerable’ in the Presence of Good Practice

Backup and documentation; preservation capability in designated repository; use of open formats and open source licencing that enables preservation; enabled export options; robust data management planning; documented and managed professionally.

2023 Review

This is a new Bit List entry nominated and approved by the 2023 Council to draw attention to the particular challenges of preservation for custom online databases. The entry focuses on the distinct risks relating to the online databases which cannot go through traditional web archiving tools.

While there are challenges to preserving databases both off- and online, this entry was nominated in context of projects which set up a custom online database to record, present, and disseminate collected data, but this data is not stored elsewhere (e.g. in a long-term digital archive) and often is locked in the database because no export or harvest options are available. The risks for these online databases include: Maintenance of the system after the end of a project is not ensured, and online databases disappear because of security issues or because the domain expires; Not all data is open and after the end of a project no one is responsible for granting access; The data is not stored elsewhere (e.g. in some trusted repository); The data is locked in: It cannot be exported in (e.g. CSV) for further re-use.

Additionally, the nomination of the entry also highlighted a gap in the Bit List for databases more broadly. The Council agreed a new higher-level Databases digital species group should be created to address this gap, inviting nominations for other database-related entries to be considered for the next major revision of the Bit List.

Additional Comments

The preservation of this entry is highly dependent on the software used but, no matter what, once the project has reached its end, it starts to become vulnerable.

Often, the online databases are of interest to a sub-discipline-specific group of people, e.g. archaeologists specialized on cuneiform tablets. But the material itself often is then invaluable for this group because of the great effort invested in compiling it.

Databases for citizen science also provide an example where the upload of information directly into it makes it distinctive.

Emulation can be used to preserve these databases. For example, Yale University is preserving databases, especially SQL databases for websites, using EAASI. There are technical challenges, but the databases can be preserved, and have found issues are often around access to data and workforce development of technical skills to undertake preservation actions. There is a risk, however, that some of the databases cannot be exposed to the web as they have no survival time and/or cannot make them available as they were intended to be used

See also:

Scroll to top