File Formats

File formats define how information is encoded a digital file. File formats can be standardised, open, well documented and possibly associated with a reference implementation for how software should interact with files of that format. But file formats are not always as clearly defined, and format specifications are not always closely followed by the software that implements them. Understanding file formats and how we interact with them in practice can be therefore be critical to ensuring effective digital preservation. This page provides some guidance on the best sources of information for further information on file formats. For a broad introduction to file formats and digital preservation, see the DPC Handbook:

• File formats and Standards, DPC Handbook

See also, the DPC Technology Watch Reports:

• File formats for Preservation, by Malcolm Todd 2009
• Preserving Computer-Aided Design (CAD) by Alex Ball 2013
• JPEG 2000 - a Practical Digital Preservation Standard? by Robert Buckley, Ph.D. 2008
• Preserving the Data Explosion: Using PDF by Betsy A. Fanning 2008

Understanding the broader challenges associated with file formats

A number of pieces of work have sought to develop methods of assessing the appropriateness of particular file formats for preservation, typically based on high level criteria. This includes the now somewhat dated DPC Tech Watch report. More recent thinking has begun to move away from this approach, due to the need to base decisions on practical experiences with working with file formats and software:

• Assessing file format risks: searching for Bigfoot?
• Sustainability Assessments at the British Library: Formats, Frameworks and Findings

Precision and completeness are not qualties that can always be associated with file format specifications, and this lies problem lies at the root of many preservation challenges:

• The Network is the Format: PDF and the Long-term Use of Digital Content
• "More What You’d Call ‘Guidelines’ Than Actual Rules”: Variation in the Use of Standards

Examples from the Information Security community, while not typical of the preservation challenges we are likely to experience, illustrate the flexibility in many file format specifications:

• “Abusing file formats” (chapter 6)

File format identification

Applying a specialist software tool to identify the formats of files to be preserved is typically one of the first steps in a digital preservation work flow. Read more about File format identification here...

Seeking reference information and guidance on specific formats

There are a number of excellent sources of information to assist digital preservationists. Wikipedia remains a good place to start for high level information about a particular file format. The associated Wikidata is the also the focus of the latest effort to build a collaborative registry of file format information.

• PDF, an example of file format information on Wikipedia

A small number of libraries and archives have been developing their own preservation focused assessments of particular file formats. These provide useful guidance on the risks associated with common file formats, and approaches for addressing them. They are located in different places on the web, but are linked from the home of a loose collaboration between these organisations on the DPC Wiki:

• File format assessments

The Just Solve wiki provides a community driven site for gathering information about different file formats and is particularly good for discovering information on more obscure file formats:

• Just Solve