The British Library is seeking support for a research study on UK Web Archive user needs. Please find the details below:
Introduction
The UK Web Archive (UKWA) is a collaborative initiative by the UK Legal Deposit Libraries—the British Library, National Library of Scotland, National Library of Wales, Bodleian Libraries, Oxford, Cambridge University Library and the Library of Trinity College Dublin – to collect, preserve, and provide access to UK websites. It ensures that digital content from UK websites is available for long-term historical, cultural, and scholarly research, especially as websites change, disappear, or are no longer maintained. The UK Web Archive is a vital repository of the UK's digital cultural heritage, preserving websites and online content for future generations.
The Challenge
Access to archived websites is challenging for several reasons:
1. Legal Restrictions
Legal Deposit Legislation allows UKWA to collect all UK web content, but access is, for the most part, limited to on-site terminals at Legal Deposit Libraries.
This means most archived content cannot be accessed freely online, even for research purposes.
2. Technical Complexity
Websites today are built with dynamic content, JavaScript, multimedia, and APIs, which are difficult to archive accurately.
Preserving how websites function (not just how they look) requires complex tools and infrastructure.
3. Search and Discovery
Archived websites comprise massive datasets of WARC files, which are hard to navigate or search without specialised interfaces or tools.
Researchers often struggle to locate specific content or understand its context due to the lack of advanced metadata or challenges with full-text search options.
4. Infrastructure and Storage
The sheer volume of data collected in web archives requires substantial storage, processing power, and long-term digital preservation strategies.
5. Research Usability
Scholars need tools for data mining, network analysis, or longitudinal studies, but these tools aren’t always readily integrated into web archive access systems.
Interdisciplinary researchers may find the data formats and structures unfamiliar or inaccessible.
We are seeking support for a research study, which will focus in particular on challenges 3 and 5 in the above list.
Request for quotes for a research study
Looking ahead, the UK Web Archive must deepen its understanding of user needs to improve and expand its services in line with modern expectations. This work is essential not only to grow our user base but also to strengthen research engagement with the archived web. To achieve this, we must develop more sophisticated and diverse access methods that reflect how users interact with digital content today. This goal aligns with the Library’s strategic commitment to put users at the heart of everything we do.
While the web archiving team has the expertise to lead this work, current capacity is fully allocated to critical priorities, including the renewal and rebuilding of the infrastructure required to bring the UK Web Archive back into full operation. To advance this user-focused workstream, we are seeking to procure research into user needs.
The researcher will be expected to investigate previous studies of user needs and behaviours, identify challenges and opportunities in accessing web archive content, and produce actionable insights. For example, we know that full-text search is highly valued but implementing it across the entire collection poses significant technical challenges. Existing research into user needs within the web archiving field can provide a useful foundation, and we expect the researcher to review and synthesise this work.
Examples of existing work include:
Big UK Domain Data for the Arts and Humanities (BUDDAH). (2014-2016). Big UK Domain Data for the Arts and Humanities. https://buddah.projects.history.ac.uk/
Healy, S., Byrne, H., Schmid, K., Bingham, N., Holownia, O., Kurzmeier, M., & Jansma, R. (2022). Skills, tools, and knowledge ecologies in web archive research (WARCnet Special Report). Aarhus, Denmark: WARCnet. https://cc.au.dk/fileadmin/dac/Projekter/WARCnet/Healy_et_al_Skills_Tools_and_Knowledge_Ecologies.pdf, https://doi.org/10.23636/9sex-kv63
Ogden, J. (2019, June 26). Research challenges for using the UK Web Archive for social science research. ESRC National Centre for Research Methods.
Ogden, J., & Maemura, E. (2020, October 7). A tale of two web archives: Challenges of engaging web archival infrastructures for research. Zenodo. https://zenodo.org/records/4058013
Ogden, J., & Maemura, E. (2021, April 27). ‘Go Fish’: Conceptualising the challenges of engaging national web archives for digital research. International Journal of Digital Humanities, 2(1–3), 21. https://doi.org/10.1007/s42803-021-00032-5
Reynolds, E. (2013, March 7). A review of contemporary research use cases for web archives. International Internet Preservation Consortium. https://web.archive.org/web/20170317151439/http://netpreserve.org/sites/default/files/resources/UseCases_Final_1.pdf
Storrar, T., & Seaman, G. (2016, April 14). Researching user needs: The UK Government Web Archive. https://digital.library.unt.edu/ark:/67531/metadc1477155/m2/1/high_res_d/WAC02-HEKLA-Tom_Storrar.pdf
Webster, P. (2020). How researchers use the archived Web (DPC Technology Watch Publication). Digital Preservation Coalition. https://www.dpconline.org/docs/dpc-technology-watch-publications/technology-watch-guidance-notes/2263-twgn-20-01-how-researchers-use-the-archived-web-webster/file, https://doi.org/10.7207/twgn20-01
Additionally, the research should refer to recent initiatives and projects led by national web archiving institutions, such as:
UK Legal Deposit Libraries. (2023, October). Archive of Tomorrow: Capturing public health discourse in the UK Web Archive (Wellcome Trust, Sponsor). British Library, UK Web Archive. https://doi.org/10.23636/6q6k-8369
Deliverables
The appointed researcher will produce:
- A detailed written report outlining findings, insights, and recommendations.
- A presentation of the findings for staff at the British Library and Legal Deposit Libraries.
- It is not in the researcher’s scope to build or implement an access service, or to provide technical details of how a service should be implemented. The final report should be requirements-focused.
The outcomes of this work will directly inform future service design for the UK Web Archive. In particular, user journeys will shape technical priorities—for instance, how we allocate development effort between search infrastructure (e.g. Solr), data processing, and access mechanisms.
It is the intention of the Library to publish the report in its Research Repository using a CC-BY license.
Timeline:
- Invitation for quotes published: Tuesday 27th May 2025
- Deadline for responses: Tuesday 8th July 2025
- Decision to appoint researcher: Friday 25th July 2025
- Research study start date: August-September 2025
- Delivery of final report and presentation: November-December 2025
Proposal requirements:
Researchers should respond with:
- A clear cost estimate
- A proposed schedule with the expected date of completion
- A brief outline of the approach to the project
- Examples of previous relevant work
Proposals will be evaluated against the following criteria:
1. Ability to meet the requirements in the brief
2. Proposed delivery schedule or timetable
3. Total cost/value for money
Enquiries and/or quotes should be submitted to:
Nicola Bingham
Lead Curator, Web Archiving, British Library