Web Crawl

To read more about Web Crawling, see DPC Technology Watch Report:

Parent Tags

Create or Receive (Acquire)

Articles

Adaptability in the face of adversity - Archiving the Web to Help Persons Forced to Flee

Tom Wilson is Associate Archivist at United Nations High Commission for Refugees. Introduction The United Nations High Commissioner for Refugees (UNHCR) has a mandate to support and advocate on behalf of one of the most vulnerable groups of people in the world today. As one of the major issues facing the world today, the plight of refugees, internally displaced people, stateless people and other persons forced to flee is an ever-growing issue. Whilst the title of this post primarily...

Read More


A Hybrid Model for Web Archive Capture

Claire Newing is Web Archivist at The National Archives in the UK 2021 is the year the UK Government Web Archive came of age. On 4 November we celebrated the 25th birthday of our oldest resource - this archived version of the Environment Agency website. Of course it shared its birthday with World Digital Preservation Day 2021 so a double excuse to celebrate.

Read More


Celebrating 20 years: DPC Prospectus 2021-2022 now available

The Digital Preservation Coalition (DPC) is delighted to share its new program of activities for its 20th anniversary year . Focused on the priority digital preservation topics Members have specified through the annual Connecting the Bits consultation process, and through the Australasia and Asia-Pacific Stakeholder Group, the 2021-2022 prospectus includes publications, training, webinars, specialist briefings, as well as a range of exciting new activities and resources.

Read More


The long WARC to freedom

Tom Wilson is Associate Archivist (Digital Preservation) for United Nations High Commissioner for Refugees in Switzerland. Our recent transfer of web-crawl suppliers taught us that the best laid plans can be derailed by factors beyond one’s control. UNHCR has been capturing content for its web-archive since 2015, working with Internet Memory Research (IMR) as our supplier to capture, store and display this content. In 2018, IMR informed us that they would be going bankrupt....

Read More


Web Archiving & Preservation Working Group

Web Archiving & Preservation Working Group What is the Web Archiving & Preservation Working Group? The Web Archiving and Preservation Working Group (previously the Web Archiving & Preservation Task Force) provides a forum for participants to share their experiences, establish common goals, and inform their own policy development. It provides a mutually supportive environment for continued programme and policy development for DPC members and a mechanism through which non-members...

Read More


Scroll to top