IT Vendors Survey

Print

Public Draft - Extracts from

A Survey of Information Technology Vendors

Prepared for:

The Digital Preservation Coalition

by:

Philip Lord

The Digital Archiving Consultancy

28th October 2002

©Digital Preservation Coalition 2002. All rights reserved.

Management Summary

Background

This study was conducted between late July to early October 2002 in furtherance of the Digital Preservation Coalition's (DPC) aim of working with industry, and followed a forum held in June 2002 with a cross section of industry on future research and development for digital asset preservation.

Objectives

The survey's objectives were:

Methods

A cross-section of 35  large, medium and small companies was approached with an e-mailed questionnaire sent out under Lynne Brindley's name, and which contained an invitation to proceed to a telephone interview. Twenty one (68.6%) companies completed the e-mail questionnaire, and seventeen (48.6%) were subsequently interviewed by telephone.

The survey was limited to suppliers of information technology products, specialist consultancies and professional and trade bodies.  Consumers of IT products and services were not surveyed.

Findings

The following summarises the major findings:

Looking forward

Introduction

The Digital Preservation Coalition (DPC) was formed in July 2001 to raise awareness of the issues raised by the need to keep and to re-use for a decade or more digital assets and resources which institutions have created or purchased. The DPC co-ordinates concerted action to address these issues in the UK and internationally. It has over 20 major organisations as members ranging from the national libraries and archives of the UK, education and scientific research, other public bodies such as the National Electronic Library for Health, and industry trade bodies such as the Publishers Association. Further information on the DPC is available from its website (www.dpconline.org).

One of the aims of the DPC has been to work closely with industry to address technical challenges and support the development of market solutions. In June 2002 it held a one-day forum with a cross-section of industry and DPC members entitled "Future R&D for Digital Asset Preservation". A meeting report and presentations from the day are available from the events page on the DPC website (www.dpconline.org/graphics/events/rdfuture.html ). At the forum the DPC undertook to follow up the discussion on future research and development with a cross-section of industry. It was also agreed to explore the level of industry interest in further engagement with the DPC and its work. As a result this survey was initiated, and was contracted to Philip Lord to conduct on its behalf.

Objectives

The objectives of this survey were:

In order to limit the scope of the project and stay within the set budget it was decided only to approach providers of IT goods and services, not consumers of these.

Methods

The following process was followed when conducting the survey:

The survey took place between 24th July and 4th October 2002.

The Sample and Response Rates

Table 1 shows the breakdown of the types of companies approached.  Half the companies were software providers, as software is where many of the difficult issues of preservation issues lie rather than with hardware.

It was not always easy to ascribe a single characterisation of a company since they often have multiple product lines and activities, so Table 1 shows the best approximation to their main line of business so far as this survey was concerned. Four out of the five companies described as hardware suppliers were mainly in the business of supplying optical or microfilm storage and management systems.  The specialist consultancies were all active in the preservation/records management area.

Table 1  Types of companies approached

Table of types of companies approached.
Type Number of companies
Consultancy - specialist (in this area) 3
Software - bespoke 4
Software - packaged 1
Software - specialist 7
Software - records management, document and image management 6
Hardware systems suppliers 5
General systems and service providers 3
Professional or trade body 5
Service companies 1

Table 2 shows a breakdown of responses to the various stages of the survey. The proportion of total non-respondents was 31.4%, after telephoning and sending e-mail reminders to those who did not respond to the initial approach.  The response rate was a little better than expected, and higher than in many surveys of this type.  Nearly 50% of those approached completed the whole process, and information was gathered from 60% of the total.

It should be noted that for two of the larger companies more than one person was approached. In both cases they elected to respond via a single individual and all figures reported here are in terms of companies, not individuals.

Table 2  Breakdown of responses

Table of breakdown of responses.

Response

Number of companies % of Total Notes
Sent first form by e-mail 35 100.0  
Replied to e-mail 24 68.6

Three of these answered but declined to fill in the questionnaire

Completed e-mail questionnaire 21 60.0  

Positive response to telephone interview

19 54.3

Two of these proved impossible to arrange interviews with before the study closed.

Interviewed

17 48.6  

Table 3 demonstrates how the different industry sectors responded to the call to participate, showing numbers of companies from which no information was forthcoming and those completing at least one questionnaire. 

Table 3  Responses by Industry Sector

Table of responses by industry sector.
Type No
information
obtained
Completed at
least one
questionnaire
Consultancy - specialist (in this area) 1 2
Software - bespoke 1 3

Software - packaged

0 1
Software - specialist 6 1
Software - records management, document management 2 4
Hardware systems suppliers 1 4
General systems and service providers 1 2
Professional or trade body 4 1
Service companies 0 1

Though the numbers are small it shows less interest from the specialist software vendors (which included a wide range, from accounting software, database management systems to e-learning software).  The trade and professional bodies were also hard to interest (with a notable exception).  This is disappointing: it is the specialist software suppliers who one could consider as being at the centre of the problem and key to its solution, and the trade organisations could be vital in helping to turn round perceptions and initiate action in their sectors.

Informal estimates of presumed prior knowledge of the digital longevity problem seemed to be associated with a high interview rate; there was a hint that those with lower awareness did not respond as strongly to the invitation.

Most the people approached were based in the UK, but four of the respondents were based overseas. All of these answered both questionnaires.

The Findings

Results of e-mail questionnaire

The results of the Yes/No questions in the e-mail questionnaire are shown in Table 4 .

Table 4   Results of initial e-mail questionnaire (Y/N questions)

Table of results of initial e-mail questionnaire (Y/N questions).
Question number Question Yes No
1 Do you believe there is sufficient awareness in your industry of the technical issues which accompany long-term (10 years+) retention and use of digital materials? 4 17
3 Are you currently developing or supplying products which are relevant to long-term (10 years+) retention and use of digital materials (examples: long-term retention of electronic records, products and publications; access to files created in previous versions of software) 20 1
4 Have you previously heard of The Digital Preservation Coalition (DPC)? 10 11
5 Would you like to know more about its work? 18 3
6 Would you be willing to participate in a second stage of the survey, which will involve a short telephone interview (20 min)? 19 2

The consensus coming from this questionnaire was that there was insufficient awareness in the industries of the respondents of the technical issues of long-term retention (but it may be that the respondents were actually answering in a broader context - industry as a whole - see below.)  That being noted, nearly all the companies responding claimed to be supplying products which were relevant to long-term retention - presumably they were not counting themselves among the ill-informed they referred to when answering question 1.

Just over 50% of respondents had not previously heard of the DPC. Almost all the respondents that had previously heard of the DPC had attended the DPC's industry forum in June 2002.  All respondents except three would like to know more about its work - including, with a single exception, all the companies with previous knowledge of the Coalition.

As noted earlier, 19 of the 21 e-mail questionnaire respondents elected to go on to the telephone interview, showing goodwill towards the process - or at least willingness to proceed further.

Question 2 of this initial survey asked "Do you have any comments you would like to make on these issues?" - i.e. those referred to in question 1.  As might be expected given the cross section of companies, responses were rich and varied.  Some items mentioned more than once were:

Some useful quotes from a DPC perspective:

From a trade body:

"We are very interested in working with you to improve the amount and the quality of the dialogue between users and suppliers, to develop good practice and to promote successful implementation."

And

"very interested in collaborating with you on events, projects etc"

From a specialist consultancy:

"This is an issue for all users, and RM or IM staff must shift from Registry mode into Education and Support mode."

From a hardware systems supplier:

"We have been championing this cause for the last 5 years, convincing people about the inherent fragility of keeping information only in digital format without backing-up onto more permanent media. We would be delighted to discuss this further with you."

From a records management software company:

"Close tracking of initiatives such as DPC is important to establish relevance and impact on the provision of current and future services."

Results from Telephone Interviews

Nineteen respondents to the e-mail questionnaire agreed to participate in a 20-minute second stage interview by telephone.  For two of these it proved impossible to find an interview date before the study closed.  Results for the remaining seventeen are reported here.  All interviews were one-to-one excepting one  who organised an audio conference with four of their personnel.

The results are discussed question by question below.

Question 1:

Interviewees were invited to propose ways of encouraging awareness of digital preservation in their industry, and to comment as appropriate.

Many suggestions were forthcoming, in order of popularity:

Answering this question most interviewees interpreted the scope of the question beyond just their own industry segment, which the formal wording of the question indicated.

Some responses are worth noting for their innovative nature.  The leader of a knowledgeable specialist consultancy in this area made a plea to engage young people, with specific ideas for projects.

Regarding scare stories, it was noted that commercial companies who have suffered from data loss over time may be reluctant to advertise the fact, being afraid to loose face and credibility.  A head in the sand attitude was mentioned more than once.

This question provided an opportunity for interviewees to express further views on the problem as a whole and thereby elicited supplementary information to question 2 in the e-mail survey.  Further themes which emerged were:

Again it is worth singling out a few quotations (as recorded by the interviewer):

From a specialist consultancy:

Urgently: get into schools and colleges.  Start projects which relate to pupil's personal records/communications.  (E.g. e-mails, photos, diaries,   ). One idea is to get school children to record their peers', parents' and grandparents' anecdotes digitally and ask them how it is to be preserved."

From a records management systems software vendor:

They have had a few tenders for archiving systems (from government bodies) - they basically say, "over to you, we don't understand the problem".  On the other hand suppliers are scared of the open-ended nature of the requests made to them.

From a hardware supplier:

(He) feels there is a responsibility in [supplier] companies to spread the word  - but it is a difficult idea to sell, and the burden of proof is often thrown back to his company

From a hardware supplier:

If you want industry to take an interest in this issue then you have to demonstrate there is a profit in it.  Can't see a sales opportunity here.

A few comments related specifically to the DPC:

From a supplier of bespoke software:

Noted that the things the DPC are doing are good, and its publicity campaign is pretty effective.

From a general systems and services supplier:

. . . but asked if the focus of the DPC was just on UK academia, and possibly the public sector, or a wider audience.  Who is its audience, and what is its constituency?

Question 2.

This asked whether more should be done using public money to help industry address the issues.

Thirteen responded positively to this and four were against it.

The question provoked a lot of comment, some of it quite trenchant.  Notable comments included:

Some of the bodies mentioned that might have a role were the BSI, The Treasury, DTI, PRO, MoReq (sic).

A few felt that government mandates on data standards were necessary.

Again this question encouraged further suggestions supplementing e-mail question 2 and the previous interview question 1.  Thus:

Again some quotations are useful:

From an hardware supplier:

"Not much government can do by shaking the trees"

From a bespoke software supplier:

The UK should do like the Dutch government is doing and have a program. The Dutch are doing more and doing it better.

From a specialist consultancy:

Government needs to take a "light, enabling role"

From an hardware supplier:

 "The DPC can make this a national issue"

From a bespoke software supplier:

It is difficult to get the attention of industry.  "A project of this size - meaning the DPC and survey - will engage attention"

Question 3.

"Proposals are currently being considered to develop a Digital Curation Centre based in the UK higher education sector to support R&D into digital preservation.  What areas of publicly funded research and activities in digital preservation do you think would most usefully support industry efforts and competitiveness in this area?"

On the whole this was said or implied to be a good thing, with only four companies dissenting.

One respondent (in favour of the proposal) clearly articulated a doubt that academic research would prove to be valuable in a commercial context, and pointed out that different archive solutions are needed in different contexts.   Others noted that management processes are just as important as technologies, but that the USA does that sort of research better than the UK; this same respondent suggested that work be undertaken in the context of the Grid.  In a similar management-orientated vein a respondent suggested economic models as one area of research.  Another suggested promoting work on the selection of materials to preserve and the purpose of retention.

A useful suggestion was made that such an initiative could be an opportunity for joint funding between industry and the public sector, and would provide a bridge between academia and the commercial world.

Existing centres such as the University of London Computer Centre and the Data Archive at the University of Essex were mentioned by a few as possible starting locations.

There was a strong feeling among some that commercial drivers will dictate what happens regardless of what theoretical work takes place. A few quotations from both sides of this debate:

From an hardware supplier:

It would be good to see a UK effort - for example based around the Grid.  Possibly we should have at most two focussed efforts in two first division universities.

From a specialist consultancy:

He would divide it up into two:

1. A university based centre for digital curation, and

2. Centres where it is practiced -such as the Data Archive at Essex, and PRO, ULCC. 

From a records management software systems vendor:

Publicly funded research is fine but technology will not wait and commercial drivers are going to dictate what happens.  We do not have the luxury of time for it to happen.

Question 4.

This question asked respondents what they felt they could contribute to a register of information on file formats etc.

There was a big "yes" to the concept (13 for, 3 against and 1 don't know).  Comments revealed the following suggestions on their companies contributing.

The dissenters pointed out:

Perhaps significantly, no respondents offered to submit their own file formats to such a register; most offers were on peripheral services; or they claimed, probably with justification, that they were not in control of this information. Companies who are producing software were reluctant to contribute, as commercial ownership and competitive advantage are obstacles.  However one company suggested that contributing to a register with a five-year "moving wall" would be acceptable, where formats over five years old were contributed; five years is a long time in the software business and such formats would be effectively out of date.

Intellectual property issues were raised as a concern in environments where software and content were not easily separable, such as in e-learning.  It was implied that it may not be just a vendor decision, but may involve vendors, customers and content providers.

There is a feeling on the interviewer's part that companies were really saying "Yes - this is a fine idea in principle, but in practice it would be hard for companies to cooperate".

A quotation is worth giving:

From a hardware vendor:

The real formats are the property of the monopoly supplier.  They have strong commercial reasons to keep these formats to themselves, and it would be a doomed effort.  Government mandating of open standards would help but would be followed by massive lobbying.

Question 5.

Does your company factor long-term retention in to its products?

The following summarises the results:

Table of summary of results for Question 5.
Answer Number
Yes 12
No 2
Maybe/depends 2
Not applicable 1

Broadly companies are saying they do take longevity into account, but there was a wide variety of comment and views, and a range of interpretations of the question depending on the business they were in. Companies were looking at it from their own perspective (naturally enough) - thus hardware vendors claiming long media life or software vendors storing their own code.

Two respondents quite explicitly said they didn't.  One, a hardware vendor, citing difficulties with spare parts after, say, five years; the other, a specialist software company, saying the company was too young and had other priorities at the moment.  It would have been useful to have the candid views of more specialist software companies to this question.

A few respondents felt they would need to refer to others more aware of the issue in their organisations.

Overall the positive replies show some attempts at tackling the question, but these only address fragments of the overall problem. The question of integrating all the pieces into a whole solution was left unanswered.

Question 6:

Are customers increasingly asking for long-term retention features?

This question elicited the following responses:

Table of summary of results for Question 6.
Answer Number
Yes 9
No 7
Don't know 1

In most cases it was clear that the answers really referred to the absolute value, not to changes in customer behaviour.  Looking at the question in that light there is a pretty even split between those who see longevity being requested and those who do not.

Looking at the results in more detail it appears that demand is driven in specific sectors or by specific drivers, and the answers reflect familiarity with these sectors: Some of the specific situations cited were:

Even those companies saying "no" made caveats.  The need may be expressed in requirements documents they receive, but it is not stressed at all. 

An e-Learning company noted that the need is not expressed now, but there is a feeling it will emerge soon.

An extreme view was that customers have always been asking for long data life - but it was not stated if the view reflected implied assumptions by purchasers, or explicit wishes; I suspect the former.

Some quotations are interesting and illustrate some of the dichotomies expressed:

From a general systems provider:

Except in specialist areas the demand does not exist (or need is not recognised), though interfaces with legacy systems is often asked for.

From a hardware vendor:

They are doing this without prompting.  - He gave example of a customer, a well known financial institution which is trying to retrieve data off an old optical disk system. The data needs to be kept for 40 years, and they do not want to repeat these rescue operations every 10 years or so.

From a packaged software supplier:

No, there is no increase, they have ALWAYS done this.

From a hardware vendor:

No - generally the opposite! - the company suggest it [longevity] to them.  It may be requirement buried in a specification, but is not emphasised.

Question 7.

Participating in DPC work.

By far most respondents showed a willingness to participate or cooperate with the DPC. The only person replying with a direct "no" was based in the USA and was working in a large corporation.  Another US-based respondent expressed doubt.  A few thought it was a good idea but said they would need to refer the question back to others in their companies.  There is an action point for the DPC to follow up these leads.

These answers show a high measure of goodwill towards the Coalition.

A number of companies clearly indicated that they would require something from any collaboration, notably introductions to (potential) customers. Two companies mentioned they would need to assess each DPC event on its merits on a case-by-case basis. 

A theme from many was payment in kind rather than a cash subscription, such as offering printing and scanning services, supply of editorial materials, case studies, intellectual input etc.  This is obviously a financially tight time for the IT sector -one interviewee characterised his company as now "being a not for profligacy company".

Question 8

Asked for further contacts.

Only a few suggestions were made. Where they fitted the criterion of being a vendor or relevant trade body, they were followed up and asked to participate in the survey.  The DPC may want to take an action point to follow up on the others.

Question 9.

Any other comments?

A few asked whether they would see the final report.  One respondent made a strong plea for anonymity.

Some mentioned the need to clarify the DPC's role.

Lastly a number asked to be kept in touch with, asked that their sector not be lost sight of, and to hear about the outcome of the study.

A few quotations of value to the DPC:

From a supplier of bespoke software:

The DPC is doing useful work and has made a good start.  Would like to know more regarding its long-term aims, and suggest it moves away from a "for the librarians" image.

From a hardware vendor:

He is very pleased the subject is being taken seriously now.


[1] See:   Museum Association,  Ethical Guidelines 3: Trading and Commercial Activities.  Accessible from http://www.infosite.co.uk/masite/ethics.htm NB: 24/08/2004 this web page no longer available - please see Museums Association statement