In today’s increasingly digital world the need and desire to have online access to collections is growing especially within the cultural heritage sector. Researchers and historians are finding new and innovative ways to share their collections with a wider (global) audience through the digitisation of their physical collections. With a wide range of equipment available many organisations are evaluating whether this activity should be undertaken in-house, rather than outsourcing to an experienced cultural heritage digital imaging provider. However, there are hidden costs to set up and maintain a digitisation facility, which is often not factored into the decision. Whichever choice is made it is important to consider international developments in best-practice digitisation methods and equipment. Additionally, it is important to deliver a programme that is consistent with your organisation’s digital strategy that delivers the best value to your stakeholders.
Let’s start with Why digitise?
Whatever the reason, by digitising your collections you can breathe new life into your history, capture additional historical information – perhaps from sources you haven’t previously engaged with, and bring significant information to the surface that might otherwise be lost.
Organisations with large volumes of material to be digitised often face a choice, either create in-house capability or outsource the work to a specialist. An in-house digitisation approach is appealing if the material is highly valuable or cannot be easily moved, or where there are other reasons for it not to leave your site or sight. This is particularly the case in the cultural heritage sector, where collections may have been donated and entrusted to the organisation to look after them, keeping them preserved and safe. However, if the collection is large or complex and you have limited in-house resources, outsourcing to a specialist team of digital experts to complete the digitisation may be the right option. One aspect that is difficult to quantify is the staff hours for training and upskilling, IT and methodology support, and the cost of equipment, ongoing servicing and upgrading. So, at the outset it is wise to understand your current status –
Do you have a digitisation strategy, and is it current? Have you undertaken digitisation projects or programmes of work recently? Do you have an online presence and is it meeting your business or organisational needs; your stakeholder needs? How do you know?
The point is, you need a digitisation strategy that is complementary to your organisational, business strategy and plan. The following steps should be considered before you begin to create or refine your digitisation strategy.
Define the reason for digitising – the Why continued
Why is it important that your material / your collection is digitised? Is it significant? Is it in demand? Is it at risk?
Your purpose and goals for digitisation will help you develop a plan and allow you to prioritise elements or subsets (based on time, condition, shape, demand) of your collection for creating a digitisation programme. For example, it might be more important to digitise some aspects of your collection for a pending exhibition, anniversary or other time-bound needs such as degradation of the original material through decay and poor environmental conditions. Likewise digitising the ‘jewels in your crown’ to get some ‘runs on the board’ to attract stakeholder support or build a community may also be a good tactical approach.
There are some quick-fire prompts in our Digitisation Planning poster here.
Assess the nature and condition of your collection – the What
What kind of material is included in your collection? Is it historical, fragile, rare, valuable?
Material that is partially damaged or degrading requires specialised skills for assessing, appraising and determining if capturing the content is feasible, in some cases understanding whether any of it is salvageable.
What are the shapes and dimensions of your collection items?
Conducting an audit and count of your material is an important step when assessing your collection and often we find that significant collections are rarely indexed and catalogued correctly and even a list of holdings often lacks critical information. Conducting a physical audit and count of your collection will update any outdated information and will provide an accurate picture of your overall collection as you know it now. This information will also provide Quality Assurance metrics for use throughout the project, and likely be supplemented with new findings as the digitisation programme progresses.
What is the format variety of your material?
Such as, bound books and documents, loose material such as business records, larger format plans, maps and artworks, 3D material, ephemera, photographic prints/ negatives/ transparencies, audio or video. The review of the material should identify sufficient detail for quality planning – for example;Are there foldouts or ‘stick-ons’ or inserts inside the collection items?Do books have photographic plates in them, are there glossaries or indexes of note? Are pages ‘dog-eared’ or in need of repair to ensure you do not damage them further while handling during preparation or scanning phases?
Has your material been digitised elsewhere?
Your material will likely require multiple and different types of digitisation equipment which in turn require different skills to operate. Once you have documented your collection items you can reach out to online catalogues and consult via your networks as to whether your material has already been digitised (or is being considered for digitisation) elsewhere? Given the nature of the Cultural Heritage Sector in New Zealand this alone is a good reason for creating and maintaining ‘National Registers’ of Books, Newspapers, Magazines/Serials, Art and Sculptures, and so on.
For example – NZMS has repeatedly called for a National Register of Newspapers in published papers or at conferences and because of our lengthy involvement in this space we have extensive knowledge of holdings in cultural heritage institutions nationwide – if we don’t know the answer there’s a very good chance we will know someone will.
And this stretches to all formats… it’s good to share information like this collegially in our part of the world! This article shows you how we (‘New Zealand Inc’) saved ourselves digitising 750,000 pages of NZ Parliamentary Debates (Hansard) by reaching out to the Hathi Trust, who shared digitisation work undertaken by the University of California and Google some years before but previously inaccessible to most New Zealanders.
Thereafter your documentation will allow you to consider:
If you have permission to digitise the material?
Have you researched if there are any copyright/rights issues for your collection?
Please refer to the ‘Further Reading’ section at the end of this document for more on this.
We think some of the above issues will dictate where you can get best return on investment early in your programme for your strategy.
Is it something visual, such as photographic content, or art, where the pictures tell thousands of words?
Is it a large volume of manuscripts, serials or books, where the price per unit is lower and the increased access, searchability and discovery of your collection is enhanced?
Is it the material that is easiest to access, or presently in the highest demand? Or, the material that is decaying so you are ensuring it is captured before important intellectual content is lost?
Is equipment and resources in-house or outsourced serendipitously available for certain formats inside your collection at the right time?
All of these issues will factor into the planning component of your digitisation strategy.
Review your resources – the How
Having decided why and what you are digitising, the next important question is how the digital files will be created and used, and by whom.
For those organisations that have enough existing staff with the time, expertise and knowledge of processes, handling and standards to complete a digitisation project, this is may be a straight forward question. For those that don’t, an adequate budget as well as time and expertise will be required to hire new talent, undertake training, and buy new equipment (if in-house) to conduct digitisation.
Effective programmes need to be structured in a way that ensures success.
A governance group that establishes the programme, monitors performance and approves changes to ensure that the organisations goals are being achieved. A programme manager reporting to the governance group has oversight over the entire programme, while technical leads and specialists deliver the outcomes. We have experienced with many organisations that the technical leads or those with specialist knowledge are installed as programme managers with mixed success.
For longer projects it is important to plan for staff turnover within the programmes governance group, management or technical teams.
Handling the material from its storage location to its digitisation location is important to protect the materials and ensure the integrity of the programme. It is important to spend time planning how you might retrieve the collection items and package them for relocation to a digitisation location in-house or offsite (eg trolleys, heavy lift equipment, crates, packaging for moisture/ vibration/ knock /drop /air quality protection).
Similarly handling the material during digitisation is critical. The equipment and the environs need to be appropriate with sufficient physical space for the digitisation equipment and processes involved with handling potentially historic and fragile material that may need to be laid out for preparation, capture and post-process purposes.
Deciding your output – the second What?
One of the most important decisions in determining your strategy is deciding what your digital output will be. This is influenced by the organisational and user requirements as well as the material itself.
Working to Standards
As with any important endeavour, creating and working to standards is a vital platform for success and a way of measuring success in some elements of your programme. NZMS holds a unique position in the cultural heritage digitisation sector in New Zealand in that we work to all pertinent standards respected or adhered to by major international libraries, universities, museums and archives. It takes years of experience to train staff and build steps into your systems and processes to work to these standards and we think it is important to recognise that you cannot form a team and ‘just do it’.
Some key standards to consider are;
- FADGI (Federal Agencies Digital Guidelines Initiative)
- NARA ( National Archives and Record Administration)
- Archives New Zealand Recordkeeping Standards – Destruction of source information after digitisation (Document Identifier: 17/G13) – this is the one with the Tech Specs included
- Guidelines for handling Cultural Heritage materials such as National Library of New Zealand, Te Papa and Archives New Zealand.
Your choice of standards will determine whether you require high resolution preservation master images in RAW or TIFF format conforming to rigorous preservation standards, quality archive files to showcase your collection well – when printed or displayed online, or simply a reduced size file for easy access to information (eg indexing, transcription projects), or merely easy to upload online and to distribute.
The material will also drive choices, for example modern documents may need to be digitised in black and white or greyscale and made text searchable with Optical Character Recognition (OCR) software. While historical documents or artworks may require high colour accuracy, tonal fidelity and colour balancing to match the original.
After capture, the images may also need additional processing that require specialist software and skills:
- image management: cropping, noise reduction, colour balancing, sharpening, resolution
- file management: standards compliance, formatting and converting file types, derivatives
- content and text conversion: OCR, OMR, ICR, transcription, indexing
- filenaming, indexing and metadata: informative or simply a unique ID, link to existing catalogue or database information, file format or perhaps JHOVE compliance, additional consistent information for metadata fields or specific information bespoke to each image.
If a programme objective is to make your material accessible, you will need to have a plan for how you will access and share these files. Often a cultural heritage organisation will implement, or already have, a smart Collection Management System or online community engagement platform to access, manage, share and engage a wider audience with their digital collections. Typically, images will be uploaded/ingested into such a platform and shared through a website and social media, perhaps also being linked to other online collections via metadata. Making your collections accessible is important, but your stakeholders can also add to the knowledge about your collections if they can interact with your content beyond liking and sharing, by adding stories, metadata, or even new digital content.
Finally, creating a new plan for managing your physical collection after digitisation enables you to restrict access, repair, conserve and carefully package and store your collections. New options become available such as remote storage or even destruction. At the very least you can return it securely to its original storage location with an acknowledgement in your systems that the material has been digitised.
Addressing the issues above will provide some of the key ingredients of your digitisation programme and strategy, and certainly the start of a business case.
Outsourcing or Inhouse – the Where
NZMS’ highly skilled digitisation and strategy experts can help you review the pros and cons of in-house or outsourcing elements of that strategy and an array of factors to consider when making this decision.
Contact us for a resource identifying the advantages and disadvantages of in-house digitisation versus and outsourced programme. And of course there’s the option to have elements of both.
Timelines and Budgets – the When
The last thing that must feature in your planning is;
How long have you got to do the work and what will it cost?
The two are inextricably linked which is why we have grouped them here. There are numerous things to consider and we have listed some of those in this section.
Balancing the people, resources, technology with timeframe and costs is the final puzzle to solve, recognising that a longer time frame means you have to remain ‘current’ with developing trends and standards over the life of the project.
The scale of the programme, be it small, medium, large, or ‘mass’ digitisation, will also feed into the equation determining the best way to amortise the job over the period available and the resources available to do the work.
The financial approach will also influence your programme design with options to capitalise the digital assets offering an alternative using an operational budget, as well the options to purchase or lease equipment.
Creating an inhouse capability can be a resource-consuming process, taking existing staff away from their day-jobs to manage/conduct this, and if you are hiring staff there are associated HR/OSH management functions to factor in. Correspondingly, if you are outsourcing the programme, you will need to consider how the project will be managed, and how quality will be assured.
Where there is a critical start and or finish date, you will need to consider more hardware and software with more staff, or maybe you can use the same equipment and work multiple shifts?
It is important to understand that the equipment and software will have cycle times, and life-cycle for critical parts (a ‘Mean Time Before Failure’ (MTBF)) that will affect how often you must replace or maintain. Therefore, the cost of maintenance, repairs and replacement must also be factored across the programme. This will change with the type of equipment, for example a DSLR will have an economic working life of around 150-250,000 images while a flat-bed scanner with a tri-linear array may exceed 500,000 images.
Those latter metrics demonstrate how the running costs of hardware (and software if it is click-based or paid monthly or annually) must be factored across the life of the project, including replacement PC’s and lighting. It also infers the upfront costs for some equipment may be higher – but they’re a cheaper option across the life of the project. Another feature of running costs is the throughput metrics of hardware and software. What throughput speed is achievable for the alternative technologies – we promise you it isn’t what is written on the spec sheets! In fact we have a typical rule of thumb we tend to apply for tri-linear scanners, and support this with good old-fashioned experience for all equipment types.
Budget – surely you didn’t set that first?
We left it to the end intentionally – because most ICT projects that fail are the result of poor planning – or unrealistic expectations. Often that’s because the project was based around available budget, and the solution was shaped to suit it, rather than the fundamental requirement of meeting your strategy, your aims. So perhaps inferior technology not up to the task for the period foreseen is then purchased, or perhaps the “lowest price-conforming quote” was accepted even though it wasn’t tested thoroughly because of procurement rules around supposed probity, or because the hardware/software could not ‘quite’ do what the specifications said.
The best way to create realistic budgets and programme expectations is to conduct a pilot to ensure everyone is on the same page. We will often confirm pricing after the pilot to keep everything above board. This allows you to approach the procurement process with the intention of achieving the best value for money – not necessarily the cheapest offering that appears compliant. Depending on the nature of the work it is advisable to pay for the pilot so that you can use the results in a procurement process if required. When you want the best possible outcome for your stakeholders this may be money well-spent to almost guarantee success.
And a closing nod in this section to sustainability… it’s unrealistic to undertake a digitisation project or programme without the organisation understanding the ongoing costs to sustain the images and data online, maintain their backups, with their ongoing maintenance and storage / access costs. The digital images require the same time and management in a similar way you would manage the originals and you may even have to cater for depreciation and disposal actions of the electronic records over time.
The shape of your solution
By having clear goals, it becomes easy to measure the effectiveness of the programme, reporting the difference the programme has made and identifying which goals have been met. This will enable the programme to stand up to any audit or scrutiny.
You can take this to various extremes – the low end being measuring your initially stated tangible aims, and a subjective assessment of the intangibles, recorded in your strategy at the outset. At the other extreme we deeply respect the Balanced Value Impact Model (BVIM) for the impact of your digitisation on people’s lives. Or to cite Simon Tanner of King’s College in London (who was a keynote at the 2013 National Digital Forum):
“The measurable outcomes arising from the existence of a digital resource that demonstrate a change in the life or life opportunities of the community for which the resource is intended.”
This is important, all too often performance metrics are stated at the outset and measured routinely, but these mustn’t merely be project milestones. There must be measures showing the impact of your work on your intended (and unintended) audience.
Here’s what our National Library experiences, through the eyes of the Office of the Auditor-General in a June 2018 report on digital access to information and services: Section 2.57:
“There is no accepted method in New Zealand or internationally that national libraries can use to assess the benefits of increasing people’s access to digital information. This is because many of the benefits are intangible. National libraries from overseas are experimenting with ways to assess the benefits of their digital services.”
Many of us understand the high moral capital the cultural heritage sector brings to our communities and lament the fact that the financial capital accorded the sector rarely backs that up.
Part of an audit should be sustainability, will the outcomes of your programmes, and your strategy survive and be effective for as long as you intended?
A frustrating feature of philanthropic funding in our part of the world is that they’re generally restricted to one-off investment. This can be terrific for getting digitisation projects done, but is rarely confidence-building for running digitisation programmes, executing a business strategy, nor ongoing sustenance of the outcomes of your projects and programmes. The reality is that ongoing costs are real: in the form of hosting, maintenance, integrity checking, also licences and subscriptions which have to be contractually renewed.
In closing, we think we all know each collection is unique, and invariably deserves to be part of a bespoke or tailored digitisation strategy. At NZMS we have a wealth of rich experience over many years unmatched in New Zealand and are experts in assessing collections, priorities and working with you to create the most suitable digitisation strategy for your organisation. We can even connect you with colleagues in the sector who have “been there, done that” and are happy to share their experiences collegially.
- Questions you need to ask when developing a digital strategy (targeted at education providers but in our opinion a good crossover for any organisation with online content intended for community engagement/interaction) – here
- Plan how you will digitise your content with a handy digitisation toolkit to help guide you through the planning and execution of digitisation activities – here
- At a higher level, what about NZ Government direction in the digital strategy space? We think this this cements the broader concepts for justifying value in your digital programmes: “It supports better public services and agency digital transformation, and puts citizens and businesses at the centre of digital services” (which is probably written into your organisation’s Plan and Strategic Direction as well) – here
- A recent independent review of our National Library is performing with respect to digital access to information and services: An impressive performance by NLNZ; they’re meeting “expectations about openness, availability, reasonable pricing, and reusability. Its strategic documents also align with the Government’s goal of openness and transparency. The Library is continually working to improve access to its collections and improve its services.”. Some nice tips here you may be able to apply to your experience. There’s also a good section on “Dealing with Copyright Issues” – find it here.