Name: Daniela Petrelli Nick Dulake Mark Marshall

Name of Team): meSch@SHU (pronounced: mesh at shoe)

Job Title: Professor of Interaction design; Senior Product designer; Research Fellow

Organisation : Sheffield Hallam University

Abstract

Many books in the BL digital collection are not books one can read, not because of preservation issues (they cannot be touched) but because the content is unintelligible. Books like Leonardo’s Notebooks have an immense attraction for the public, but people do not have the means to understand what they see. Those mysterious books are ideal to be used in interactive experiences: by creating smart replicas of those books and embedding technology in them, it is possible to reveal the mysterious content when the visitor directly interact with the book, i.e. touching a drawing will play Leonardo’s voice telling of his inventions or using a magnifying glass to look at the text will display the text in English on the lens.
Revealing Books aims at exploring the possibilities pervasive computing holds for bridging the gap between physical books and digital content. Digital technology in the form of sensors and actuators can bring a magical dimension when interacting with books, e.g. conductive ink can control the play of content by simple touch on a normal paper page and display other than on screen (electroluminescent ink or projection) engage visitors at an emotional level. By using a research-through design approach we will investigate different forms of physical interaction with books that control digital content. We propose a co-design co-creation approach and we will team with curators in BL for the selection of the most appropriate books and digital content. Smart replicas of the books will be created that will completely conceal the technology but will react to human interaction. The technology within these artefacts will connect with BL digital collection, some reformatting / processing may be needed as ubiquitous computing technology have limited computational power.
The project will progress iteratively alternating design, prototyping and evaluation within the team including BL curators and technical experts. A final exhibition open to the public will be our means to collect data for our research. By observing visitors and invite them to offer their thoughts and suggestions we will be able to see if the physical engagement augments the appreciation and understanding of the book.
We have used this type of approach to integrate digital and physical with three museums exhibitions used by over 20,000 visitors over a period of 6 months. We have therefore convincing evidence this could bring novelty and excitement among the visitors of the BL collection.
Relevant URL (if available): examples of our work can be seen at
permanent installation: http://www.museodellaguerra.it/visita/mesch/voices-from-the-past-in-fort-pozzacchio/
temporary exhibtions: https://www.youtube.com/watch?v=sK3AdQU9kkc

Research Question / Problem

The research question you are trying to answer
Please focus on the clarity and quality of the research question posed.
We are a multidisciplinary team interested in exploring the opportunities ubiquitous computing offers to bridge the gap between the digital and the physical and integrate the two as part of the same experience. While we have already taken this approach very successfully on the exhibition floor of two museums, we would like to take up the challenge of engaging people with books they cannot understand. Digital content can be added as a layer to the physical embodiment creating a magic experience when the physical book is manipulated and the digital content is revealed.
Our research questions can be expressed as: how can readers engage with books they cannot touch and cannot read? Is there an added value in a physical interaction (instead of a screen based interaction)?
Some ‘books’ in the British Library collection are world famous and have been extensively studied. Despite being available in full as digital resource, those books cannot be read by the general public as they are unintelligible. Information that is likely to be of interest to readers is available but not connected to the book. The result is that those fascinating objects are not widely accessed despite their electronic form because the reader lacks the knowledge to interpret what is on the page.
A well known example in the BL collection is Leonardo’s notebooks: his right-to-left cryptic writing is like a secret code and the diaries cannot be read by a lay-man, even by an Italian speaker. Other examples are: the Lindisfarne Gospels written in Latin and with transcription in Old English in between lines or the Mewar Ramayana.
All these book are richly illustrated and have a clear visual appeal. However without understanding the text they cannot be appreciated in full.
Smart replicas can be created using conductive ink (and maybe electroluminescent ink to add a sense of magic); NFC or other tags can be embedded within the replica and tools given to readers (a magnifying glass, an ink-pen, etc.). Different interactions for the same content can be created and the delivery of content can be done in different ways, e.g. a voice reading, ghostly translations appearing when the original book is touched, projections on walls when the book on a lectern is approached / open / read.
While we can offer the hardware + software expertise (to some extent) we would need the help of a curator or expert in the content and technical support from he BL Lab in order to design interactions that are smooth and compelling, e.g. zooming on a page where specific content is linked would trigger a special effect.
We would do the design of the installation and interaction in collaboration with BL labs.
We have all the workshops and material needed for industrial and professional production of the final installation.
Our research is a combination of the process of making and the evaluation of how the final artefact performs when placed in-the-wild. So while part of the research is the design process, the completion will be the data we will collect at the exhibition. Here we will implement a consolidated evaluation schema that included unobtrusive observations of visitors paired with interviews at exit time. Methods of qualitative data analysis are then used to determine if the intended goals of engagement and emotion are reached.

Showcasing BL Digital Collections

The project will explore new ways to consume digital content that is not screen based and in front of a computer. We have so far found in our evaluations with museum visitors that the combination of digital and material is compelling. We believe this would give the opportunity to the British Library to showcase their digital collection in a completely different way that holds much potential for the public and for other uses, for example deployment in schools as part of existing curricula. As the replicas can be remade and the digital content is always available multiple new uses could be considered too, such as deploying an interactive “The Origin of Spices” at Down House.

Methods

The research is practice-led with iterative cycles of design, fast prototyping and tests. It is an established research process in design and arts, i.e. disciplines that are practice-led and in which the knowledge and understanding derives from the making of artefacts and observing them in the real world to see how people react / interact with them. Visitors to and friends of BL will be periodically involved in the different cases of the research, from the generation of concepts and ideas to the testing of prototypes.
A public event will be organised at the end of the project where the final research products will be let to the public to use and explore. Our research products have been already successfully installed in three museums and over 20,000 visitors have interacted with them. We are currently working with English Heritage to design and deploy an interactive installation for Hadrian’s Wall. We are therefore confident our research approach is solid and can deliver research products of interest to the public and the institutions.
The process we use is summarised in this paper: http://mw2016.museumsandtheweb.com/proposal/do-it-together-exploring-tangible-and-embodied-interaction-as-a-team-of-curators-designers-and-technologists/

Evidence that Entrant(s) can successfully complete the project

As part of the meSch project, over the last 3 years we have been investigating how digital content can be integrated with material collections to create novel experiences for museum visitors. We have explored a number of different strands in the context of temporary exhibitions as well as permanent collections receiving excellent feedback from both visitors and curators. More information can be found in the project website http://www.meschproject.eu/ including academic publications

How idea is achievable on a Technical, Curatorial and Legal basis

Technical
Some of the technology we intend to use was installed and used in three museums and over 20,000 visitors have used it. Additional lab-test were done with conductive ink with excellent results. We have technical expertise that complements that of the BL Lab as we cover all aspects of electronics and installation.
Curatorial
The content will be completely controlled by BL curators. At the beginning of the project we will run a co-design workshop to brainstorm in interaction ideas as well as to identify the most appropriate books in the BL digital collection for the purpose of the installation. Therefore, apart from the availability of curators, we do not see any issue emerging.
Legal
We do not see any legal implications as the interactive books will have a physical form and therefore easy to control. We will request the possibility of publishing the work done in academic and professional fora.

Plan

June 2016
Activity described here (e.g. what, when and by who)
1-15 BL: to identify a range of possible books to be used in the workshop, e.g. the manuscripts in Turning the Page, that could appeal to the public, are in digital form and for which content is available or can be easily created.
1-15 BL: to identify and book a suitable space for the exhibition in October.
1-15 SHU: plan and run a Co-design workshop with BL curators and BL Lab experts to generate concepts on the interactive books. The books / material identified will be used
15-30 SHU: to provide a storyboard that illustrated the envisaged interaction of visitors with the books / installation. The concepts will be those agreed upon in the co-design workshop.
July 2016
Activity described here (e.g. what, when and by who)
BL-Lab and SHU to work together on the first technical prototype. Issues of API and access, computational resources, conversion, scaling etc. will be solved at this time allowing the project then to progress smoothy on the bases of an agreed technical setting / platform.
August 2016
Activity described here (e.g. what, when and by who)
SHU: to develop the first prototype.
BL: to select the final content to be used in the exhibition.
September 2016
Activity described here (e.g. what, when and by who)
1-7 BL: to provide the final content in the final digital form.
1-7 SHU: to organise and run the first evaluation of the prototype(s) with the curators and BL lab personnel that participated in the co-design workshop in June. The content may of may not be the final depending on possible needs to process it for the technical constraints of the final use.
7-21 SHU: to reflect on the evaluation, re-design what is needed. Final development and testing.
October 2016
Activity described here (e.g. what, when and by who)
7-15 SHU + BL: to organise and run an evaluation of the interactive books within BL. Collect feedback for the final installation.
21-31 SHU + BL: to install and run an interactive exhibition in a dedicated space identified by BL.
November 2016
SHU: data collected at the exhibition will be analysed and a report (likely an academic publication) will be written.