Desmond Schmidt, Anna Gerber

Winner's blog posts:
Text to Image Linking Tool

Submitted Entry for 2014 Competition


Many digital humanities collections, including those belonging to the British Library, contain images of manuscripts and early printed books that may be of considerable cultural significance. To make these materials useful to the general public they need to be made accessible over the Web. The text itself may be hard to read, and the page images of even clear texts are of little use on their own without a good-quality transcription that can be reliably searched, annotated or edited online through crowd-sourcing. Often transcriptions of the contents are already available, for example, transcriptions of famous works of literature, or important documents etc. In such cases there is a hard visualisation problem to solve: how can the reader make sense of the transcriptions and correlate them to the images of words on the facsimiles? The two types of information can be easily displayed side-by-side but it is easy to lose one's way, not knowing to which piece of the image a particular word or line corresponds, especially since the transcription and page images usually have different lineation. Approaches to date have been very labour-intensive. Individual links between words in the text and the facsimile have been constructed by hand. Clearly some form of automation could solve the problem, and some software for the AustESE project was developed called TILT (Text to Image Linking Tool). TILT used a Java Applet to recognise shapes in the facsimile without attempting to recognise their content and then correlate them to words in the transcription. In this way it is hoped that large numbers of such text-to-image aligned pages could be produced semi-automatically at low cost. The subject of the proposal is thus a revision of TILT by splitting it into two halves. The first half would a Java-based Web processing service that would identify word-shapes in facsimiles and return that information to the browser. The second half would draw the shapes and facsimiles and display their links to the text using standard Web technologies, i.e. HTML5 and Javascript.

Assessment Criteria

The research question / problem you are trying to answer

Please focus on the clarity and quality of the research question / problem posed:

The problem is essentially a HCI (human-computer interaction) problem. How can the user connect the contents of digitised printed books or manuscripts on the one hand with a readable and searchable transcription of their contents on the other? Without such assistance the user would spend too much time trying to work out what corresponds to what, where he/she is currently on the page. A linking mechanism to provide visual feedback would overcome this.

Please explain the ways your idea will showcase British Library digital collections

Please ensure you include details of British Library digital collections you are showcasing (you may use several collections if you wish), a sample can be found at

The mechanism would work with any of the digitised books in the BL digital collections that have sufficient cultural significance to make construction of such an interface worthwhile. By providing a transcription next to the page facsimiles that the users can annotate, read and potentially edit through crowd sourcing the user experience would be greatly enhanced.

Please detail the approach(es) / method(s) you are going to use to implement your idea, detailing clearly the research methods / techniques / processes involved

Indicate and describe any research methods / processes / techniques and approaches you are going to use, e.g. text mining, visualisations, statistical analysis etc.

The image recognition is implemented using the Java class library, which has already a fast and fully featured set of tools for that purpose. The server program will deliver standard HTML5 and JSON drawing instructions for the browser. Drawing and user interaction at the browser end will utilise Javascript technologies such as jQuery, and HTML5.

Please provide evidence of how you / your team have the skills, knowledge and expertise to successfully carry out the project by working with the Labs team

E.g. work you may have done, publications, a list with dates and links (if you have them)

The work is an extension of part of the AustESE project at the University of Queensland. Both applicants are experienced software developers who worked on that project. DS will handle the Java server coding end and AG will take care of the user interface design. The applicants have sufficient experience to cover a wide range of suitable technologies with which the BL Labs team may wish to integrate the solution.

Please provide evidence of how you think your idea is achievable on a technical, curatorial and legal basis

Indicate the technical, curatorial and legal aspects of the idea (you may want to check with Labs team before submitting your idea first).

The proposed revision of TILT is described here:

The code will be an Open-Source project and part of ongoing development of AustESE, and will be incorporated in the current Charles Harpur (University of New South Wales) and De Roberto I Vicere projects (University of Sassari, Italy).

The existing code is already licensed under GPL2, and any new code will conform to the same license.

Please provide a brief plan of how you will implement your project idea by working with the Labs team

You will be given the opportunity to work on your winning project idea between May 26th - Oct 31st, 2014.

May 26 2014- Onwards
DS: Reorganise TILT1 code into server-side Java service through negotiation with BL Labs.
AG: Sketch design of UI through wireframes and seek potential user feedback

June 2014
DS: Incorporate HTML generator from AustESE project for standoff markup representation of text to image links. Design suitable JSON protocol for sending drawing information to the client
AG: Build prototype UI using jQuery or other JS libraries

July 2014
AG: Seek feedback from potential users and by showcasing test interface on Labs website or elsewhere. Use contacts on Humanist discussion group and elsewhere to garner feedback.
DS: Extend and maintain server code in response to identified user issues

August 2014
AG: Extend and maintain user interface in response to user issues
DS: Extend and maintain server code in response to identified user issues

September 2014
AG: Extend and maintain user interface in response to user issues
DS: Extend and maintain server code in response to identified user issues

October 2014
Present final prototype with examples images from BL collection.