CHAPTER 9 Jeffrey P. Emanuel EC TE D Harvard University, Cambridge, MA, United States PR OO F Stitching Together Technology for the Digital Humanities With the International Image Interoperability Framework (IIIF) 9.1 INTRODUCTION UN CO RR Compartmentalization of information, communication, and effort are an inevitability in an organization of almost any size. Despite wide recognition of the risks they pose, and repeated commitments to cross-divisional collaboration, silos develop quickly, and once in place, it is difficult to effectively work across them with consistent success[1–4]. Similarly, digital material held by li-braries and museums around the world can become trapped in silos, with users afforded very little choice about how they will access and interact with individual objects, and very little consistency in these methods across institutions and repositories. A clear, focused emphasis on collaboration and interoperability, between both institutions and technical approaches, can provide significant relief from this conundrum, and can provide value for organizational members and technological end-users alike. Harvard University is an example of a large organization, with a complex internal structure, that can make interdivisional (and, in some cases, intradivisional) collaboration difficult, particularly on projects of larger scope. Over the last five years, though, a significant digital humanities project has provided several groups at Harvard with an opportunity to work together toward a shared goal: the implementation of a standard delivery framework for visual material in repositories across the university, Digital Humanities, Libraries, and Partnerships ISBN 978-0-08-102023-4, DOI: 10.1016/B978-0-08-102023-4.00009-4 © 2017. 1 2 Digital Humanities, Libraries, and Partnerships PR OO F and the development of an image viewer for internal and external (library, museum, research, and teaching) image display. Despite the difficulty of pulling numerous disparate groups together on a large, multiyear project, the participants in the International Image Interoperability Framework (IIIF) and Mirador projects found tremendous success, and were able to forge a project- and standards-based alliance that was greater than the sum of its individual parts. This chapter will describe that partnership, and its success, with an emphasis on the IIIF and Mirador projects that it was built around. 9.2 THE INTERNATIONAL IMAGE INTEROPERABILITY FRAMEWORK UN CO RR EC TE D Library and museum holdings are being digitized and made available as pixels at an ever-increasing rate. This serves both access and preservation needs, while also increasing the potential for deeper interaction with digital visual material by the end user. However, efforts to share content across institutions, and even across repositories within an institution, continue to encounter numerous obstacles. Aside from rights and permissions, the most significant obstacle may be the differences in the handling of storage, management, and delivery of digital material across repositories, which frequently use separate frameworks, formats, and applications. This is a widely recognized issue, but efforts to resolve it have all too often fallen victim to what has been called “the not-invented-here syndrome”: the conviction that “you and I will collaborate just fine if you adopt my system and abandon yours.”1 For example, a library has an image in a database that a scholar wants to see. This is a simple undertaking, but the difficulty is compounded when that scholar also wants to access an image from another repository. Because of differences in storage and delivery across institutions, that scholar may have to use an entirely different application to view and interact with the second image, thus creating a very different experience. Because of this, the easiest way for people to access, curate, and view images in the environment of their choice has been to download the image and make a local copy. Add this up over the course of time and increasing viewer counts, and the result is tens, hundreds, thousands, or 1 Waters [5], p. 14. Stitching Together Technology for the Digital Humanities 3 CO RR EC TE D PR OO F more copies of individual images, potentially living on hard drives all over the world, in countless different resolutions and formats, and with wide variations in the associated metadata. This is clearly a problem—but how can it be solved? IIIF was designed to address this barrier to access and sharing of content by “promot[ing] the building of a global and interoperable framework by which image based resources could be easily shared and reused across institutions using any combination of image servers or client viewing software.”2 Questions that governed the development of IIIF included: • What if deep zoom were standard and fast? • What if you could compare across sites, hosts, and institutions? • What if you could collect items that belong together physically—not just intellectually (e.g., the miniature from a medieval folio was physically extracted, and lives at a different library than the folio itself)? • What if you could access images using the viewer of your choice, instead of a certain one tied to a particular repository? A community-developed and -maintained protocol for standardized image retrieval, IIIF was created in an effort to make affirmative answers to these questions possible, by collaboratively producing both interoperable technology and a common framework for image delivery. You can think of the countless locally-stored copies of an image as one of a countless number of identical water bottles, filled and shipped to the end user, where it will potentially be isolated from the source and from others forever. The impetus for IIIF was an effort to alter that method of image consumption[7], p. 5: UN The [IIIF] community set out to create pathways for images to flow through systems from their source, rather than trying to get all repositories and service providers to build everything the same way. Digital image content generated and served up by a range of different image servers could be standardized as output (both the individual image and the structure of an image grouping) for re-use…and could flow, rather than be handed off like plastic water bottles. 2 Snydman, Sanderson, and Cramer [6], p. 17. 4 Digital Humanities, Libraries, and Partnerships PR OO F It is axiomatic that academic and cultural heritage institutions are not going to drop their own tools and development efforts en masse in favor of uniformly adopting yet another one-size-fits-all third-party tool. The growing IIIF effort addresses this reality by focusing on common APIs, eschewing the quixotic quest for universal tool adoption in favor of creating a common method of content description and delivery. An API, or an Application Programming Interface, is at its most basic a means for applications to communicate: EC TE D An API is a software intermediary that allows two applications to talk to each other. Each time you use the Facebook app, send an instant message, or check the weather on your phone, you’re using an API. When you use one of these applications, the application on your phone connects to the Internet and sends data to a server. The server then retrieves that data, interprets it, performs the necessary actions and sends it back to your phone. The application then interprets that data and presents you with the information you wanted in a readable way. This is what an API is—this all happens via an API.3 UN CO RR IIIF currently consists of two APIs: image and presentation. The Image API simply brings the pixels, via a structure specifying the image’s source (URL), region (its specific location as a part of the original image), size (zoom percentage), rotation (in degrees), quality (color, grey, or black and white), and format (file type). The Presentation API then brings just enough metadata to drive a remote viewing experience, from the repository name and logo to the table of contents (if applicable) and image thumbnails. This method has several advantages over its download-and-view predecessor. The first is simultaneously the most simple and the most important: no images are leaving their home repositories, and no new copies or versions of them are being created. Instead, the image is accessed virtually and displayed within the end user’s image viewer, with the Image and Presentation API requests specifying just what portion of the original image should be displayed, and how. In this way, images from any repository in the world can be accessed, viewed, and interacted 3 https://www.mulesoft.com/resources/api/what-is-an-api Stitching Together Technology for the Digital Humanities 5 4 5 6 7 UN CO RR EC TE D PR OO F with as though they were stored locally—both individually, and in comparison with other images from the same or from completely different repositories (Fig. 9.1). In other words, circling back to the water bottle analogy, instead of handing off water bottles, users can fill the glass of their choice from the same tap. The number of potential sources is constantly growing: worldwide, over 100 institutions have exposed some or all of their collections via IIIF thus far, for a total of between 335,000,000 and 1,000,000,000 shared images.4 As this community grows, the world’s digital archives are made even more available and accessible to scholars both within and across repositories. A further advantage of IIIF is that the user is not locked in to using a single particular viewer. Many applications now feature IIIF compatibility, including (but not limited to) the Universal Viewer, Leaflet, OpenSeaDragon, and Omeka. Another IIIF-compliant viewer, used for visual material by Harvard and several other institutions, is an open source JavaScript application called Mirador.5 Primarily developed by Harvard and Stanford Universities, Mirador is a “multirepository, configurable, extensible, and easy-to-integrate viewer and annotation creation and comparison environment for IIIF resources, ranging from deep-zooming artwork to complex objects.”6 It “provides a tiling windowed environment for comparing multiple image-based resources, synchronized structural and visual navigation of content using OpenSeadragon, Open Annotation-compliant annotation creation and viewing on deep-zoomable canvases, metadata display, book reading, and bookmarking.”7 In briefer terms, Mirador currently provides scholars who work with digitized objects with a responsive, high-resolution viewer for digital objects [8]. Images from any repository that have been made accessible via the IIIF APIs can be viewed individually or together as a slideshow, while books can be viewed as individual pages or displayed in two-page view, complete with “opening” to emulate what has historically been one of the most important experiential aspects of the codex ([9], p. 51). OpenSeadragon’s deep zooming functionality allows large objects to be viewed with ease, while the layout of the viewer can be altered by adding http://iiif.io/community http://iiif.harvard.edu/mirador http://projectmirador.org/ http://projectmirador.org/; openannotation.org/ https://openseadragon.github.io/; http://www. Digital Humanities, Libraries, and Partnerships PR OO F 6 EC TE D Figure 9.1 Side-by-side comparison of digitized statues from two repositories, the Harvard Art Museums (left) and the Yale Center for British Art (right), accessed via IIIF API and displayed in the Mirador viewer. CO RR and removing “slots,” creating a grid of separate images so that multiple items from one or more repositories, or multiple aspects of a single item, can be examined in a single window (Fig. 9.2). UN Figure 9.2 A medieval Book of Hours (MS Richardson 7, Houghton Library, Harvard University) displayed in the left slot in Mirador’s “Book View,” with page thumbnails and table of contents included. In the slot at right is the denoted portion of the illumination on folio 15 recto, displayed at full zoom level (highlighting added). Stitching Together Technology for the Digital Humanities 7 9.3 ONE FRAMEWORK TO RULE THEM ALL: IIIF AT HARVARD UNIVERSITY 9.3.1 Risks EC TE D PR OO F Within Harvard, a multiorganizational effort to adopt IIIF has resulted in digitized content from multiple sources being exposed via a shared API, from the Harvard Art Museums and Library, the Faculty of Arts and Sciences (FAS), and the University’s massive open online course provider, HarvardX (Fig 9.3) [10,11]. The number of objects being made available is remarkable: the library has now exposed over 195,000 books and manuscripts, and over 14,000,000 total images, while the Art Museums have exposed over 250,000 art objects as IIIF manifests. In addition, image collections used in residential courses within the FAS and image content contained in HarvardX-produced learning experiences is also accessed via IIIF API and displayed in Mirador. Further, by sharing knowledge, expertise, digital content, and Mirador, multiple “heads” have sprouted: an image viewer and annotation application for HarvardX courses, a new Harvard Library Viewer, faculty image collections embedded in the Canvas course platform, and walls of images in the Harvard Art Museums, all making use of IIIF and instances of the Mirador viewer. 8 9 UN CO RR Such an undertaking was not without risk, of course, and in many ways, the undertaking on the planning, coordination, and development side mirrored, in human form, the aforementioned technical challenges that led to IIIF in the first place. The solution was similar, as well: greater coordination and interoperability between component parts being key. With its 12 schools, 70 libraries,8 and independent networks of museums,9 the University has a complex internal structure, which can result in communication gaps and replication of effort even in the smallest under http://asklib.hcl.harvard.edu/faq/82186 The major museum networks are the Harvard Art Museums (http://www. harvardartmuseums.org/), made up of the Fogg, Busch-Reisinger, and Arthur M. Sackler Museums, and the Harvard Museums of Science and Culture (hmsc.harvard.edu), which consists of the Harvard Museum of Natural History, the Peabody Museum of Archaeology and Ethnology, the Semitic Museum, and the Collection of Historical Scientific Instruments. Digital Humanities, Libraries, and Partnerships PR OO F 8 EC TE D Figure 9.3 Multiple uses of Mirador at Harvard, including library exhibits, library page delivery, residential and online learning, and museums, all utilizing the IIIF APIs to access images. UN CO RR taking. One of the risks identified in committing both to adopting IIIF and to helping drive the development of Mirador was the large number of organizations involved (both internally and externally), and the multiple handoffs in development that could result. An additional risk was the externally collaborative nature of the program, and the fact that Harvard did not have control over the IIIF and Mirador road maps and resources. IIIF and Mirador were “born” at Stanford University, which utilized funding from the Andrew W. Mellon foundation to finance work on both the IIIF specifications and the Mirador application. Further, the number of people acting in project management roles greatly outweighed the number of developers dedicated to the project. The sole software engineer working on IIIF and Mirador on a full-time basis was located at HarvardX, while management of the project extended from HarvardX to the Libraries—via the Preservation Services division and Library Technology Services, which is organizationally a part of Harvard University Information Technology (HUIT)—to the Arts and Humanities Research Computing group, which is associated with both HUIT and the FAS’ Arts and Humanities Division. This had the potential to create a “too many cooks in the kitchen” situation in every phase of the project, from requirements gathering, to development, to implementation. Stitching Together Technology for the Digital Humanities 9 9.3.2 Benefits EC TE D PR OO F Fortunately, the “too many cooks” issue resolved itself with surprising ease: the HarvardX software engineer hired to work on the project rapidly ascended to a position of product ownership within the university, and leadership within the broader IIIF consortium. This allowed the project management team to focus less on management and more on coordination and support, sharing use cases and project while acting as a united front to drive prioritization and development. This also allowed Harvard to move quickly on time-sensitive needs. For example, when outside assistance was needed to update the coordinate system used in OpenSeaDragon, the image display technology used in Mirador, it was funds provided by Academic Technology at Harvard that allowed the contract with an OSD developer to be finalized and executed in a timely fashion. UN CO RR On the other hand, the potential benefits of IIIF for the university were clear. This approach could open Harvard’s library, museum, and course-based digital content for reuse over the web, while also allowing the university to reuse digital content from peer institutions. Use cases spanned the continuum from teaching, to research, to library and museum exhibits. For example, a “virtual manuscript” could be compiled using pages held by both Harvard and Yale; Harvard and British Library copies of the same work could be compared in great detail within the same viewer, and online exhibits—or virtual companions to physical exhibits—could be created using material from multiple institutions. The exhibit-based research case was exemplified by the 2016 project “Beyond Words: Illuminated Manuscripts in Boston Area Collections,” an exhibit that spanned three institutions: Harvard’s Houghton Library, Boston College’s McMullen Museum, and the Isabella Stewart Gardener Museum. “Beyond Words” featured more than 260 manuscripts and printed books from 19 Boston-area collections, dating from the 9th to the 17th centuries (http://beyondwords2016.org). Co-led by Harvard’s faculty champion of IIIF and Mirador, Jeffrey Hamburger (Kuno Francke Professor of German Art and Culture), the exhibit was set up with both material and virtual components: physical codices on display at each exhibit location were accompanied by tablets running Mirador, thus allowing visitors to appreciate the manuscripts’ materiality 10 Digital Humanities, Libraries, and Partnerships CO RR EC TE D PR OO F and construction, while also being able to page through each, and to appreciate images and text more clearly though OpenSeaDragon’s deepzooming functionality. For the Harvard Library, IIIF provided further advantages. Firstly, the API-based framework offered a more efficient and standardized way to deliver visual material from the Digital Repository, and allowed Harvard scholars easier access to other institutions’ collections. Secondly, Mirador offered the library the much-needed opportunity to replace its outdated Page Delivery Service (PDS), which had been used for years to deliver digital material despite its lack of suitability to noncodex formats (whether codices or another physical form, like scrolls, the PDS uniformly delivered visual material as paged objects which had to be clicked through for viewing). In teaching and learning, on the other hand, IIIF and Mirador offered a highly interactive viewing experience for visual material while affording faculty and course staff the opportunity to dynamically access library and museum material, and their own collections, without having to replicate images at every turn. The Art Museums’ attraction to IIIF came from its ability to help meet three particular goals: enhancing the public’s desire to view the physical objects in the museums’ galleries, expanding image comparison capability in the museums’ digital tours platform, and serving as a study in the effort to demonstrate that museum data can be interoperable. As IIIF standards further mature, the Art Museums anticipate using this framework to deliver three-dimensional objects, both for viewing and for virtual reconstruction, as well as complex living documents like curatorial object files and archives. In this way, the tablet approach utilized in the “Beyond Words” exhibit can likewise be applied to physical art exhibits. 9.4 CONCLUSION AND NEXT STEPS UN Despite its breadth and complex internal structure, several organizations within Harvard have found IIIF and the Mirador viewer to be a standard around which they can rally in collaborative and communicative fashion. From the Library to the Art Museums to the classroom—both residential and virtual—Harvard has embraced the shared-API approach that IIIF offers for image delivery and retrieval, and the openness that comes with being part of a hundred-institution consortium dedicated to the sharing of digital visual material. Stitching Together Technology for the Digital Humanities 11 REFERENCES EC TE D PR OO F The potential next step of this process for the University, which has just completed the requirements-gathering phase, is to construct an IIIF-compliant scholars’ workspace. This workspace would allow scholars to take full advantage of IIIF and Mirador, while affording them the ability to collect, store, share, annotate, and arrange high-resolution digital images from multiple repositories worldwide for both teaching and research [12]. Drawing on the remarkable interoperability afforded by IIIF and the deep interaction that Mirador brings to the study of digital visual material, this workspace would provide users with the equivalent of a digital office or library carrel holding personal images, as well as references to, and notes on, as many web-accessible digital works as their research or class projects require. UN CO RR [1] R.M. Kanter, Collaborative advantage: the art of alliances, Harvard Business Rev. 4 (1994) 96–108. [2] A. Kezar, Redesigning for collaboration within higher education institutions: an exploration into the developmental process, Res. Higher Educat. 46 (7) (2005) 831–860. [3] L. Marchese, How the ‘silo effect’ is hurting cross team collaboration. <https:// blog.trello.com/tips-to-improve-cross-team-collaboration>, 2016, May 10 (accessed 1.03.17). [4] R. Schaubroeck, F. Holsztejn Tarczewski, R. Theunissen, Making collaboration across functions a reality. McKinsey Quarterly, March 2016. <http://www. mckinsey.com/business-functions/organization/our-insights/ making-collaboration-across-functions-a-reality>, 2016 (accessed 1.03.17). [5] D.J. Waters, Digital humanities and the changing ecology of scholarly communications., Int. J. Human. Arts Comput. 7 (2013) 13–28. [6] S. Snydman, R. Sanderson, T. Cramer, The International Image Interoperability Framework (IIIF): A community and technology approach for web-based images. Paper presented at the conference Archiving 2015, Los Angeles, CA, May 19–22. <http://purl.stanford.edu/df650pk4327>, 2015 (accessed 30.09.17). [7] W. Ying, J. Shulman, Bottled or tap? A map for integrating the International Image Framework (IIIF) into Shared Shelf and ArtStor, D-Lib 21 (7/8) (2015) 1–12. [8] V.J. Harward, J. Hamburger, J.P. Emanuel, R. Singhal, R. Stern, Oculus: Using open APIs to share Harvard’s digitized books and manuscripts. Paper presented at the 4th Annual Harvard University IT Summit, Cambridge, MA, June 4, 2014. [9] J.F. Hamburger, Openings, in: G.C. Kratzmann, J.F. Hamburger, R. Minion (Eds.), Imagination, Books & Community in Medieval Europe, Macmillan, South Yarra, 2009, pp. 51–133. [10] R. Stern, J.P. Emanuel, V.J. Harward, R. Singhal, J. Steward, IIIF as an enabler to interoperability within a single institution. Paper presented at the conference IIIF New York 2016: Access to the World’s Images, New York, NY, May 9–11, 2016. [11] J.P. Emanuel, A. Barrett, J. Steward, R. Stern, One framework to rule them all: The unifying impact of IIIF on teaching, research, museums, and libraries at Harvard. Paper presented at the annual New Media Consortium Summer Conference, Boston, MA, June 13–15, 2017. 12 Digital Humanities, Libraries, and Partnerships FURTHER READING PR OO F [12] V.J. Harward, J.P. Emanuel, Planning Phase for a IIIF-Compliant Scholars’ Workspace for Visual Material. Harvard University ITCRB Grant Proposal (funded FY2016), 2016. UN CO RR EC TE D Emanuel, J.P., Morse, C.M., Hollis, L., 2016. The new interactive: Reimagining visual collections as immersive environments. VRA Bulletin 43 (2), 1–16.