This article originally appeared in the September/October 2011 edition of Museum magazine.

An excerpt from Mobile Apps for Museums, available from the AAM Press.

In the past year, the innovative forms of augmented reality (AR) appearing on smartphones have proven to be exciting playgrounds for curators and museum educators. These AR tools offer users the possibility to deploy their phones as pocket-sized screens through which surrounding spaces become the stage for endless extra layers of information. This visual collision of the real and the virtual—made possible by using GPS and a compass—could culminate in what we have seen in movies like Minority Report (2002), where Tom Cruise physically navigates through 3D data: a seamless interface between the body, the virtual and the real.

Currently, however, AR technology (Layar or Junaio, for instance) is still a kind of experimental medium, as yet lacking the total immersion that science fiction promises. Moreover, its mediation through a tiny hand-held screen poses several challenges to augmented storytelling. What, then, does this contemporary form of AR have to offer the museum today?  Why would a museum want to develop augmented reality tours? What kind of user experience does it entail? Is it, in this day and age, the ultimate app? These questions will be addressed here by taking a closer look at the experiences of the Stedelijk Museum’s AR project, ARtours, which explores a number of augmented reality applications in order to experiment with these new platforms in different contexts and with different kinds of art.

Lieux de Mémoire, Space Hacking and Artistic Platform

Taking a closer look at the deployment of AR by museums, it seems that the attraction of this new medium is often found in the act of returning cultural heritage to the streets where it was originally produced and/or that it depicts. As the apps of the Powerhouse Museum and the London Museum effectively illustrate, AR allows users to see photographs on their smartphones of old city views overlaid on the places where they were shot. Comparing a “real” contemporary with an “augmented” older view offers a moment of reflection on history, modernization, and change.

The Netherlands Architecture Institute (NAI) included even more time dimensions in its unequaled application (UAR), as visitors are not only treated to former architectural drawings of the locations where one is positioned, but also to unrealized designs and future projects. The strategy, however, remains the same: using AR as a medium to layer the urban realm with a museological collection in order to compare its current outlook with that of other times and ages. In a sense, it is using AR as a form of what French historian Pierre Nora would describe as lieux de mémoire (sites of memory).

For a modern and contemporary art museum like the Stedelijk Museum in Amsterdam, this strategy for AR deployment is relevant in that the word “Stedelijk” means “municipal,” and parts of the collection are produced by or related to the Amsterdam cityscape. However, layering the streets and canals with local artworks has certainly not been the reason for investing so much energy in the development of ARtours. First and foremost, the museum is known for its extensive international collection of art, photography, and design, which itself asks for a different curatorial approach and visitor experience. Secondly, the Stedelijk Museum has been closed since 2004 due to a renovation of its original building and construction of a new wing. AR was therefore primarily embraced because of the possibilities it offers for exhibiting the collection, as the museum has lacked an analog venue to do so. In other words, in addition to lieux de mémoire, the Stedelijk opted for “space hacking,” a strategy in which augmented reality is used to present the collection in spaces with which the art has no relation whatsoever, but are used simply as a new stage.

We experimented with this strategy in the ARtours project entitled “ARtotheque.” the idea is simple: The Stedelijk Museum holds thousands of artworks in its collection, so why not lend copies to the general public via the medium of augmented reality so that people can place the artworks wherever they choose? The project location can be anywhere; we experimented at Lowlands (a Dutch music and arts festival with 50,000 visitors) and at the innovators’ festival, PICNIC. Participation was relatively simple: The visitor could choose an artwork from a selection of 160 masterpieces, all printed on A4 cards, scan the QR code on the card and thus activate the “ARtotheque” (art loan) layer on the Layar platform. The visitor could then choose a position for the artwork, hang it and share it with all other works in the public “ARtotheque” layer.

As the Stedelijk Museum is also known for its contemporary art projects, another utilization of AR appeared relevant: augmented reality as an artistic platform. In the ARtours project entitled “Me at the Museum Square,” ARtours experimented with this strategy by asking students from various Dutch art schools to design an augmented reality artwork to be virtually manifested on the large square adjacent to the museum. Stedelijk curators made a selection of the most promising ideas, and together with students from the University of Amsterdam and the School for Interactive Media (project Medialab), the 3D “Artworks” were realized. Besides helping the project to get a better grip on the possibilities of Layar and the practical problems AR applications pose to users (too much sunlight, battery consumption, etc.), another result of this project was that several of the created artworks reflected upon the new medium. For instance, in one work audience members could virtually augment themselves with auras in various colors, deriving from the artist’s idea that AR is, similar to auras, visible for some and not for others. Another artist placed a springboard next to the small pond on the museum square. The title of the work, “The most fun you will never have,” addressed the fact that, in augmented reality, the virtual is colliding with the real but not transforming into the real (in a material sense). It is this kind of self-reflexivity that helps us come to terms with AR’s cultural significance.

Let’s Go Inside

In the summer of 2010, the Stedelijk Museum got the old part of its building back. The renovation was almost finished and, although the additional wing was not yet ready, the museum could make a start with temporary exhibitions and public programs. For the ARtours project, this signified an interesting strategy shift to bring AR out of the streets and into the white cube.

As early as 2002 media theorist Lev Manovich claimed in The Poetics of Animated Space that, with augmented space,… museums and galleries as a whole could use their own unique asset—a physical space—to encourage the development of distinct new spatial forms of art and new spatial forms of the moving image. In this way, they can take a lead in testing out one part of the augmented space future…

Having stepped outside the picture frame into the white cube walls, floor, and the whole space, artists and curators should feel at home taking yet another step: treating this space as layers of data. This does not mean that the physical space becomes irrelevant; on the contrary, as the practice of [Janet] Cardiff … shows, it is through the interaction of the physical space and the data that some of the most amazing art of our time is being created.

The ARtours project selected for its first indoor AR(t) project artist Jan Rothuizen, known for his hand-drawn maps on paper. In the AR application, Rothuizen’s drawings are virtually appended to the spaces of the building to which they
refer. Using a smartphone you can open the tour and follow Rothuizen’s childhood memories of the museum throughout the gallery spaces. Also included are his references to the Stedelijk’s renowned history and close observations of the institution made while spending a night in the building.

The result is a layering of the real with virtual information, bringing the objective outer world of material spaces into collision with the subjective inner world of conceptual memories and storytelling: a mapping of the museum inside
the museum that echoes the psychogeographical maps produced in the 1960s by the French Situationists.

Of course, the move from outside AR to inside was not that easy, as current technology (Layar) relies on GPS to attach the virtual to the real. GPS has difficulty in distinguishing vertical levels inside a building; thus additional interfaces are needed to delineate one’s location inside the building. Since these methods of interface have not been perfected yet, we are pleased that AR providers are exploring new solutions to the problems of bringing the technology indoors. The ARtours project will experiment with these in the near future in collaboration with Fluxus artist Willem de Ridder, who is working with us on one of his “Secret Exhibitions” in AR. Moreover, we are exploring possibilities of bringing a selection of the Stedelijk Museum’s famous exhibitions back into the building by means of AR, reusing the museum archives and documentary material.

Innovation and Collaboration

Besides all these more practical and media-related strategies that readily illustrate how and why a museum might use smartphone-based augmented reality, there are more overarching reasons as well, of which “innovation of audience participation” seems the most pivotal one. For the Stedelijk Museum, this seems to fit a long-established tradition: The museum is said to be the first in the world to have created “audio tours,” in 1952. Of course, the radio broadcast technology used in that time was far from perfect, and the experience was almost identical to a conventional guided tour (for instance, people were bound by the tour’s time constraints and were not free to move around, being required to follow a linear story). However, as a specialist in the field Loïc Tallon rightly makes known, this was not the point. What mattered most was that the audio tours of 1952 were launched by the Stedelijk at the same time that the ICOM conference was held in Amsterdam that year. Consequently, the entire museum world took notice of this new development and many immediately started to develop similar systems. Therefore Tallon concluded in a 2009 Musematic blog post:

Above all, I believe that it was the innovation and potential embodied within the audio guide that best explains why the Stedelijk Museum ‘invented’ it. Whilst one could claim that what was achieved by the system could have been achieved through trained docents, this is too narrow a perspective. After all, this innovation went on to spawn what was arguably the most successful museum technology of the 2oth century, and one of the most exciting of the early 21st century.

In 2011, “innovation and potential” also seems to be the driving force for augmented reality applications. It is not about offering the most perfect technological solution and radical new user experiences. Moreover, it is often hard to define differences with respect to existing multimedia tours. However, the potential for bridging the gap between the virtual and the real world in a single visual interface is a dream shared by many and thus a great stimulus for future innovation.

Innovation can only exist through collaboration. In 1952 the Stedelijk Museum created its audio tours with the renowned Dutch enterprise Philips. At present the Stedelijk works with several technological and design partners, such as Fabrique, 7scenes/De Waag, Tabworldmedia, and Layar. Collaborations with educational partners (University of Amsterdam, Hogeschool van Amsterdam, art schools) and cultural organizations (Tate, Virtual Platform, ISEA, kennisland) also exist. These partners should not only receive full credit for the ARtours project, but should also be thanked for the innumerable innovations inside the Stedelijk organization they have triggered thus far and will continue to do so in the future: from fundamental changes in museum technology (ubiquitous Wi-Fi access) to new takes on copyright issues; from changes in media awareness and the programming of our educational and curatorial departments to new policies on the future of audio tours in the museum; and so on. For a museum reinventing itself in the 21st century, this is invaluable, and leads to the idea that a museum should always incorporate at least one innovative project like ARtours every other year.

Paratouring

Can we already draw some conclusions about the outcome of the first one and a half years of the ARtours project? Findings that may help other museums decide whether augmented reality can be their ultimate app? Insights that may fuel debate on the future of mobile technology in the museum?

Inspired by the “un-conference” concept, museum professionals at the industry conference “Museums and the Web” and elsewhere have discussed for the past couple of years the “untour,” referring to the manifold possibilities in our current 2.0/3.0 phase where mobile tours can go beyond the traditional audio tour format.

The ARtours project defines another interesting development in the usage of mobile media inside the museum: the “paratour.” The term “para” refers to the extra information that normally accompanies the core text of a publication:
the introduction, conclusion, notes and additional literature, often provided by the editor, which are collectively referred to as “paratext.” They are the discursive elements that frame the text, positioning it through an extra layer of information.

Of course, the traditional audio tour can itself be considered a “paratext,” as it frames art with an auxiliary text. However, the ARtours project indicates that innovative museum tours, like augmented reality applications, become especially significant by way of extra communication tools and additional layers of information. Significantly, the tours elicit communication among the users. In order to use an AR tour, generally one has to join forces, as not everyone possesses the appropriate smartphone, the user interface is still challenging for some, data traffic is not equivalent for all telecom providers, using the app tends to drain batteries quickly, etc. This turns the AR tour into a social event, something the Stedelijk Museum facilitates by organizing a public program and opening event every time a new project is launched. This form of “paratouring” among users exists not only in the analog world, but extends into the virtual one via social networking services like Facebook and Twitter. In addition, the ARtours project has opened the eyes of the museum to a ceaseless flow of professional “paratouring” by museum and other mobile
technology experts. The innovative mobile museum tour has an amazing, extended lifespan mediated through videos, PowerPoint presentations, lectures, Twitter feeds, blogs, conferences, roundtable discussions, expert meetings, wikis and remarkable press coverage. It may even be the case that the ARtours project has more followers on Twitter and via our blog than people who have actually experienced the ARtours themselves.

Of course one can denounce “paratouring”—or, in terms of AR, “pARatouring”—as a distraction from what the tour is really about, namely, mediating knowledge and enhancing visitor experience both inside and outside the museum. This is a risk, and we should take care that it does not obstruct the actual encounter with the museum, collection or exhibition. Still, we cherish the fact that a museum that has been in hiatus for over seven years is suddenly back in the spotlight! If this can happen in the world of mobile media, why not in other fields as well?

If we now return to the central question of this discussion—”Is augmented reality the ultimate museum app?”—we must conclude that, at first sight, it certainly is not: The technology is experimental, the user interface problematic, and we are as yet very far from the ideal future of total immersion and seamless interfaces (as visualized in movies like Minority Report).

On the other hand, we have seen that AR can be significant for museums in many ways, both outside and inside the museum, as it:

• offers interesting collisions between virtual (digitized) heritage and real (analog) spacee
• provides a new platform for artistic experimentation
• is a perfect medium for museum innovation and collaboration
• generates enormous amounts of communication, interpretation and contextualization (the so-called “paratouring”)

For the Stedelijk Museum, in its current “temporary” phase within and without its building and in the process of reinventing its institutional identity, AR has proven to be the ultimate app! For other museums, the best recommendation may be to consider all relevant strategies … and then engage in it anyway.