Remember when you looked at a painting, listened to music, tasted your food, smelled perfume and touched a (real, physical) object? The rich messiness of the five human senses has always been hard to record in tangible media, but now, thanks to emerging technologies, sensory impressions can be captured, mixed and presented in new ways to enrich and refresh traditional cultural experiences. Emerging technologies hold out the promise of recording and remembering scent as easily as we snap a picture, and the ability to (virtually) touch anything rendered as digital data. And as the technology for capturing and (re)combining sensory experiences becomes more common and more effective, people may become less interested in traditional experiences that appeal primarily to one sense at a time.

The demand for multisensory experiences is accelerated by discoveries documenting the utility as well as the artistic challenge and the sheer fun of engaging all the senses. Recent research suggests the best way to remember facts may be to set them to music, while the ancients long ago discovered how to use our innate spatial sense as a memory tool. One reason so much time and effort is being spent on developing scent technology is a growing understanding of the power of smell to stimulate memory, engage emotions, and affect mood and health. Some multisensory experiences are already being used to meet the needs of special audiences, as with the Memories in the Museum project for people with Alzheimer’s, which inspired the Cincinnati Art Museum to take a “holistic approach, incorporating sound and other sensations rather than exclusively visual” to better serve its audiences.

But much of the demand for multisensory experiences is driven by pure pleasure. We can wear multisensory fashion, such as designer Iris Van Herpen’s costumes that generate their own “embossed sound.” We can stroll the city enjoying multisensory public art, such as Di Mainstone’s project that wired the Brooklyn Bridge as a giant harp. Multisensory dining experiences are a hot thing: Ultraviolet, in  Shanghai, combines food with moving images projected on the walls of the dining room, music, and aromas (other than that of the food—the scent of the ocean for a lobster course, for example). In Barcelona, Ferran Adria’s 41 Degrees “plunges guests into an art installation… exploring how combined sensory stimuli create powerful, emotional and memorable ties between consumers and restaurants.” Hotels are expanding their sensory experiences, from the Nordic Light, with rooms featuring individually programmable lights chosen to create calm, relaxation, romance or whatever mood guests desire, to the 21c Museum Hotels inviting guests to “bed down with art,” including video, sound, light, digital and text installations.

The digital revolution was, at first, aural and then largely visual. But we are rapidly developing digital technologies that channel the other senses as well. Digital scent technologies are especially promising. The Smelling Screen makes specific objects on an LCD screen release particular odors at pre-set intervals. The Ophone, currently in development, takes smells mobile, enabling users to “text” any of 320 different scents, creating personalized aromas that are produced by a cylinder synched to a smartphone via Bluetooth. Inventors are even tackling the fleeting nature of scent: an analog camera dubbed the Madeleine captures smells, which can then be sent off for analysis, supporting the creation of “bespoke memory capsules.” Smell isn’t the only sense making a new leap into the digital realm, either: researchers at the National University of Singapore are working on a “digital lollipop” that can simulate taste.

Skip over related stories to continue reading article

Haptic technology explores the realm of digital touch. AIREAL uses puffs of air to create tactile sensations in midair, corresponding to a user’s body gestures in front of a screen (playing goalie, if you go to “block” that digital soccer ball,you actually feel it hitting your hand!) Another gizmo uses electro-vibrations to simulate the feel of textured surfaces as rendered on a touchscreen via digital data—letting you feel what you are looking at on your screen. MIT Media Lab’s inFORM allows users to “reach through” a computer screen to manipulate a real object in a remote location. Researchers at UC San Diego are even working on devices that would digitally record touch in much the same way we record sound. Their work was initiated in response to a request from the UCSD School of Medicine’s neonatology group, which wanted to record the sensations of a mother holding her baby in order to “play it back” to premature infants confined to incubators.

We don’t need haptics or “scent texts” to create multisensory experiences—sometimes it’s enough to create immersive experiences that co-opt the sensory landscape where they take place. This is the case with immersive film” such as The Alter Bahnhof Video Walk (2012) designed for an old train station in Kassel, Germany, which creates an immersive “physical cinema” environment. Participants check out an iPod and headphones and cue up an alternate reality app that overlays events playing out on the small screen onto the location where the scenes were shot. Or even more low-tech, in the Invisible Cities opera performed at Los Angeles’ Union Station, performers move throughout the courtyards, waiting rooms and ticket halls of the station. This presentation combines sight, sound, movement and audience participation for a heightened emotional impact; it appealed to younger audiences that do not typically attend operas in concert halls.

What This Means for Society

The psychology of scent has implications for architecture and urban planning: “medicinal urbanism” could expand the design palette to include scents to improve mood and promote good civic behavior. (One study from the Netherlands shows that people exposed to citrus-scented cleaning fluids were unconsciously prompted to keep their environment cleaner.)

Multisensory experiences can be more accessible to people with one or more sensory limitations. Haptics can enhance the experience of people with compromised vision. (Using haptic devices can even rewire the visual centers of the brain to process touch input.) Sound can complement or replace visual cues.

Commercial marketing firms, always poised to exploit technologies that create compelling and addictive experiences, are finding that if a picture is worth a million words, all five senses can be worth a million bucks. Brands are commissioning well-rounded sensory identities for their products—not only obvious candidates like food, drink and beauty products, but even travel and insurance. As marketing becomes more multidimensional, it may shape consumers in either of two ways: we may learn to “tune out” this sensory assault, just as we learn to ignore sidebar ads in our browsers, or our attention may become “retuned” to expect multisensory cues.

What This Means for Museums

Heritage is increasingly being defined as multisensory as well. The Fontoteca National (National Sound Archive) of Mexico, for example, declared the sounds of Mexico City to be part of the nation’s cultural patrimony. Many countries are identifying and controlling the names associated with traditional foods, such as France’s appellation d’origine contrôlée, or Italy’s denominazione di origine controllata. People are starting to recognize “smellscapes” as important and distinctive characteristics of cities that deserve to be mapped and preserved. As keepers of cultural heritage, museums may need to expand the boundaries of what they choose to preserve as well.

Growing demand for multisensory experiences may pressure museums to routinely use more modalities in their exhibits and programs. MoMA even invited Prairie Home Companion’s Garrison Keillor to lead art lovers in a serenade to works in current exhibitions, and indeed, New York Times culture critic Robin Pogrebin recently proclaimed music to be the “next big thing in museums…What you see in…exhibitions is a real coalescing of art forms—music, painting, sometimes a live element, video. It’s all a big mix now.”

While museums have long used relatively lowtech ways of incorporating scent into exhibits (“lift and sniff” panels; pumping scents into the air of walk-through dioramas), touch (via demonstration carts), sometimes taste and, as noted above, sound, the technologies reviewed in this essay provide more creative, seamless, personalized ways of creating multisensory museums. The availability of these techniques, along with consumer preferences for multisensory experiences, might reset the baseline expectation for museums, from providers of primarily visual experiences, occasionally embellished with scent, sound or taste, to experts in synesthesia.

Museums Might Want to…

Routinely use multisensory design to provide accessible experiences for people with disabilities, such as the Met’s “multisensory stations” that invite all visitors to experience the exhibits though scent, touch, music and verbal imaging. Or like the Brooklyn Museum’s sensory tours, which include 3D printed replicas to create touchable experiences for visually impaired visitors—a low-cost method of making hands-on models of any object that can be digitally scanned.

Use the museum’s building as a setting for immersive experiences that are inherently multisensory. See, for example, the Sheldon Museum of Art’s “Naked Museum exhibit,” which celebrated the 50th anniversary of its Philip Johnson-designed building by removing all the art from the galleries and Great Hall and filling them with poetry, music, drama, history, dance and performance art.

Partner with other organizations that specialize in other sensory modalities. The Science Museum in London took this approach to create the “Universe of  Sound,” working with the Philharmonia Orchestra to create a high-definition, interactive, immersive experience. Giant screens distributed throughout the museum presented the 10 sections of the “virtual” orchestra performing Gustav Holst’s The Planets; visitors can step inside the heart of a symphony orchestra, taking on the role of a musician, conductor or composer. Philharmonia musicians staffed the exhibit each day, playing along live with the recording and  answering questions.

Consider their role in preserving a sensory patrimony that exceeds traditional collections boundaries. Historians and scientists are taking the idea of smells as historical artifacts seriously, questioning whether we can really understand a past time if we can’t immerse ourselves in its scent. Is it incumbent on museums to take on the role of preserving the smells of a given time and place, or the sounds? How would this change the scope of our collecting, our concept of “complete” documentation or the technologies needed for preservation?

Combine their intellectual resources and digital expertise with the new sensory reproduction technologies to provide well-rounded remote access to collections. Using the full palate of sensory tools, museums could go beyond augmented reality, and beyond just sharing artifacts via 3D digital scans. Can museums provide digital libraries of historic smells? Can an immersive virtual museum enable visitors to see, handle, smell, taste and hear a reconstructed world?

How can museums encourage artists (and others) to conduct their own synesthetic experiments as part of the Maker/remix culture, using museum resources?

Museum Examples

Museums and other cultural organizations are exploring adding taste to their sensory repertoire. The MUSEUM OF MODERN ART (MoMA) recently hosted a culinary class that reinterprets René Magritte’s art as a five-course tasting menu. The Hartford Symphony’s “Playing with Food” series challenges chefs at local restaurants to create dishes paired with specific pieces of music, while the orchestra’s conductor chooses one piece of music inspired by a regular item on a restaurant’s menu. (Selected drinks and desserts from the series have been offered for sale to concert-goers.)

Also at MoMA, rAndom International’s “Rain Room” caused quite a splash this summer, with visitors queuing early in the morning and waiting in line over five hours to spend up to 10 minutes, 10 people at a time, walking through the immersive installation in which water rains down everywhere except where sensors detect people, giving visitors the illusion of walking between the drops.

The new SHIJIA COURTYARD MUSEUM in China, which is dedicated to preserving the traditional architecture of the hutongs (alleyways) and siheyuan (courtyard homes) of Beijing, is determined to preserve the traditional sounds of this disappearing culture as well. Volunteers are creating a database of different sounds that might be heard in the old alleyways to stream at the museum: street vendors calling out their wares, pigeons cooing, local doctors announcing their arrival.

In Venice, the PALAZZO MOCENIGO is adding perfume to its traditional focus on clothing and textiles. Noting that the modern perfume industry began in Venice in the 10th century, the new galleries allow visitors to smell traditional ingredients, as well as featuring a collection of perfume flasks dating to 2000 B.C and a 1555 Venetian publication thought to be the first manual in the West that scientifically catalogued cosmetic formulas. The Art of Scent exhibit at the MUSEUM OF ART AND DESIGN in New York consisted of nearly empty galleries filled with this “invisible art form.”

The CLOISTERS museum in Manhattan chose as its first-ever presentation of contemporary art an installation of Janet Cardiff’s “The Forty Part Motet,” an 11-minute performance by the Salisbury Cathedral Choir rendered through 40 high-fidelity speakers arrayed in the museum’s Fuentidueña Chapel. One visitor described the experience as “transcendent,” and some emerged “wobbling, blissed out, a few in tears.”

The desire for immersive experiences gives museums yet another avenue for “Takin’ it to the Streets” (as we explored in TrendsWatch 2012). The MINT MUSEUM UPTOWN in Charlotte, North Carolina, incorporated live music into their marketing for their Romare Bearden “Southern Recollection” exhibit, creating living billboards that featured reproductions of Bearden’s work activated by live performances. Periodically, musicians dressed as characters in the paintings would show up and play unannounced gigs in front of the billboard. The RIJKSMUSEUM staged a recreation of The Night Watch in Amsterdam, with costumed reenactors invading a public shopping mall on horseback and swinging from the balconies.

Further Reading

Diane Ackerman, A Natural History of the Senses (Vintage, 2011). A “lusciously written grand tour of the realm of the senses.”

Rachel Herz, The Scent of Desire: Discovering Our Enigmatic Sense of Smell  (Harper Perennial, 2008). A psychological study of the importance of smell in our lives.

Liz Neely, Miriam Langer, Please Feel the Museum: The Emergence of 3D Printing and Scanning (paper from the 2013 Museums and the Web conference, digital download). This paper documents and explores how 3D printing and scanning can be used to help audiences feel the museum, starting with an introduction to 3D scanning and printing, and discussing how printed models can become “social objects” that provide access to collections for a wider and more diverse audience.

Mark M. Smith, Sensing the Past: Seeing, Hearing, Smelling, Tasting, and Touching in History (University of California Press, 2008). Smith has consulted with several museums that were interested in recreating sensory experiences from the past.