Museums often create great digital content that lasts only until the funding runs out or the point person moves on. How can we create digital platforms and programs that will last? This session will profile projects that have baked sustainability into their design.
Speaker(s):

Moderator: Jeff Martin, Director of Communications, Philbrook Museum of Art

Panelists:

• Steve Brady, Deputy Director for Digital Initiatives and Chief Technology Officer, The Barnes Foundation
• Douglas C. Hegley, Chief Digital Officer, The Metropolitan Museum of Art

Transcript

Jeff Martin: Thank you all for joining uh my name is Jeff Martin I am the communications director at the Philbrook Museum of Art in Tulsa and I’m also the host of the Museum Confidential podcast Republic radio we’re so happy to have you here I’d like to welcome our panelists today we have Steve Brady deputy director for digital initiatives and chief technology officer at The Barnes Foundation in Philadelphia and we have Douglas Hegley the chief digital officer for the Metropolitan Museum of Art in New York City. So glad to have you both here to talk about what we’re calling The Longevity Strategy or Designing Sustainable Digital Content so it’s a big topic. I guess I will throw this out there for whoever wants to catch it and start start us off which is why do so many good ideas flounder after that initial excitement because there are so many new things coming at us you know I very much am in the podcast world but you know different programs it seems like there’s always this big excitement and then a quick fade out. What do you think are the root causes that you see for those, whoever wants to jump in on that first. You can start Douglas.

I see we both be so polite that nobody starts.

Douglas Hegley: I’d say, Jeff, it’s a straightforward but I think the answer is multi‑varied and I’m sure Steve will have things to say here. I would say from a very surface perspective the way that we fund new initiatives in our sector it sort of sets all that up. We are very project‑oriented and the funding streams are usually temporary so you get a flash of excitement and some new idea, you manage to find funding for it that will temporarily keep it up and running then you kind of exhaust yourself and your team launching it and then you move on to other things. The funding runs out, the operating system version changes, time marches on and the thing has to be sunset or is no longer working or it would require another investment of time and money to get it working. And that funding model is not built for sustainability. What I have found, too, is when I’m negotiating with funders and begin to talk about sustainability that’s when their eyes gloss over, right, they want to donate a set amount of money, they want to see an output of money, hoping some measurement in the success one way or another but they’re not in the business of funding a 10‑year run they’re not in the business of replacement charges, kind of not what they do. Because I think that funding model comes more out of public programs and educational programs and things like you get the money, you develop it, you do it and then you move on, it’s a little different with digital, Steve, what do you think?

Steve Brady: Agree 100%. Usually when you take on like a building project, you know, there’s so much to build a building, there’s so much for the reserve that you hold on to to maintain that building for several years and eventually you can roll it into the endowment of the organization. And I think the other part of it is back to Douglas’s comment about where the funding is coming from a lot of these are extremely long time lines. A lot of times certain projects are almost aged out by the time they’re completed in a lot of ways. If you spend time for an iOS, by the time you release it the new version has broken. Not a planned but an unavoidable be obsolescence that projects don’t upfront have a sustainable revenue model or funding model or built on self‑sustaining technologies.

>> Go ahead, I’m sorry.

Douglas C. Hegley: I was going to jump on that technology comment because I very much agree there. The lack of foresight in imagining which technologies to choose in order to elevate sustainability, right, I think now at least now in this moment in digital time, if we’re using web technologies we’ve already increased the possibility of sustainability what tenfold a hundred fold whatever it may be but we have had a tendency having very small teams who are already very busy with things to farm some of these projects out to vendors. And vendors are going to get it done and make it look elegant but they’re going to get it done in the quickest way possible. Vendors are, right, it’s capitalism they’re in it to make money so what I have in my career is sort of like embarrassing trail behind me of having engaged third‑party vendors to build digital interfaces that were elegant and beautiful when they came out but in the end were very unsustainable because we don’t have the code base, we don’t have access to it, we can’t recompile it and sometimes vendors, in order to get things out by deadline will do silly things like put content into the code so now you can’t even update the content and you end up with this kind of a brick of a thing that you have to pay the vendor to do any kind of upgrades to.

So just having some simple digital strategy or digital guidelines around anything we make, what technology will we use, if you need to give it to a vendor that’s fine be very specific about what code base they’re going to use so you can absorb the need of updating the content later.

Jeff Martin: That touches on something I wanted to talk about, I’m glad you brought up the concept of third‑party vendors because I want to touch on how important it is to have buy‑in from your institution and how much of this in‑house, you know, there was a time I think and it’s less so now where it was so common for people to basically farm out so many different elements especially in that early, early web period when everything seemed so new and there was not much knowledge base amongst people who had been in the museum world for two or three decades at that time, we’re talking late ’90s, early 2000s but it seems to me for whatever you’re doing it doesn’t matter if it’s a digital tool or it doesn’t matter if it’s a podcast whatever it might be having someone or a team focused on that and having that buy‑in is one of the keys to making sure it has passion behind it, people who are making sure it’s shepherded in the right way, you know, talk about that and the personnel side of this strategy.

Steve Brady: I can start, you know, kind of, again, kind of an inflection point for a lot of organizations were closures due to COVID and how to sustain work and how to work differently. And I kind of jokingly refer to us as being in a gold did I lock association at the Barnes where we’re just big enough but we’re not too large so it allows us to get things done that smaller humans would have really issues just resourcing but it also prevents, you know, when we chat with larger organizations, you know, departments are big enough to have a very ‑‑ a lot of velocity in their own direction. And so COVID and online work and hybrid work really kind of, for us, flattened the organizational structure in a way that everyone was in the same size box at the table, things were a lot more ‑‑ and we all understood kind of the severity of the situation that we were in.

So I think what ended up happening there is not only getting buy‑in but ongoing team work across ‑‑ in interdepartmental groups that, you know, we’ve had working groups that formed right in March and April of 2020 that are still going strong today and have added in more stakeholders occasionally in a way that makes sure that we don’t design by committee but that we’re all in agreement that the long‑term goal that we’re all looking at is something that we can all participate and contribute to to achieve. I want to give time to Douglas but, you know, content and technology and like there’s so many different groups that need to contribute to an overall successful project at an institutional scale or even a partial institutional scale that it needs to be an ongoing thing and not just this is our project and do you give us a thumbsup do you buy it ‑‑ you know, do we have your buy‑in or do you want to champion it for your department but really how do we continue to keep this narrative going throughout the project.

Douglas C. Hegley: I love if, actually, I’m a big fan of sort of carefully configured cross functional teams, right, and a couple things about that. One, A, you use a cross functional team when you need cross functional perspectives to bring a thing to life and make it great so you bring those people together and that’s their remit. Two you have to keep that team small. If you’re in the goldilocks zone I’m on Jupiter I have a lot of gravity and it’s heavy and hard to change but we’re working on it, right, but on that cross functional team it can’t be a committee, right, because teams get things done and committees where were work goes to die, right? So committees maybe for oversight, committees maybe for vetting but teams ‑‑ and to me team I think Jeff Bezos said it, right, is the two‑pizza rule if you can’t go out for lunch order two pizzas and leave one slice on the table and search full it’s not done it’s a committee if you need 10 pizzas, so keep it tie tight. You mentioned COVID and two things in my mind, right, whenever we tech people reference COVID we often talk about the acceleration of digital or the capacity for online meetings and the change to workflows. I never want to lose sight of the fact that it was a deadly pandemic, it’s still with us, millions of people died, it’s a tragedy, right, it’s miraculous how quickly we got our vaccinations, it’s a tragedy how divided this made our society. I want to put that in its box and just acknowledge it but also acknowledge that this idea of accelerating the way we work, the way that we can quickly access one another, the way that we can track project progress using some much more effective digital tools are actually positives and those are things to hang onto a little bit and to acknowledge, Steve, I’m completely agreeing with you on that.

The other thing that I was thinking about, and this is that as we have dealt with trying to build these teams and trying to find a way to work more collaboratively, one thing that we’ve certainly observed and I’ve observed is that roles and responsibilities tend to get broken or blurred or not carefully delineated. And so I’m a big fan of the racy matrix, actually I prefer racy O, I’ll talk about that in a minute, the racy matrix has a way to assigning roles and responsibilities for an initiative, the always out of the loop, right, and I think our organizations no one ever wants to be O which is a little sad, right, I shouldn’t weigh in on guard uniforms that’s not in my role see I should be in the O category but everyone wants to be C or R in some way but to be clear about it and then if you’re thinking about the thing around a RACY matrix you should have several line items and getting back to the topic of this particular session one of those line items should be a RACY for maintenance and sustainability, right, who is responsible for that, who is accountable for that how are you making sure that there’s a tension and sort of serious commitment to making something sustainable and that there’s a person or a team that’s responsible for ensuring that.

Jeff Martin: I want to get back to what Steve was talking about about the size and scope of organizations and how that does impact your opportunities or your maybe things that are challenges too.

Douglas, when you think about your organization, the work you all do ends up oftentimes being very influential to the industry and then ‑‑ has at times set models. At the same time being of that size and having that many eyeballs on you does it make it harder to experiment and fail because what you’re doing needs to be at a certain level whereas we can kind of play around when we’re at these midsized or smaller museums and if it’s great people love it and if it’s not no one notices in the first place so we can kind of tuck our heads and try again.

Douglas C. Hegley: I ‑‑ it rests son natures with me what you’re talking about. This fear of failure is a fascinating concept across the sector, not only true at the Met it’s across the sector and I think it comes from a perfectionistic mind‑set. Somehow if it’s not perfect, if it’s not going to impress my colleagues, if I haven’t vetted it through five different committees or people in authority above me, I’m at risk, I’m somehow it creates a lot of anxiety in me. And I’m actually attending a conference on innovation and the very first keynote speaker was talking about risking failure and the capacity to learn through failure and when we say failure, right, to be very clear, we’re not idiots, everyone in the sector’s very smart, we make educated guesses, we make hypotheses, we should run pilot programs and we should pay attention to the results. Failure doesn’t mean the organization goes upside down and goes bankrupt, it means we thought this interface would work well with this audience type, we tried it, it didn’t work so now you have the capacity to learn, why didn’t it work, what else can we do, what else can we try? That is the idea of sort of failing forward, right? I think what I do I bet you Steve does the same thing, I don’t use the word failure I’m talking to senior management I’m talking to the board I don’t want say we’re going to fail forward now everyone’s ears are pricked in a way it’s not fail what we’re talking about is we’re learning, right, we’re trying things and we’re learning. Whatever we find that is proving to be successful we’ll do more of that and when we find things that are proving to be less successful, we probably shift or pivot or change that. And that somehow offense the blow for people. If you have the expectation that we will try things, some will work, some won’t be as successful, that’s OK, we can pivot, we can learn, they will be mutually informative. Steve, are you careful with words?

Steve Brady: Absolutely. I think we ‑‑ when we talk about, you know, failure of projects or of ideas we still do and we used to talk a lot about piloting projects but no one wants to have a hundred pilots concurrently running that never end so you have to kind of pick your winners and losers we have been saying this for years luckily pre‑COVID everything here is a learning opportunity it’s just another point of data. Because no one takes their darts in their hands and throws them at the board the first time and hits bulls eyes every time and things have to evolve. I think we’ve also leaned on iterative processes and agile processes and agile kind of thoughts kind of thinking. A lot of ways Douglas mentioned the two‑pizza role which is an Amazon leadership principle and we’ve done the same thing especially over the last few years and as our relationship with AWS has deepened to really look at being customer‑obsessed and where and you don’t have to basically take somebody else’s Bible and make it your own but you can see that there’s a lot to be gained through that as we evolved as a platform in an agile way we looked at the feedback kind of what they’re telling us but more importantly what we can observe them doing because that’s more critical, you have that bias, that observer effect that happens that a person that answers the survey always wants to please the surveyor in some way. Great inflation and all that but if you watch what they’re doing and experiencing and have the opportunity to build that into your structure of how you move forward with things it’s really valuable and really useful to kind of take that customer obsession principle of Amazon and apply it to your visitors whether they’re physical or virtual or otherwise.

Jeff Martin: One of the areas that I think has been really challenging for the industry as large because as you travel around and you see this, too, when you go to museums it’s always different which means there’s not a set standard for it and that’s touring mechanisms, right, there was a time say 30 years ago where most larger scale museums had basically the same thing which was you’d go in and they had this thing you’d hold out, put on the headphones and listen to those old school audio tours and there had been kind of an adopted norm in terms of the form of that kind of thing.

Since the advent of smartphones and everything else we’ve tried a zillion different things and every museum seems like they have a different take on it now it’s QR codes for this place, for someplace they’re using Oncell some places they have an app within their proprietary organization, it’s always different. Why do you think that that ‑‑ because that really is such a core element ‑‑ when people think of museums an audio tour or something like that is one of the things they think of. Why has that been such a difficult nut to crack for kind of a best practices standpoint?

Douglas C. Hegley: You know, I have a sort of perspective on it and let me try it out often you and we can poke holds in it. I think I’m going to oversimplify it by saying historically people would partake in a museum experience. It was a sort of an assumption that I’m a kind of empty vessel and a museum has all kinds of important information that is going to pour into me and somehow make me a better person, it’s sort of a rather passive experience and I wait to be told and. And instead of partaking modern audiences are interested in taking part and I think that means more actively deciding what their experience goal might be, whether or not they even have a learning goal. I think modern audiences, it’s relatively rare that people go to museums by themselves so they’re with other people who they would like to socialize with during the experience and putting on headphones and immersing yourself into an audio tour is not the way to do that. Statistics across the sector here I think vary a bit. I think what I’ve seen in the organizations I’ve worked is that uptake ‑‑ uptake was never fantastic but as time has marched forward uptake has gone down even a little more, maybe some of what you’re saying, Jeff, is there’s some confusion over what the interface is or what to expect but even when uptake is OK dwell time is pretty short so what that tells me is that people might be attracted to say, oh, that might be understanding let me try it but after a minute or two the decision is this isn’t really for me, I can’t talk to someone else, it’s not moving at the pace that I want it to move, it’s not the story I want to hear, people want to be more active in sort of creating their own experiences in interacting with the institution so we are at a place right now where we are piloting a number of different approaches to audio. I don’t think audio is dead. I think what’s different is the two headphones, march from work to work for 40 minutes and be told a story. I don’t know that modern audiences are going to engage with that paradigm anymore. But the fact, you know, the idea of augmenting a visit with some kind of audio interventions so, for example, I think we’re going to test out the idea of maybe we create a short 10‑minute podcast‑y kind of thing and kind of encourage people to listen to it before they arrive that it would set a frame or a set of lenses before you observe an exhibition or gallery experience or maybe we have a longer form podcast that we then share a much shorter versions available while you’re in the galleries maybe with a QR code activation that kind of thing, what’s happening in that audio, are you being lectured to are you hearing a dialogue maybe an argument or debate that would encourage you to turn to the people you’re with and say, hey, people are different with the way they feel about this part of the human experience so I’ll stop talking now but I think we’re at an inflection point with audio in my perspective.

Jeff Martin: Steve, what are your takes on that, in the time you’ve been there have you seen multiple variants of trying new things or pretty much the same system?

Steve Brady: We switched from a traditional audio tour. We used observation to really see what Douglas just described. People come, you know, and we went right to the data. We’re very data‑driven in a lot of what we are looking at. We can see our ticketing system we can see that 85‑95% of people come in groups of two or more, very small groups, you know, a very small amount of ticket purchasers come by themselves and what we saw was we had a master works tour with like over a hundred stops and within 15 minutes of being in the gallery one ear was off within 25 minutes both ears were off and the audio tour is done its effectiveness as Douglas has said, you know, that was the dwell time, it was over, it was never going back on their ears. So we were also, you know, one thing I like to recommend to other institutions when they say how do we do this or how do we look at these problems is, you know, use your constraints as guiding principles and they will lead you to ‑‑ if you can’t have a one‑size‑fits‑all solution and you have unique constraints those constraints will lead you to the best outcome. So at The Barnes Foundation we don’t have any didactics on the wall there are no ‑‑ other than a couple of maybe a little plate at the bottom of a painting frame that says the artist and some of them are actually factually incorrect but they’re not changing we had a system that A solved the problem that we saw with observation of peek abandoning audio tours but B could take all thousand works that we have on display and, again, our constraint which is a blessing in this case is that our collection never changes our permanent collection only comes down for conservation before it rotates, we knew if we did make a one‑time solution for how to identify the artworks it would be one that served us well in the long run.

So we basically have something where we show the tombstone information by you just point, you know, through a web app as Douglas said earlier, you point at an object on the wall, it identifies it wherein a few seconds, we gave it that elevator button rule of no more than just a few seconds of waiting and it gives you the tombstone information and most of them allow give you a short paragraph and that short paragraph is informational but it’s also intended as a conversation starter it’s something where you can say hmm, and turn to the person next to you and say here’s what I think about that and you can create that social experience as I think Douglas also made a reference earlier to human flourishing and positive psychology. You know, art experiences and especially the shared aspect of art experiences are critical to that social/emotional learning, to that well‑being, the value that museums provide to others. And so, it’s a bit of a meandering answer but in a lot of ways I think there’s a lot of alignment on those viewpoints.

Douglas C. Hegley: Can I jump in?

Jeff Martin: Yeah, go ahead.

Douglas C. Hegley: First of all, hear, hear on well‑being and human flourishing I think it’s so vitally important my own academic background is in psychology and I think the overlap between the field of positive psychology and what we’re trying to actually accomplish in museums which is impact, right, as having some change in the people who experience our content, I think that’s really important.

I’m going to try to tie this back to sustainability a little bit too. Barns foundation is unique, you’re not collecting and you’re generally I don’t think loaning objects out, right, so to create an audio experience or interpretive experience it probably doesn’t need to evolve too much over some reasonable period of time, a few years. At the Met we’re constantly changing things, we’re still collecting, we’re moving works of art, ascend ‑‑ sending works of art on a loan so when we’re creating audio tours sustainability is an issue. When we create an audio we want to create it in 11 languages, that’s expensive, it’s also hard to maintain, doing a content update is not a simple task, right?

Sustainability, very interesting around audio because it’s not a building and leave it alone, perspectives change over time. Words and semantics that were used ten years ago are no longer culturally appropriate, are no longer even culturally sensitive, the way our audio interpretation talked about native Americans or aboriginals or artists from non‑western cultures, some of the language is not really appropriate anymore. It has to be changed. And it ends up being a ton of work. So that’s why I’m saying I think the future of audio has to be different in some ways. The relevant recording hours and hours and hours of stuff in English and then kind of expecting it to last forever I think either a shorter shelf life or somehow, you’re creating content that is packaged as of a place in time so that when people are experiencing it they understand this was a perspective from November of 2023, right, and your mileage may vary.

Jeff Martin: No, that’s a great point, the idea of kind of trapping it in amber for a moment and creating context for when it was done.

Steve Brady: I’d like to jump in, Jeff, sorry. One thing to kind of build back and go to sustainability as well is that Barnes focus experience that we built and is actually open sourced, very unfortunately very few organizations have again the capability of taking up that offer but we’re always happy if anyone wants to reach out to us but we had a recent exhibition which was massive, it had an extremely large kind of amount of research and that was done around it and so we wanted to make that part of the onboarding experience for our guests. And we had considered what would have been a quarter million-dollar VR project that would have literally only lasted the length of the exhibition itself. It really wouldn’t have had any value as an evergreen experience. So, the other thing, too, is not only start with your constraints but start with what you have and think incrementally. What do you currently have and what can we add to it in a marginal way where the costs are very low but then you have the same maintainability that you had for the first project. So for probably less than 10% of that budget we were able to in a short time frame pivot ourselves and basically make a kind of an extension of Barnes’ focus which felt ‑‑ it was the same interface but allowed us to have the works in any rotating exhibitions, you know, be scannable as well as have content that we were much more freely able to work with as opposed to just pulling from the database the way we were.

So in a lot of ways that, you know, creating that aspect or that feature kind of gave us, again, extended life on the main platform and it allowed us to connect works that were in our permanent collection by (saying name) that weren’t able to move out of the galleries into the exhibition space so you could get the additional content on there and now we’ve already used that technology for three additional exhibitions that had gone on so to Douglas’s point, yeah, we saw ‑‑ we looked at our constraints initially and said this is our permanent collection and then we said we need something for the rotating collection and then we became a lot more like the general use case of museums where things move in, things move out and how do we make content around that and the content management system is very graphical, very simple and so our researchers and our interpreters are able to put that text in on their own time, they write their article and as Douglas said instead of turning over to a company with a professional voice actor and the cost and turnaround time associated with that, within 24 hours is shows up as available for scanning so, yeah, again, work to constraints that you have and then take what you already have built and see how to build on top of it incrementally.

Jeff Martin: I want to get to a few questions, and I want to encourage anyone watching, please go ahead and drop in your questions and we’ll try to get to some throughout the conversation. Kind of a big picture question from Kirsten who asked what are some tips for creating sustainable digital content I would say from my perspective and my institution is kind of three things it’s make it easy, make it fun, make it worth your time in a lot of ways and that’s very much like we’re competing with everything now, right, a lot of people think well, what are you competing with in the past, with other arts organizations, other museums, other institution. We kind of think we’re competing against Netflix as much as anything else, we’re competing against YouTube, we’re competing against TikTok, we’re competing against anything that’s going to take your time so when you’re at our museum and we’re asking you to spend a minute, 30 seconds, five minutes, whatever it is on a piece of content or whether it’s on social media or whatever it is, that’s the battle we’re fighting, right, so by making it interesting, fun, making it worth your time it doesn’t have to be fun in the sense that the content needs to be light, that’s where, you know, you can talk about very serious in‑depth things but it needs to be consumable and it needs to be easy for them to do, right? If there’s a barrier technologically, if there’s a barrier perception‑wise like it’s too long or I don’t like just the first two seconds of something they’re making a judgment call on that stuff and then they’re opting out, so that’s kind of our space and I’ll open that up to anybody just to jump in on this big picture question.

Douglas C. Hegley: I think what you’re saying is all true. What I might add to that I think there is a need for longer form content. Someone in the chat said just tell a story and that’s right like tell a story some stories have nuance, they require multiple chapters, right, so the thing about making sustainable digital content is, first of all, have a plan, right, don’t just go, have a plan what are we doing. Digital content works best when it’s modular when it can be consumed in smaller pieces. You can tell a story in several chapters. You don’t need to have a 15‑minute unending sort of stream of content coming out.

Simple rules, always make your content platform agnostic, right, do not tie content production into particular platforms, particular code bases, particular interfaces, make content, make content in standard formats that can be shared across multiple channels and then go find the places where it’s best shared when we talk about things that are more evergreen it’s tough, right, it’s hard to predict what the future will look like. For us, I think, strategically we want to spend a little more time instead of like being on the constant treadmill of creating content for rotating exhibitions we want to spend a little bit more time with our permanent collection and think about what are the stories in that permanent collection what are the stories we can tell that are relevant to this collection or cross collection stories that will remain relevant for a long time, like the history of how the Met collected these things or how they’re displayed in a certain way or conservation or science work that’s going on, those stories have really lasting power.

Lastly, we have found the content that we make that’s specifically for families and children has a much longer engagement life‑span than the kept we’re making that’s more aimed towards expert adults so kid‑oriented content seems to be useful for a much longer time.

Steve Brady: And I would say when you take a look of both the content and the technology that conveys that content, always make sure that it’s easy for that system to ingest. Douglas had a great example early on when you have contractors sometimes you end up finding content in the code and then you’re, like, well, unless I have a developer at a developer’s hourly rate to edit text that text is never going to be able to change.

So, I think again think about, you know, the long‑term value of any given project and, you know, it was probably one of the smartest things we could have done was to stay away from the flashy quarter million dollar, you know, three‑minute VR experience that would have only had a three‑‑month life‑span. So I think again, it’s really important to be able to create ongoing content which extends the life of the platform and to ‑‑ for the platform to be able to ingest that as easily as possible and whether that’s YouTube or Vimeo as your distribution system that holds your videos and it can be simple off‑the‑shelf things that have minimal or no cost you shouldn’t be reinventing YouTube just to serve some videos or to your visitors or guests.

Jeff Martin: A question from Robin and this touched on something I wanted to talk about too the question is how quickly does technology become obsolete and this factors into a much bigger question it might be the question of the session which is is the idea of sustainability and digital content kind of mutually exclusive because we live in a world where everything’s changing so quickly, you know, everything is supposed to be new, everyone wants something new, whatever’s ‑‑ it’s just the ‑‑ you know, can we even plan for the future? So, talk about two of those things, how quickly do things become obsolete and how do we think about sustainability in a moment where every year every six months it seems like there’s something new?

Douglas C. Hegley: I wish there was an easy answer. I think I’m going to separate again the content from the technology, right? I love my latest iPhone it will be obsolete in a couple years, right, and there will be something bigger and better and different in some kind of way so I want to be really careful that we’re not tying a lot of funding or effort into building something that only works on the current version of an operating system that’s vitally important, right, so relying on the technology is very risky.

We have time‑based media artworks in our collection that date back into the 1980s, we hire outside companies and internal personal r person to think about concentration of those things because they were built on proprietary technologies that are no longer effort. That’s a whole effort that I’m not an expert on and won’t try to speak to. Saw in the comments talking about Cleveland and their art lens gallery I’m not going to speak to Cleveland, I don’t work there, Jane Alexander is a good friend of mine they have a strategy of how they fund it and look at it. In terms of planned obsolescence this is a challenge, right, I think I’m trying to work with my team to think about product life cycle management, right, so a lot of times when we’re thinking about product management we’re thinking about just getting things out of the door, what is my project, what are the outputs go‑go go what’s the next project but can we think about project life cycle, right, so if we’re going to do a product, an interface of some kind is it ‑‑ first of all, is it an actual pilot we talked about pilots before I had a great conversation last week someone said well, it’s a pilot I said, actually, it’s not it’s going straight into production that’s not a pilot. A pilot would be a thing we’re trying out, we have some metrics we’re going to pay attention to, if there’s a time frame during which we’re going to run this pilot, check the metrics thing something will change. Possibly it will go straight into production if it was ultimately wildly successful. More than likely some new version of it is what should go into production. We have a tendency as a sec to call something a pilot and put it into production. That’s one of our challenges with sustainability pilots aren’t built to be sustained. I lost my train a little bit. But this idea of not ‑‑ you can’t assume that the technology is not going to change. That would be silly. All of us know that.

So, by building content that isn’t so dependent on the tech you protect yourself a little bit. Steve probably has some thoughts.

Steve Brady: I might answer it a little bit more from the technical side as well. I think Douglas said it at the very beginning and then alluded to it in his last answer. You know, web‑based technologies a Web page built in 1995 won’t look flashy or great but will still function because every advancement with HTML and HTTP have been backwards compatible and will be for a long time. If there’s a functionality even things like augmented reality there are libraries and services out there that will do that through a browser now, you’re still relying on a third so you have to kind of weigh the risks in that, like we ‑‑ the image recognition engine that we use that was, you know, as a service from a third party, they just ‑‑ they said in four months they said we are going into a different market and we don’t want your business anymore, we really like you but it’s not where our future growth is after an acquisition. We had to kind of like leg o bricks pull out that module and look at a different one so look at modularity and not monolithic solutions, I need to plug this in make sure you can plug it in easily and replace it with another component. I think, again, going back to sustainability shameless plug, there’s a link in the handout section if anyone wants to check later in the day after all this great programming ends but our visual experience platform was one where we said, you know, we did really well with text content and Barnes’ focus for an in gallery experience but we wanted to look at the most sustainable way the most value per dollar spent in creating content for online education. Whether asynchronous or asynchronous. A lot of ways that came to the procedure font, zoom in on a contrast or rotate a 3‑D object even though the video is essentially the three‑of‑us, 720p quality good value, not CD quality or studio quality but if you make, go back to the idea of the primacy of the object and you make that the center of attention of these platforms and the platforms are web‑based, then it just becomes a long‑term sustainability model around generating new content for that and, you know, our model also has a revenue stream associated with that. So, every time we have a class that’s four weeks long we get $200 per registrant in that class. And we let them keep that kept kind of, you know, as long as we can in almost perpetuity because storage is cheap and creating new content can be expensive if the mechanisms aren’t there to make it cost‑effective.

Jeff Martin: I want to talk for a second about a couple of things. We have a comment, this is a comment from Kelly who says someone said in a presentation yesterday if people knew a typewriter would become obsolete mean we shouldn’t use it in the first place that’s a good comment, but it makes me think of a couple things. When you go to Target you see a huge selection of LP vinyl records and it makes you think I thought those were gone and now they’re back and Target isn’t selling them to be cool there’s an obvious market. Makes me think about we often use technology but sometimes the best answer might be something that’s more traditional, it doesn’t always have to be that so sometimes a sign on the wall might make the most sense.

And also, so many museums have gotten into engaging people’s personal devices in one way or another, quality comes into play at some point where there’s an assumption that visitors all have certain access to certain things. And if someone on the lower income range does not have access to a smartphone or certain technologies their experience will be lessened compared to other visitors and so just kind of talk about that and how we take those things into account.

Douglas C. Hegley: Well, one of the strategies that I’ve tried to apply in my career is to say, you know, let’s make it an exception case that any ability to enjoy this museum visit is dependent upon a technology. Particularly one that would be sort of personally activated, right? So this is different when you say immersive sound or projection or something everyone with experience that’s great and wonderful but are we actually ‑‑ we can sit in a room a year or two before we’re going to do an experience and I will be the person who will say really, really, someone has to have an iPhone of the latest version or they can’t have the full experience of this installation is that really where we want to go? And if the answer somehow ends up being yes, then the next question well, where are we going to buy a few iPhones so we have some to hand out to those who don’t have one or people’s whose battery is dead or people who don’t want to use their own device because they don’t understand what the roaming charges may be et cetera, et cetera. From my perspective we’re an art museum, maybe if we were some kind technology‑based museum or experience center and on the ticket it says warning, this is lousy unless you have an iPhone, I don’t know, I don’t want ‑‑ just because a thing can be digital doesn’t mean it should be digital. How do we make sure that we’re elevating ‑‑ you talked about customer obsess sieve, Steve, how do we make sure we’re elevating the audience and by audience I don’t mean only on‑site visitors, right, we can’t lose of the fact that online visitors elevate online visitors, how do we seek their motivations, make them happy, if we ask those questions first we make fewer mistakes about assumptions about Jeff things you’re referencing I have an iPhone, my kid has an iPhone, and my wife has an iPhone, so everybody has an iPhone, that’s bias, blind spot, we can’t approach public engagement and apply our bias as our business model.

Jeff Martin: I’d agree.

Steve Brady: I’d agree. Sometimes you can go back to kind of the first principles thinking where Elon Musk has said the best part is no part the west process is no process. You want something and be inspired by the imagineers down at Disney. You want something that doesn’t put the technology first, the technology is even at best a window into something else and you don’t want people to hold that technology again unless it’s a technology music to hold that in their hands and to have that be the center of their visit. We have solved that problem with technology in gallery with devices and have never been asked for but we just have them there just in case and online, you know, access is important to us. So, you know, we give out very liberally scholarships to online platforms because again the nice thing about when you build something digital, each additional person to experience it is almost a negligible cost, almost zero. So, you know, like 10% of our students in a lot of programs when you look at our scholarship numbers online, our scholarship recipients and they either apply or, again, tie it into what you already have, we have a community pass program so anyone with like a social benefit card becomes kind of a member of their own level at the Barnes Foundation and then by using the same technology we built for our traditional membership system those folks can go in and get free enrollment in any class we have online instantaneously they don’t have a gatekeeper there’s no one they have to talk to it’s just part of their benefits package. And we built ‑‑ always inspect your technology so it works on a three‑year‑old Chromebook at a school. Always build things so that if it has to use techology it’s very accepting of older technology, slower connection and it meets all of your audience or as large an audience as possible and always make sure accessibility is, captioning for audio things like that. We have ‑‑ last point: We have acceptance criteria for every feature, every user story that we use to manage our products. It doesn’t get approved if it doesn’t have accessibility checkoff. So, you know, if we put an audio feature in it has to have some form of captioning and if we put in navigation with a mouse there has to be some form of keyboard navigation so always be sure you’re building for the widest possible audience.

Jeff Martin: There’s so much to talk about, we obviously just scratched the surface but thank you to Steve and Douglas for their expertise and passion. Thank you to our behind‑the‑scenes folks Grace and Barbara for making this possible. Thank you all for watching and have a great rest of the conference, thank you very much.