The Collaboration for Ongoing Visitor Experience Studies (COVES) is designed to unite museums in systematically collecting audience-level data with a focus on institutional and field-wide improvement. Learn about how we are engaging in critical conversation with art museums as we review and revise the instrument to serve their unique needs. For over five years, COVES has supported science centers and children’s museums through ongoing data gathering to inform decision-making and strategic planning. COVES is just beginning to expand into art museums and invites AAM members to consider how this data and system could support your institution as well as the possibility for field-wide comparison.

Transcript:

Erica Kvam:

Here we go, all right Hello and welcome to today’s CARApy webinar session. My name is Erica Kvam coming to you from Purdue University galleries in West Lafayette, Indiana. I’m just going to review a few housekeeping items before we start. Please keep muted as we are recording. Ann and I will ask you to mute if for some reason you come off mute. Please try to utilize the chat for Q&A. We’ll kind of go through those as the session continues, but we will also have time at the end for discussion. We will be posting this session to AAM’s YouTube. So if you miss anything or you have to duck out, you will be able to find that later.

This session is brought to you by the American Alliance of Museums Committee on Audience Research and Evaluation professional network. These webinars are free and we would like to encourage you to join as an AAM and CARE member. To become a member and find out more about what we do is a professional network, please visit AMM’s website at aam-us.org and with that, I will turn it over to Sarah. All set for you.

Sarah Cohn:

Thank you so much, Erica, I really like the start of coming to you live from I’m in Minneapolis in my home in my spare bedroom. Sara Cohn, they, them, she, her. I’m past chair of CARE and I’m also a member of COVES and we have a lot of CARE representation today. So any questions that you all have about the Committee on Audience Research and Evaluation, volunteering with AAM, being a host and a moderator with Ann and Erica for future CARE webinars, throw them in the chat. One of like seven of us will happily answer it for you. But I am actually just going to get out of the way and hand it over to other COVES leaders.

We’re really excited to think about bringing this conversation to AAM. We’ve had a lot of conversations within ASTC, Association of Science and Technology Centers about the COVES, which is the Collaboration for Ongoing Visitor Experience Studies. And as we move beyond science centers and children’s museums, and thinking about how can this be a benefit and something that we get all different types of museums and informal learning environments to join. We’re excited to come through CARE to the AAM audience. So we’re really excited for you all to be here with us today and I’m going to hand it over to Rachel.

Rachel Wolff:

Thank you, Sara. Thank you, Erica. Thank you, CARE. Thank you, AAM. I’m also coming from Minneapolis today. My name is Rachel Wolff. I’m the audience insight analyst at the Minneapolis Institute of Art. And I’m in that roster that Sarah mentioned as the incoming chair for CARE. So I may not be able to answer those questions in the chat, but I’m very excited to be here for both Mia or for all three of Mia, COVES, and CARE. And I’m really excited that all of you are here too. So we’ll have a few people talking today and then as Erica mentioned plenty of time for discussion today. Our presentation is about 25 minutes long to make sure that we can answer all of your questions and come back together out of the PowerPoint to have time for that discussion.

But to get to the main event here to tell you today about the Collaboration for Ongoing Visitor Experience Studies or COVES as we refer to it, and a project that has been funded by IMLS over the last year, and really led by COVES in support with Crystal Bridges and the Minneapolis Institute of Art. So Ryan, if you want to bring us to our next slide. This is it. We’re diving right in, we’re starting with the juicy part, more juicy sentence. Really why we’re here today is that we are super driven, super passionate about bringing rigorous and reportable visitor experience data to every Art Museum in the US, and to create field-wide conversations that are not currently possible. So we’ll talk through what that looks like today, and we’ll actually end with coming back to this sentence too.

But we’re really here to talk about a system that allows for this, that encourages this, and how we’re building that system in partnership with art museums and creating those field-wide conversations as we go. So Ryan, next slide. Who’s the “we,” in that sentence? It’s many people including the named participants in the grants that you see here. Sarah Cohn, as our principal consultant in Aurora Consulting, but also a member of CARE, of course. Ryan, who is also on the call and will be available for questions. And Alex will be speaking both working in a dedicated way on COVES as part of the Museum of Science, Boston team. We have two members of the Crystal Bridges team, Juli Goss, the Director of Data Strategy, and Kate Meador, the Audience Research and Evaluation Associate, who’s also on the call today and will speak for a little bit. And myself, I already introduced myself as the Audience Insight Analyst from Mia.

So this is the team that has been working on this grant over the last year. But as you’ll see in the presentation, really in partnership with lots of other wonderful thinkers, museum practitioners, and data users. So really big picture poses designed to unite museums in systematically collecting audience-level data with a focus on institutional and field-wide improvement. Some of these terms are probably already familiar from the very first slide. But it really is about having a shared survey instrument, so that allows us to do that systematic data collection around understanding visitor experience, but pairing that survey instrument with a real community of practice. So that’s that focus on institutional and field-wide improvement, to understand that we can collect that data for ourselves in our own institutions. But then we can also come together to use that data to make comparisons to have conversations about what we’re seeing among visitor experiences and to drive the field forward in that way.

Really, the big takeaway is that COVES provides a common language through data to help with decision-making. That’s the big goal for institutions that are part of COVES and for COVES itself as a community of practice. And at this point, there are just over 40 museums across the all of the United States, who are using the same survey to collect data about experience on-site and demographics of visitors and their group members. And we’ll talk a little bit more about those details. But like I mentioned, it also again comes with that common language, that community of practice, that is about bringing that data back to your own institution and growing that common language through your own work. And then also in conversation with other practitioners across many museums.

So we have a couple of slides just to talk a little bit about what COVES is really aiming to do and then some slides to talk about what that actually looks like. And COVES is really committed to better understanding visitor experiences through collecting the survey data. And we do that, we believe in better understanding visitors’ experiences because we think when you better understand what visitors are doing, how they’re feeling, what they’re thinking, that we can improve the experience overall. So here, we say that rising tide lifts all boats. And then we also think that that can drive and encourage repeat visitation among those visitors. If we really understand what that experience looks like, and we’re aiming to improve it, that means that people will be driven to continue to engage with us as institutions.

And we know that continued attention to improving the experience also does some other really key things for our institutions. It increases the likelihood of belonging, feeling like a place is paying attention to and listening to your experiences, and meeting your needs, which can translate often into things like memberships, donations, or feeling represented in that museum community. It can also lead to more positive word-of-mouth promotion. So this idea that if we’re constantly improving, monitoring, and improving the experience through actual visitors responses, that when they’re then out there in the world talking about their experiences, they’re more likely to say, “Yeah, I went to Crystal Bridges, and it was super cool. And here are the things that I did while I was there.” A really important piece of COVES that we’ll talk about in a lot more detail today, especially in Alex’s section is that COVES is by museums for museums.

The original instrument as it was created now almost eight years ago, I believe is the right number there, was really came directly out of conversations among museums about what their needs were in terms of understanding the visitor experience. And that model continues into today. It’s not an outside consultant telling you what you should study. It’s not market research, trying to apply a sort of blanket set of terminology or motivations or demographic patterns to what you’re looking at. It really is about what museums need, what they’re seeing happen among their visitors, and how you can actually measure that and make that data usable. So when thinking about what visitor data do we actually collect? What is that data that we want to make sure is usable? It falls into these four categories.

So we’re thinking about who visits, that’s that demographic piece, and we measure quite a bit of demographics and then are really active as COVES as a community of practice in terms of putting them into context with things like the census or local demographic information to your state or your region or your city. We’re also visited in, or we’re also interested in when someone is visiting, that who, what do they see and do while they’re on site. So what types of things at our institutions are they experiencing? For a place like Mia, that doesn’t just mean the art galleries or special exhibitions that we have on view. That also means, did you go to the family center? Did you visit our store today? Did you stop at the cafe? We’re asking about all of those questions to really get a fuller picture of that complete visitor experience.

We’re, of course, thinking about why they visit, what drives people to visit, what types of experiences or feelings are they looking for. And this is a part that we’re really thinking about right now, especially in relationship to art museums. What is that why that’s driving people to visit art museums? What experiences are they looking for? And then how they feel about all of that. We actually have a few different questions inside of COVES that get to this question of what type of experience did you have? How do you feel about what you did today? And what would you tell others about that experience once you’ve left? Then how do we collect that visitor data? So a little bit more of those nuts and bolts pieces.

COVES is an on site electronic survey with an exit intercept. So what that means is that it’s an in person survey with your visitors where you’re engaging with visitors as they’re exiting the building in order to provide the survey to them, ask these questions have that intercept moment. I’ll make a note here and we can certainly talk more about this in the discussion part that of course over the last year COVES has adapted to figure out, okay, in-person serving might not always be an option for institutions right now, how can we work with QR codes? How can we work with email serving? How can we do what’s in the lower right-hand corner here where we might have an iPad stand and instructions so that visitors can engage with a survey on their own without necessarily having that on-site intercept to happen?

But at its core, and at its root in terms of its methodology, that’s how COVES is structured. And of course, we can talk more about that, as I said, because part of that exit in our survey or exit intercept survey, and that methodology is really linked to systematic random sampling. And this is the piece that makes COVES rigorous and reportable in the way that we’re driving for. So it’s not just asking questions of visitors who are already really excited to answer questions, maybe they already really like you or they really want to answer questions because they had an experience that might not be so positive, and they want to tell you about it. We’re probably all familiar with that sort of version of comment card data collection.

It’s not just talking to your members. It’s not just talking to people who visit on Saturdays. It’s really making sure that you’re getting that systematic random sample across your entire visitation that allows you to extrapolate and understand that full picture. And that’s where COVES really brings a level of expertise around the methodology and around sampling to support institutions to help them understand how best to do that. And then that really ties into this final point here, which is that that sample size is proportional to your institution size. So again, thinking about how do you appropriately measure on how among your visitors whether you have 1000 visitors a week, or 100 visitors a week, or 10,000 visitors a week, maybe in some cases. So really mapping that out to make sure that you feel really confident that whatever results you’re getting back again, represent that larger visitation overall.

Yes, thank you, Ryan. I know it. I lagged my slide there for a second. So to talk a little bit about what this might look like day-to-day, I’ll share some examples of how Mia is using COVES and this visitor surveys to collect data and use it for day-to-day decision making. And then I’ll pass it to Kate to talk about an example of some longer-term decision-making with COVES from Crystal Bridges. So though we’ve sought out and planned to pilot COVES pre-pandemic, or in sort of what we might think of as typical circumstances. Of course, the last 18 months have brought a lot of atypical experiences. So my day-to-day decision-making is actually sort of rooted in how we’ve really actively used COVES as an institution and use the data that the survey is collecting as an institution to make decisions at a time that otherwise felt pretty up in the air.

We were changing a lot, we weren’t quite sure what to expect from visitors or what they were expecting from us. And we really leveraged COVES in that moment to help us make decisions really in an active way. So first and foremost, probably no surprises, who is actually coming, who’s coming back right now? Or is it mostly members? Is it mostly volunteers and docents? Is it new people? Is it families? Is it not families? All of these questions that felt like we weren’t sure what our visitation would look like in this pandemic moment, COVES answered really directly for us. And then of course, that key second piece of, are they having a good experience? So we have put lots of new policies and practices into place to try to provide a safe and healthy experience for both our visitors and our staff. And then we also tried to keep the joy and the curiosity of visiting the museum even during such a challenging time.

So we were really trying to understand how are our COVID-related policies and practices working? And are we still making sure we’re delivering a really good experience for those who are coming back? And those COVID-related policies and practices, like I previously mentioned weren’t just about getting tickets ahead of time, which was new for us, or some galleries have limited capacity in terms of how many visitors could be there at a time. It was also about all of the other things that are part of visiting an art museum. Things like sitting at the cafe, things like visiting the store, how did those things feel in this moment? How could we be adjusting again our policies and practices in a really active way, based on direct feedback that visitors were giving that we had immediately available through COVES as a platform?

We were also able to answer some really important questions, particularly for our leadership team, around are our current visitors different than prior to the pandemic? We could make these types of longitudinal, or more long-term comparisons to our previous visitor survey, to help us understand who was coming back in this moment? And then how might that change how we message to them? Can we really lean on email because people are already in our sort of our conversation or do we need to expand our messaging about the experience of the museum? Because we know that we’re welcoming back first-time visitors? And then this final thing, once we make adjustments based on all of those kind of on-the-ground data pieces, do we see a positive impact? So if we’re seeing concerns about how we’re messaging around social distancing, well, we can add more signs, we can add some more information to our website.

We can also do some more targeted messaging, either through email or on our social media channels about how we’re observing and putting into place social distancing. So then do we see positive experience scores go up in that area? Do we see less visitors leaving comments saying this was a problem, or if this area was confusing to me, so we could really again, kind of measure that on the ground and make sure that when we were making decisions, we were making them in a way that was really rooted in visitor experience? And then now with COVES, as part of this community of practice, it’s not just our data that we’re thinking about, we’re also thinking about what are other museums seeing and doing? What are they noticing? What are their visitors asking about or worried about?

How can we see ourselves in this larger constellation of museums in this moment, to understand what we’re doing well, what we could improve on, and how we can really make sure that we’re providing experiences for visitors that are positive, that are safe, that are healthy. So now I’ll pass it on to Kate to talk a little bit about Crystal Bridges and their use of COVES data.

Kate Meador:

Hi everyone, I’m Kate Meador, I’m an evaluator Crystal Bridges Museum of American Art. And like Rachel said, our museum has been dedicated to audience research data for a really long time now. And so I’m here to kind of briefly present to you a very abridged summarized version of what that’s been like, and also talk a little bit about why we are now part of COVES and what that system is offering to us that we didn’t have before. So to start, Crystal Bridges has actually been collecting audience data for over five years through an instrument like COVES. And it’s been a really invaluable resource to have for decision-making and sparking conversations about where we’re at, and also where we’re going. And this also just kind of emerged from this question of who is coming to the museum? Are they having a good experience? What do we know about our visitors?

So an instrument was put together five years ago to really answer those questions, and it was administered on-site, much the same way as COVES. But however, most of you probably know that as time changes, new questions arise, and there are new data needs. And so for example, as our leadership was interacting with the data that we were providing, it became important to also look at who’s coming to the museum compares to our read demographics. So for instance, the question was, are our visitors reflecting the broader community of Northwest Arkansas? And likewise, who are we not representing? Who is not coming to see our exhibitions? So over the time, this led to more and more specific questions and conversations. For instance, the new question that came up was, what does audience visitation look like on specific days when museums have family events, for example?

And through analysis, we were able to say yes, in fact, we know that our audiences do change on those days, when you do have those events, our demographics are broader, our visitors are more ethnically and racially diverse. But at the same time, we were still not really hitting that target of reaching all socio-economic groups, where people have varied education backgrounds. So in a way, what that instrument was able to help us with was to really confirm or refute what we’re up to that point sort of akin to intuition. And so by collecting data over the years, we’re also able to do a comparison, of how we’re doing compared to how we were doing the year before. And that really helps us tell the story about where Crystal Bridges headed.

So fast forward to now, Crystal Bridges has been part of COVES for over a year. And why would we choose to migrate to this new system when we have already had an estimate in place? And the reason for that is that it really helps us answer this key question that we would never have been able to answer before, which is how do we compare across the United States? And so, without an instrument like this, we would not be able to have these conversations, or also connect with like-minded colleagues that are working in the field, to be able to learn from one another and to be able to see, not only how do we compare, but what are other folks doing that we might learn from?

So like I said, we’ve been collecting data actually through most of the pandemic, really through a modified protocol. We were disinfecting the screen between each use. And with the data that we collected, we were really able to obtain a greater understanding of what audiences were like, even during these trying times. So if any of you have questions about what COVES looks like in practice, I’d also be happy to answer those later as well. But for now, I’ll turn it over to Alex.

Alex Lussenhop:

Thank you, Kate. And thank you, Rachel, as well. So I want to talk a little bit about what COVES is doing right now to really try to bring in art museums in a big way. So in September of 2020, Ryan and I, and the COVES team at the Museum of Science in Boston where COVES sort of originated, we along with Mia and Crystal Bridges sort of said, “Hey, this system would be really good in art museums. There’s a real need, there’s a real hunger and there’s a real… We could do some really powerful things.” IMLS, the Institute of Museum and Library Services agreed, and they gave us this grant to expand COVES into art museums. And so the plan with this grant really was to move COVES from where it started, which was really primarily serving science museums, natural history museums, Children’s Museums, to include art museums as well.

The funding was providing us with the ability to have a convening, which we had wanted to do in person in Minneapolis, but of course had to change to a series of virtual convening over the month of April, which was still really great, but we did miss out on that in-person experience. And then to really start the conversation and build up a community of practice in art museums around data, data sharing, institutional improvement, field-wide improvement, and just kind of set the stage and build the foundation for that to move forward. And I will say in addition to Crystal Bridges, and Mia, we had some awesome, awesome thinking partners on the grant, for example, the Cincinnati Art Museum. I know we have a few folks from Cinci Art on the call right now. Probably there may be others that I can’t see on the list too, but just big shout out to our grant partners. You can go next slide, Ryan.

So the convening itself, again, I said it was virtual convening of the month of April every Wednesday, we gathered together. We had 24 arts and culture institutions represented, small large all across the country. As you can see the dots on this map represent our partners. We also had representation from AAMD, the Association of Art Museum Directors. Of course the Museum of Science, we had a lot of folks there. And then the Science Museum in Minnesota, we had sort of an original COVES participant kind of joined to talk about what COVES looks like on the even longer-term scale, five, six years down the road. And so I also know that there’s some folks here on this call who were at the convening, or whose institutions are represented at the convening. So just another shout out to I think I saw Portland Art Museum folks here, Saint Louis Art Museum, LACMA, really, really great to see you all.

It was so fun to spend time with you together in April. So more about what actually happened during the convening and what’s been building since then. So that convening was really, we kind of had these four sessions, and tried to build up questions, a lot focused around the data, as well as the big picture. So really thinking about, are there experiences that are unique to art museums that we need to measure? Like what is the museum experience at an art museum like? And how do you sort of build capacity for data collection in your institution? Both doing the data collection, but also thinking about data and having sort of a culture of using data and talking about data and using it to improve the experience. So just some examples of some things we ended up talking about, like on day two, within the experience, there was a big discussion about welcoming and belonging and how do you measure that in an art museum?

We also discussed sort of the aspect of outcomes that many art museums want their visitors to experience, things like wonder or transformative experiences, like what does that look like in a data collection environment? I also I’ll speak personally as a science museum person, I learned a lot about some of the similarities and differences between what art museums and science museums might want to collect in terms of experience of demographic data. So thinking about the way our instrument was built in the first place, we talk a lot about groups, and so many groups visit science museums, but the individual experience is also a really, really big deal in art museum. Science museums also had some different interest in the amount of data we collect from children. There’s some differences there.

There’s also a lot of similarities that we learned about. I think there was some discussion about the barriers to visiting art museums and the feeling that you might need a lot of knowledge before you go in. And maybe that was a different, but I think that’s something that happens in science museums as well. So one thing that I’ve learned is that while there are differences between our types of institutions, there’s really so much that we share in common and so much that we can learn together as a museum field. And so then, kind of the next steps from the convening is one of the biggest ones is that we’ve got a smaller group of art museum folks, most of whom are currently starting the piloting process and starting to actually use the COVES instrument at their sites, who are thinking about what is actually going to be on that art museum survey instrument.

It’s probably going to be very good at covering art museums but what needs to change? What needs to be different to really represent art museums? So this list here sort of depicts who is currently collecting coves data. And then on the left side, as well as who’s kind of getting started, whether that’s just an email of interest, or really actively building up a survey or something like that, going through the training process. This cohort is incredible. It’s huge and I’ve just been so excited to see this community grow in Art Museum. So a lot of these folks on the left, and on the right were at the convening and are getting COVES built up at their institution. So you don’t forget, returning to this, still this, what do we want to do? What are we doing right now? We’re bringing rigorous and reportable visitor experience data.

We want to bring it to every art museum in the US and create field wide conversations that are not currently possible with the systems that we have. And then so moving on to the nuts and bolts of it. So for those who do want to get involved in COVES, for those folks that you saw on that slide before, what it looks like, what’s kind of involved in, it’s an annual membership model. So everybody kind of pays in, there’s a cost associated with it, but it’s sort of scaled to your institution budget. And so a smaller museum would pay less than a larger Medium, large in terms of annual budget. And the membership covers our online survey platform. That’s a huge part of it, share this big database, as well as a dynamic dashboard reporting tool that enables you to see your data in real-time, which I know during COVID was extremely, extremely important for a lot of people.

Ryan and I and the COVES team take care of instrument development and maintenance, data cleaning, the type of statistical analysis that we use for our aggregate reporting, and as well as productions of our annual aggregate report, and then dissemination of our work across the field, and growing the COVES community through things like webinars and creating online spaces where we can talk. We have historically been gathering in person at ASTC conferences in the past when we were more of a science center-based collaboration. But I would suspect that as our conferences move back to in person, we’ll have COVES gatherings and convenings at more and more conferences moving forward. And then I believe the last piece here on the next slide, Ryan, is just to say that art museums are already collecting data. Anyone can get involved at any time. We’re still building up our piloting cohort. The training used to be all in person, we switched it to virtual for COVID and we’re sticking with that.

Takes about a month from start to finish to kind of get everything on board and involves a couple of virtual meetings to learn about the project, to learn about the nuts and bolts of data collection, and really, really understand the protocol, build up your survey instrument, if you do have the opportunity to add a couple of questions that are specific to your institution, as well as having sort of the shared core COVES instrument. So there’s usually some back and forth around that. And if you do want more information and want to get started, the first step is really to shoot an email over to Ryan, as well as myself. But if you email him, I’ll see it. And then you do expect to follow up from COVES, just with everything that you need and everything that you can expect to get started on. And at this point, we’re going to move into discussion and questions.

But I wanted to pivot over to Caitlin at Cincinnati Art Museum just to talk a little bit about someone who’s even newer to COVES like Rachel and Kate at Crystal Bridges, what this is looking like, what the piloting experience has been like for you. You can touch on what the convening was like, if you like. Just kind of provide that on-the-ground perspective.

Caitlin Tracey-Miller:

Great. Hello, everyone, it’s good to see you. Hello COVES team. So it was a really great experience to do the COVES convening. It was really energizing and I found that during the pandemic, it’s been really lovely to have sort of a community of researchers to engage with. And it was really exciting to kind of talk through the possibilities of a future where we could have more comparable data, where we could kind of see what our colleagues are doing in terms of visitor experience metrics. And just like Crystal Bridges, we are not able to tell if our net promoter score is strong or average, or whatever it may be. So we only are seeing things in the vacuum of our experiences. And at our institution, we have had exit intercept in the past that we created on our own.

So we use this time to kind of merge our interests and with the COVES tool, and we’re able to work with Ryan and Alex to kind of make sure that the questions are answering what we need for our institution. And then we’ve been prototyping. And it’s been a little bit challenging because of the restrictions with the pandemic that’s ongoing. So we’ve been collecting our exit surveys outside on our front Plaza, which has some challenges like rain and heat and Wi-Fi dropping and some of those things that would not be happening if we were able to do it inside under normal circumstances. But we’ve still been able to get some data.

And we also just like other museums, because we’re in this strange time, we’ve also been preparing to do some email blast to kind of boost some of those numbers of this prototyping stage. And then we’re hoping to join as full-fledged members in the fall. So if anyone has questions about prototyping, I’m happy to talk about that. I don’t know if Rachel or Ryan, you wanted me to go into anything in more depth.

Rachel Wolff:

No, that was great. Thank you so much, Caitlin. Hopeful to have multiple perspectives from along the journey. Museum of Science has been with COVES for a long time now, of course, and then Crystal Bridges and then Mia, now Cinci, so it’s nice to have the broad. And now we really very intentionally wanted to move through some of that stuff we realized that are pretty good clip but also wanted to make lots of time for discussion questions. Joe, I really appreciate your question in the chat, and I’ll read it out in case anyone is not able to see it in the chat. Dr. Joe Elliott shared, “Surprised to see several metro areas not represented, not asking you to name names, but were there some museums you invited that declined to participate?”

So I’ll talk about it kind of in two phases, maybe and I would love to have Ryan and Alex and Kate jump in too when you have a moment and maybe especially to talk about historical COVES growth too. So when we were thinking about planning the convening, and kind of who are we trying to get in the room to have this conversation and help begin to shape an instrument or talk about a community of practice, we were pretty intentional as Alex said and showing that map of trying to find museums across the country. And then also museums, some of which were small, some of which were large, some of which were in between. Also, museums that were single artists are only contemporary or affiliated with a university. And then within those also inviting museums that may have dedicated researchers or evaluators, and museums that may not.

So we had a real mix of colleagues in terms of people who are doing this work already, maybe daily in some fashion at their institution, two people who consider themselves data users that might be in marketing, or advancement departments or learning and education and programming departments. We had some curators in the mix. We had some deputy directors in there. So just to kind of recap that a little bit. But to get to your question, when we extended invitations for the convening, we really found across the board that everyone was really excited to participate in that space. And we did have a few declines in there that were mostly due to like, we’re going through a major transition at our institution, and I just don’t think we can commit to four, two, and a half hour sessions for a whole month. Like seems like a lot, but keep us in the loop, things like that.

When it comes to actually piloting, it’s really been institution-led. So there hasn’t been anyone who said, “No, we’re absolutely not doing this.” But I think we’re really experiencing and this is where Ryan and Alex have some more historical perspective too, that institutions move at really different paces in terms of how they take this work on or what the conversation looks like, internally. And I’ll say, particularly because you call that metro areas, and sometimes when we think about larger cities and the institutions that are rooted in them, sometimes those can be the slowest to take on some of those conversations because they tend to have lots of moving pieces, long term strategic plans, that they’re already working around things like that.

And also, what we found in having those conversations is that certainly was the case that Mia and Crystal Bridges is that a lot of large institutions have some kind of surveying practice already in place. So it really is about working through, what do you gain when you join something like COVES where then you do have that larger community of practice, that comparison data, and the ability to really adjust. COVES does provide you the ability to add questions to your survey around your specific neighborhood, your specific institution, your specific exhibitions, all of that flexibility is there. But then there are some things that are really standardized. The way that we ask questions, particularly in the demographics section are standardized so that we can do those comparisons. And sometimes that internal conversation and decision making and bias, I think just takes take some time.

And I would say that that’s kind of my sense of how it’s growing right now. We also have I can speak specifically for a place like the Twin Cities where the Science Museum of Minnesota and the Bell Museum of Natural History were already COVES members. So then when Mia started, the Walker Art Center, which is our contemporary space and sculpture garden, in the city was like, “Oh, well, now we want to push this forward because now it means we have three other institutions immediately in our region to make comparisons to.” So I think sometimes it spreads that way to a little bit more organically and a little bit more regionally. That was a long answer, Ryan, and Alex, Kate, anything you would add?

Ryan Auster:

I’ll give us succinct version, but you hit it all, institutional readiness. And in fact, Rachel and I talked about this at a VSA webinar just last week, that building a culture of data use means that there’s a lot of moving pieces. And some organizations that you’d really expect to be involved in something like this, have so much history or so many different layers, that getting them to make changes or adopt something new can often take a long time. So we try to emphasize, we meet institutions where they are to assist them with decision making, which essentially means it’s up to them. No one can force people to collect data, no one force you to use data, no matter how valuable you think it is, or how strong of an argument you make.

So this ground roots effort that COVES started years ago and has now the attraction of many museum members is really just about that, meeting institutions where they are to support them when they’re ready. Thank you, Sarah. Sorry. We use a lot of acronyms and jargon, Visitor Studies Association.

Sarah Cohn:

Are there other questions that folks have? Oh, please Dr. Joe, go ahead.

Dr. Joe Elliott:

Was just going to say thank you. Thank you very much. And also right after I wrote the question, the word pilot finally made it through my skull. Well, that would be part of it too, but also the aspect of the biggest places probably have included that component that they haven’t yet seen that they would want to look at your technique comparatively. So thank you very much.

Ryan Auster:

Absolutely. And building off of what else Rachel said is there’s strength in numbers. So COVES originally, what we didn’t mention was the IMLS grant we got years and years ago started with a cohort of science centers. We were focused on understanding visitor experience in science centers. And it started small, we started with seven or eight museums, just like we’re doing with art museums now. And as that traction grew, as more people figured out, oh, Museum of Science is doing it, oh, Exploratorium is doing it, Science Museum of Minnesota. It grew and grew across all levels of science centers, and we’re starting to see the same thing now with our museums, except that we’re in our infancy with spreading the word and trying to get more people to participate through the pilot.

And we’re really excited based on the conversations that we’ve had. And as that list grows, that one that Alex mentioned, like these are the people who are starting to reach out, starting to set up their trainings, starting to work through the instrument, that’s where we’re going to really hit our tipping point for a full cohort.

Stephanie Parrish:

Hey, everyone, my name is Stephanie Parrish, I’m from the Portland Art Museum. And we were one of the member organizations that were in the convening in April, and my colleagues, Julissa Johnson and [Laura Bartroff 00:41:34] in marketing participated. I was not there, so I am playing a little bit of catch-up here. So I’m really grateful for you to do this because I think we’re an institution that’s really interested. But your comment, Ryan, like institutional readiness, like we’re psychologically ready, but we are not physically ready. And so I guess my big question for you is, what advice would you give an organization like ours? We’re half the size that we were before the pandemic. We’re really like lean and mean but of course, this is also the time when you want to be gathering Visitor Information and data.

So I think we’re trying to figure out if we are ready, how do we make this happen with the staff that we have. And we don’t have a culture of evaluation. We’ve done evaluation as part of grant-funded projects, usually with IMLS, and AAM, and things like that, but it just isn’t in our culture. But it’s psychologically in our culture in terms of the interest. So I’ll just throw it out there to all of you who have now been doing this and have a lot more experience. What advice would you give us as we try to push the case forward? And we don’t want to burn people out? I mean, we’re just trying to kind of rebuild and do this in a holistic way.

Ryan Auster:

Yeah, it’s a great question. I think a lot of people on this panel might have some ideas, but because I’ve had two potential new member calls in the last two days, this has been a recurring theme. Like post-pandemic, how do we get this up and running? I can tell you I’m interested, that doesn’t mean that anything’s going to happen, because who collects the data? Who’s the point person for organization? How do we even move forward? And unfortunately, there’s no simple answer, except that you already advocated the biggest reason for trying to make it work. Which is, there’s probably never been a more important time to talk about the visitor experience internally, with who you’re attracting, how you’re retaining them, what their experience is like, and how it can be improved, because so much is changing right now.

So you already have all the ammunition you need, it’s really just a conversation of like we can we need to commit to this. I would argue that many if not all of the organizations on this call have probably openly talked about DEI, making equity a priority if it wasn’t already a priority, maybe two years ago. And I think that data use can inform equity work, and that data use for many other reasons needs to be a priority. Maybe not the same priority as equity work, and we know that there’s always going to be a list of priorities, and how many can we actually put at the top or prioritize but it’s as simple as that. The conversations that I referenced, we talked about leveraging college students, if you’re near a college campus, for some data collection work, or volunteers as they start coming back on site. I can speak to our own personal experience at the Museum of Science, Alex and myself, who haven’t historically done a ton of data collection. We are now actively collecting.

We’ve spread our data collection out amongst a wider range of staff. Because as soon as you do that, you risk what you just mentioned, burnout, relying on too few people to do too much. So we’ve just tried to say like this is important, this is part of our institutional culture. And rather than just saying it out loud, we’re going to commit to it by bringing more people into the smaller circle of data collectors. So those are a few ideas, but again, I think this works differently in every organization, and I’m curious what some of our other presenters think.

Caitlin Tracey-Miller:

At the Cincinnati Art Museum, we found that COVES piloting has been a really wonderful way to engage some of the volunteers that we otherwise haven’t been able to re-engage because of COVID protocols. So we’re not yet having volunteers in the building, at the desk, or other places. We’ve had them in the past. And so we’ve been kind of able to say, “Hey, but do you want to come help us with this exciting new project?” And that’s been a nice way. So that’s been kind of a silver lining for us in terms of volunteer engagement because I think the hardest lift of COVES is just collecting all those physical intercepts for sure because a lot of the other tough work is happening on Brian and Alex’s side.

Rachel Wolff:

Yeah, Caitlin, I was thinking of that, I knew that Cinci was using volunteers, and also shout out to Kate and Crystal Bridges because I know that you’re part of an evaluation and research team, Crystal Bridges has also used students, volunteers, and visitor experience like front of house staff, some staff from other departments like marketing, and fundraising and other places to help them kind of understand what the process looks like. And again, that can expand who can do this work. So I think we do have some models among institutions. And then among the broader COVES community, it’s really helped… There’s a science museum that we have in Florida that has one staff member on-site. And because of how they are trafficked, that person collects data, not all day, every day, and you don’t need to collect data all day, every day, but works that into that flow of that moment with the visitor as they’re exiting to have that.

So I do think we have some models of the kind of nuts and bolts of that. And then also Mia is a large organization in some ways, but I fairly lean in terms of what you might call evaluation staff. And I actually don’t have evaluation in my title, there’s one person who was really focused on sort of grant reporting and evaluation and impact on those pieces. And then Mia who’s working with COVES but then also with lots of operations and revenue-based data and ticketing systems and things like that. One of the conversations that I’ve had internally too, is that like you mentioned, Caitlin, in our survey that we had before, I was doing all of the data cleaning, and analysis, and pulling that reporting together, that responsibility now sits with COVES. So while we need to do the data collection, a lot of that kind of down and dirty data work is happening somewhere else, which I’ve really been able to evidence with my institution of like, so now I’m working on this instead.

Like, yes, there’s a cost associated with this, yes, there’s like the lift to get it running and to get that data collected. But now we have it, and it’s freed up me in different ways to be a different kind of partner, a better partner to some of my other colleagues and some of the other projects that they’re trying to do around data. I mean, that you say that the psychological desire is there, like what really helped drive it at Mia was our leadership team, especially our director was like, I want to see a weekly report that is actually showing me voice of audience, is showing me direct quotes alongside quantitative data about the experiences that our audience is having. And I was like, “Awesome, we can do that. And here’s what it takes.” Like I want to feed that hunger, I want to support passion, and then here’s how we actually like make that happen on the ground.

I think being able to show that that’s the outcome, you’ll see the quantitative and the qualitative data side by sides, and someone else will do the nuts and bolts of the cleaning, and the sort of dashboarding of it, can make that, can really help sort of move some of those logistics pieces along.

Sarah Cohn:

That was a lot of ideas for you, Stephanie, hopefully, note-taking was easy for you in that list of options. I would also say that AMSI is a local partner. I don’t know how close you all are, but AMSI is a part of COVES. And so thinking about, connecting back to what Rachel said previously about we have regional existing science centers, natural history museums, children’s museums, who are already part of it. Reaching out to them and saying, “Support us figure it out.” And I think Saint Louis Art Museum, you have that with the Saint Louis Science Center. There’s a map with all the things. I won’t guess as to who’s part of it anymore, too many organizations. But just thinking about who is regional and who can support and I think some of these pieces through with you.

We’re starting to have that kind of networking as well. Other questions or thoughts coming up for folks, or people who are at the convening have other things to share? Well, please keep putting things into the chat, it’d be great. Following up with us, yeah, following up with emails, thanks, Ryan, is wonderful. And I have another thought. Rachel put a Google folder in the chat as well, that has a bunch of different documents in it. If you are someone who enjoys thinking a lot about language and words and want to sort of follow along, or participate, as Alex shared, we are working on the instrument and thinking differently about how are these, as these different sites pilot in that long list, what are the changes going on that are being made? And we will be sharing that through our website as things go forward.

So you can sign up to hear from us there or reach out and say, I’ve got time or interest and we’ll see if we can pull you in to be another thought partner. Because the more people in this network thinking about all this with us, we think the better. I think if there are no other questions that are coming into the chat. I’ll send it back over to Rachel or Erica, do you have any closing thing, COVES? I don’t remember.

Rachel Wolff:

I think just thank you. Thank you to all of you and CARE and AAM, again. We have the website up there. We have the Google Drive out there, and our emails. So you can probably tell we love to talk about this stuff, so please feel free to reach out with any kind of question level of question, happy to keep it going. But thank you, all of you for being here today.

Erica Kvam:

With that, we’ll close it out. Thank you all so much for presenting today. And again, if you want to learn more about the CARE professional network or AAM head over to aam-us.org. And again, we’ll post this session on YouTube so you can send it along to your colleagues that are interested as well. So thank you very much and we’ll see you at the next CAREpy.

Ryan Auster:

Thanks, everyone.

Alex Lussenhop:

Thanks so much, everyone.

Sarah Cohn:

Thanks so much.