Research and evaluation projects allow museums to engage their communities, visitors, and non-visitors in various ways, but without anyone to participate in them, we are left with only our protocols and questions. That’s why strong recruitment strategies are essential, as several host museums participating in the Measurement of Museum Social Impact (MOMSI) study shared on the blog last year, and so are strategies to retain participants once recruited.

As it turns out, retention has been an equally important piece of the MOMSI study as recruitment, coming with its own set of successes, challenges, and lessons learned. With the study’s long timeline (between October 2021 and August 2022), and multiple commitments (three visits to the host institution), it has seen higher-than-expected attrition. Despite steady reminder emails and up-front incentives like free admission, only 2,560 (49 percent) of the 5,238 people recruited to participate in the study completed at least one visit, and only 1,162 (22 percent) completed all three.

We will never know exactly why the attrition has been high, though our theories include ongoing wariness around indoor spaces due to COVID surges, lack of time or motivation to participate fully, or even that our reminder emails went to a junk folder. Nevertheless, we are taking the opportunity to pivot the project and find more useful information to share as we draft the museum social impact toolkit that will conclude the study in summer 2023. Instead of sending the concluding social impact survey only to participants who concluded all three visits, as we originally planned, we have decided to send it to anyone who completed at least one visit, with an additional question asking how many visits they made, for additional data-parsing. The completion rate for the survey is near 70 percent so far.

Despite the high attrition overall, some of the thirty-eight host museums found successful strategies for retaining participants throughout, which are helpful for anyone planning a future research and evaluation project to consider. Below, three of these museums share their unique circumstances with the study, how they kept participants engaged, and the incentives they offered.

Jule Collins Smith Museum of Fine Art at Auburn University

Skip over related stories to continue reading article

As a teaching museum at a nationally ranked Carnegie R1 institution, our mandate is to serve students and faculty. However, we are also the only university art museum in our state engaged in PreK-12 and community outreach. Therefore, our recruiting strategy needed to attract each of these audiences. In addition to utilizing word-of-mouth, we submitted calls for participation and announced the project via a press release in weekly email digests distributed to students, faculty, staff, alumni and select regional media; developed an article for Auburn Research; targeted the local community through social media posts; and distributed application links to email subscribers within a two-hundred-mile radius. As the study guidelines stated, student employees (like other museum employees) were ineligible; however, we encouraged enlisting friends, roommates, classmates, and family members. This peer-to-peer recruiting proved highly effective. Students served as great ambassadors, and the turnaround in reaching the minimum threshold (one hundred participants) surprised the team. The Auburn community stepped up and eagerly supported the effort.

As a museum with free admission, we adopted a “breadcrumb approach” incentive structure. At each visit, participants received a gift card of varying amounts, starting with a five-dollar Starbucks card, then a fifteen-dollar Kroger card, and finally a twenty-dollar restaurant gift card. We knew early on that we wanted to use gift cards as incentives. Still, our team based the final decision on conversations with our student workers about which services or vendors would be most appealing and useful to students.

We tasked the visitor services team with identifying MOMSI participants and logging each visit. This proved challenging, as we had anticipated the volunteers would self-identify as part of the study upon arrival. But as the study progressed, we realized that not all of them were doing this—sometimes because they did not think it was required. After all, admission is free; they didn’t need to announce themselves to receive that part of the incentive package. As a response, the visitor services team added a question to their standard welcome: “What brings you in today?” This prompt enabled front desk associates to easily identify participants, mark off their MOMSI cards, update the tracking spreadsheet, and distribute the applicable incentive.

Despite our robust incentive structure and the initial eagerness to sign up, we did experience a slow start. However, after participants received official MOMSI reminder emails in February 2022, our attendance skyrocketed, with twenty participants completing their first visit to the museum that month. At the end of the project, 56 percent of participants visited at least once and received the survey.

The data collected through this national research study will help inform the museum’s future strategies for outreach and engagement and reinforce the museum’s vital role on campus with university leadership and other high-level decision-makers. As many indicated that they had never visited the museum before, we are excited to see what the social impact data sets reveal about new audiences and the ways in which the museum impacts the community.

Saint Louis Zoo

The Saint Louis Zoo is admission-free for its two to three million annual guests and enjoys the ardent and continual support of residents, who express great pride about its role in the community. In research studies of St. Louis area residents, half consistently report visiting the zoo at least annually, and more than 90 percent have been at least once in their lives.

Given this high attendance level, it would seem that recruitment and retention for the MOMSI study would be simple, but we faced several obstacles during both phases. Our original recruitment method, placing flyers at various St. Louis city and country libraries, yielded only a handful of participants in the first week. Then we decided that we could use our online guest reservation system, which we had adopted shortly before the study due to the pandemic, to recruit and invite a representative sample of guests to complete the study. Email addresses were utilized for communication with guest permission.

Although the data from the reservation system proved cumbersome to sort through, it gave the MOMSI project team an automatic way to track participant visits. In communication with the participants, the team asked them to use the same email address to reserve their tickets that they used to sign up for the study, so they could cross-reference the data sets. In May 2022, however, the zoo removed the system with little notice, and we had to revisit our tracking methodology. We decided to place signs with large QR codes at each zoo entrance for the participants to self-report their visits, as some other sites in the study have done. We notified study participants of the change via email, but it is unclear if it may have caused any attrition.

The other major challenge was to find an incentive other than free admission, since we always offer that. We decided on entering participants into a drawing for five free one-year zoo memberships. While we were worried this incentive might not be enough to attract the repeat visitation we needed, it turned out to be quite successful in retaining participants. With 81 percent of the people accepted into the study at the Saint Louis Zoo visiting at least once, we maintained one of the highest first-visit completion rates across the study sites. Of those participants, many are completing the social impact survey, at a rate similar to the average for the study overall. Our participation hasn’t been without challenges, with our systems changing mid-project and our need to find alternative incentives, but we are excited to see the data from the survey.

Museum of Science, Boston

Recruiting participants for the MOMSI study turned out to be more difficult than we had hoped. Initially, we partnered with several community groups with which we had working relationships (such as non-profits serving under-resourced families and community colleges) in the hopes of attracting participants who didn’t necessarily have a relationship to the Museum of Science (MOS), or museums at all, or science and STEM at all. However, the combination of our passive early recruitment methods and continued diminished visitation trends caused by public health concerns left us scrambling to increase our sample size. Ultimately, we had to rely on an email campaign that pulled a large number of contacts from various constituent lists, including recent ticket buyers, program participants, and a small number of existing members.

Once we had secured a sufficiently large participant group, we had another hurdle to clear: offering free admission. As a large organization with a complicated ticketing system, we had to ensure everyone had a means of entry—and we had to find a funding source to cover it. As a cost-saving measure, we issued all participants a one-year membership granting free admission for up to two individuals, which additionally provided perks like discounts on parking, food at the café, and purchases from the museum store. This method was cheaper than offering six tickets per study participant (one for the participant and one for their guest for three separate visits) and still met the project requirement of providing free admission. Most importantly, providing memberships up front to participants enabled us to track visitation effortlessly without requiring additional staffing or signage—MOMSI memberships were tagged so that we could run reports on visit history each month.

While we envisioned that this free up-front membership would act as an incentive to participate in the study—and to some extent, it did get folks to sign up—we didn’t strategize our efforts to promote repeated visitation beyond that point. Just like the free gym membership that allows one to say, “Ehhh, I’ll work out tomorrow…” folks still needed motivation to come to the museum! And while free admission certainly removed one barrier to entry, it did not solve other potential barriers to visitation, like travel (recent public transportation issues in Boston have caused both trains and traffic to be quite a headache) or longstanding issues of inclusion in museums and STEM.

To encourage the three visits and completion of the culminating social impact survey, we dangled the fairly typical “randomly selected winners!” research incentive, offering a large-ish prize (more than one hundred dollars) and better-than-average odds (one in ten) to those who successfully completed all study activities. Unfortunately, for a study period that was nearly a year long, this ultimate reward was far away—both in terms of timing and number of objectives to complete—and of course, there was no guarantee of winning. Despite this, we fared relatively well: 92 percent of our recruited sample visited the museum at least once, and 60 percent completed three or more visits. Interestingly, due to a handful of outliers who visited more than ten times during the study period, the average number of visits per study participant who came at least once was nearly 3.6 (n=123).

Although all MOMSI study participants received ongoing communications directly from the museum as part of our member newsletter, the tactic that seemed most effective in encouraging visitation was direct messaging about the study itself from the MOMSI research team. In addition to reminding them of their involvement, the value of the study, and the activities to which they committed (however soft a commitment it may have been), nothing motivates quite like a final deadline: The last month of the study alone saw forty-two additional visits (just under 10 percent of the total)!

Recommendations

Throughout these three examples a few key strategies stood out. Here are some tips on strong retention that we’ve learned:

1. Be flexible but purposeful.

Sometimes a study will not unfold exactly as you plan, but there are opportunities to pivot and still get the data you need. In our case, we changed our original plan to only send the survey to those who completed the “required” three visits, instead sending it to everyone who visited at all, with an additional question about number of visits. In some ways, this adaptation could make our study stronger, as the differentiation it allows for might find even more interesting data than we planned for—we will find out!

2. Create an incentive structure.

The best way to find strong incentives is to work with your community to identify them. You also need a structure that motivates participation in all aspects of the study (not just recruitment!) and ensures you have the appropriate resources, including admission and finances. Provide adequate time for pursuing multiple strategies in the event that one doesn’t yield the numbers you need, especially if you’re targeting specific populations or new audiences. To the extent possible, balance the necessity of multiple visits with the length of the study itself so that participation remains top-of-mind.

3. Set up a communications plan.

Does your community prefer emails, texts, phone calls, or another type of communication? How often do they want to hear from you? Do they want to hear more about the museum or just information related to the study? Do they want to hear specifically from the project staff, from the same staff member each time, from the communications team? Do different participants have different methods/timings they prefer? Be sure to consider the different modes and timing of communication and set up a plan accordingly that covers the duration of the project.

4. Develop a way to track participants.

Don’t assume that participants in a study will identify themselves as such at every visit (even when an incentive exists!). Set up a tracking mechanism to document participation over time, which most likely means working with visitor services, marketing, and/or membership staff. And then, monitor your tracking system and be prepared to pivot if larger changes happen at your institution. Although this might not seem likely a likely scenario, it happened to several participating institutions in MOMSI and required quick thinking amongst staff.

Learn More!

CAREapy session: How to Measure Museum Social Impact: From Participant Recruitment to Retention

February 23, 2023 – 12:00 pm EST

As the national Measurement of Museum Social Impact (MOMSI) study comes to a close, learn how study sites navigated visitor participation in the project. Hear from three different museums—a rural museum, an art museum, and a zoo—on their successes and challenges in all steps of the project, from recruitment to participant retention. No study is perfect, and the recommendations we share just might help you on your next evaluation project.

Register here