As many of you know, between September 2019 and March 2020, The Audience Agency (TAA) and Counting What Counts (CWC) are running a trial with a small number of National Portfolio Organisations (NPOs) to test the integration of the Audience Finder and Impact & Insight Toolkit systems. This month’s webinar was focused on this trial, and you can check out the recording here.
Both Audience Finder and the Impact & Insight Toolkit are sector-led and strive to meet the audience data and evaluation needs of the cultural sector. During the launch of the Toolkit, NPOs were keen for the two platforms to be integrated, enabling them to collect all their Arts Council required data in one platform.
They raised queries about:
- Survey fatigue for audiences
- Capacity and resource challenges within organisations to get the most out of both systems
Subsequently, TAA and CWC met to explore how NPOs can most efficiently and effectively use and get the most value from both Audience Finder and the Insight & Impact Toolkit. We identified four possible pathways for NPOs. Two of these pathways involve using Audience Finder and the Impact & Insight Toolkit separately, and the other two routes use one system to collect all audience data.
In consultation with the Arts Council, we agreed to trial the four options with a small sample of NPOs. The trial is enabling CWC and TAA to test separated and integrated methods of data collection whilst liaising closely with participating organisations to gain feedback on all aspects of the process. The trial aims to provide recommendations to the Arts Council on how Audience Finder and the Impact & Insight Toolkit can be used successfully and efficiently, with minimum burden to NPOs.
As we evaluate the four options, we will consider:
- Are the options straightforward to deliver in practice?
- Which options involve the least time, effort and complexity?
- Which options produce data in the most accessible and useful formats?
- Do they deliver value to organisations?
- Are some options better suited to certain types of organisation?
The four options are:
|Group||Use of Audience Finder||Use of Impact & Insight Toolkit||Currently available to all NPOs?|
|A||Use the AF survey at any event, where a Toolkit evaluation is not being undertaken.||Select 4 events over the organisation’s programme to evaluate using the Toolkit. Send the public survey to all attendees at these 4 events.||Yes|
|B||Use the AF survey at all events but distributing it to only half of the attendees where a Toolkit evaluation is also being conducted.||Select 4 events over the organisation’s programme to evaluate using the Toolkit. Split the sample by sending the public survey to half of the attendees||Yes|
|C||Decide which 4 events Toolkit questions will be used at; insert the Toolkit questions into the AF survey. When a respondent selects which event they attended, show/hide the Toolkit questions accordingly.||For the 4 chosen events, use the Toolkit to collect peer and self responses only. Public responses are being collected in AF.||No|
|D||Contact TAA to agree any ‘premium question’ to be added to AF survey.||For the 4 Toolkit events, incorporate the AF survey questions (inc. any ‘premium question’) into the public Toolkit survey. For all other events, use a survey which contains only the AF questions.||No|
For more information on options A and B, the currently available methods of using the Toolkit and AF, please see this guidance file.
At this stage, it is too early to say for sure whether using one platform to collect data for Audience Finder and Toolkit truly addresses the queries raised by NPOs. However, having conducted some initial interviews and working alongside the organisations as they progress through the trial, it seems as though the process of setting up surveys in the opposite platform is reasonably straightforward. The big unknown is what, if any, unforeseen difficulties will be encountered as data is transferred between the two systems. This is being investigated and requires great consideration before any offers are made to the portfolio.
We are looking forward to learning more as the trial continues. In the meantime, please do get in touch if you have any questions regarding this trial or if you require any assistance.
With thanks to The Audience Agency for their continued collaboration throughout the course of this trial.
Image credit: Lloyd Coenen Art – ‘Solar Flare’