Touring with the Toolkit: China Plate Theatre’s Approach Many thanks to China Plate Theatre for sharing their experiences and insights with us. In particular, we would like to thank Sarah Isaacs and Ed Collier for giving up their time to participate in this case study. 15 minutes A Family Business by Chris Thorpe. Photography by Andreas J. Etter, design by Eleanor Field. China Plate Theatre is an independent theatre producing studio based in Birmingham. They work across making, development, touring and training to deliver their mission of, Developing a new model of creating and producing theatre that opens up the way performance is made, who makes it and who it’s experienced by. China Plate Theatre is an Arts Council England-funded National Portfolio Organisation (NPO). We were very fortunate and thankful to have the opportunity to speak with Sarah and Ed from China Plate Theatre, to listen to their experiences and to share their learnings with you. To make this case study easier to navigate, we are dividing it into sections for you. Of course, we would recommend that you read the whole thing, but, if you’re particularly interested in a specific section, simply click on the appropriate link below to be directed: Our Approach to Using the Toolkit Evaluating and Collaborating with Venues Benefits and Challenges of Using the Toolkit Closing Messages The following text is China Plate Theatre’s own words, responding to questions posed by us. Image 1 – A Family Business by Chris Thorpe. Photography by Andreas J. Etter, design by Eleanor Field. Our Approach to Using the Toolkit How did you decide to approach the Impact & Insight Toolkit? We use the Impact & Insight Toolkit in conjunction with our Audience Aims flowchart. The Audience Aims flowchart provides the team with a framework to keep track of how their projects are meeting the company’s audience* targets by setting: Objectives: SMART (Specific, Measurable, Attainable, Relevant, Timely) Outputs: Measure of what you produced, but not value or impact Outcomes: Measure of value, impact or effectiveness These inform both marketing and recruitment strategies, as well as contributing to project evaluation frameworks that can effectively measure progress. The Toolkit supports organisations in understanding the impact of their work and how it is perceived, which maps very helpfully onto generating outcomes as part of our Audience Aims flowchart, the main aim of which is to ensure our business plan is audience-focussed. Image 2 – An extract from China Plate Theatre’s Audience Aims Flowchart. The flowchart helps the producer articulate what they are hoping to achieve from the project in line with the company’s audience aims, which are outlined in the evaluation section of a funding application. The Toolkit helps the producer measure how effectively this has been achieved, which is reported on as part of the final project evaluation. The Toolkit supports us to measure the value, impact, and effectiveness of our Audience Aims outcomes by conducting self (prior), public and peer surveys. Conversely, the Audience Aims outcomes can inform which dimension statements to include in those surveys. Once we have the results from the surveys, the comparison of our expectations (self prior assessment) with the experiences of those that receive the work (peer review and public response) provides us with an intersection between our creative communities and industry. It’s important that we consider these as tools to be used together, as both the Toolkit and flowchart can generate insights that allow China Plate to judge the extent to which we are meeting our vision and mission and make strategic decisions on how we develop shows and projects. The Toolkit also allows us to measure and evaluate in relation to Arts Council England’s Ambition and Quality Investment Principle, which we can report back to our Board of Trustees. Image 3 – A Family Business by Chris Thorpe. Photography by Andreas J. Etter, design by Eleanor Field. Evaluating and Collaborating with Venues How did you approach venues about evaluation? Evaluation is often discussed as part of arranging venue contracts and partnership agreements, specifically in relation to the data-sharing sections. Where we are the lead producer and want to integrate our Audience Aims flowchart and the associated evaluation frameworks, we will converse with the venue/partner about how we collaborate with them alongside their existing evaluation processes. Often, this conversation is instigated by our Marketing Consultant, who is already working closely with the venue/partner and will connect them with our Digital & Data Manager. An example exchange might be: we provide them with our survey link and mailing list sign-up link for inclusion in their post-show email, on the basis that we will share the results of the survey with them. Did you have a data-sharing policy in place with the venues? Yes. As part of venue contracts and partnership agreements there will be data-sharing policies in place. If we decide to collaborate with a venue/partner on an evaluation process that isn’t outlined in the standard data-sharing agreement, then we will look to create a specifically-tailored addendum which will cover anything additionally agreed in terms of data-sharing. How were surveys distributed? We often work with Marketing Consultants, who act as a bridge between ourselves and the venue/partner. When they loop in our Digital & Data Manager, they will offer some options on how we can collaborate around evaluation to start the conversation. This has previously included: Dividing the audience 50/50 – Suggest sending China Plate’s survey to 50% of the audience and sending their own survey to the other 50%, ensuring that the project can contribute to both parties’ reporting requirements. Importantly, this would also ensure that we get responses to the outcomes we need to evaluate against from our Audience Aims flowchart. We may also subsequently agree to share our survey results with each other. Sending China Plate’s survey separately – Ask the venue/partner if they can send out our survey. Offer the opportunity to include questions that the venue/partner might want to ask. Liaise with the venue/partner to sign off on the questions. Build the survey in Culture Counts. Provide a link for inclusion in a post-show email, alongside a sign-up link to our mailing list. Some venues/partners were happy to do this without receiving the survey results, and some would on the basis that they would receive the survey results. Sometimes, a venue may agree to send out a survey, but only at the end of a run (as a final request to audiences from the producer/artist), whilst still sending their own survey after each performance. Sending the venue/partner’s survey – if they are unable to send out China Plate’s survey, we would ask if we could include some of our questions in their survey, which would enable us to get responses to some of the outcomes we need to evaluate against from our Audience Aims flowchart. If this wasn’t possible, we would ask if they could share the results from their survey, so we would at least have some idea of the audience’s experience at that venue/location. How did you obtain qualities-focussed data if the venue/partner did not share your survey? As we are a non-building-based touring organisation, we rely on working with venues and partners to distribute surveys, as we are not in direct contact with audiences (who will have booked tickets directly with the host venue). To mitigate against venues and partners not sending our survey, we have also used the QR code function in Culture Counts to link to surveys on printed materials. A recent example is alongside our production of A Family Business, where we created a resources postcard to be handed out at the end of the show. The primary aim of this was to provide audiences with five tips on ways they could raise awareness / advocate for change in relation to nuclear disarmament. However, if one of the tour venues couldn’t support us in sending out our survey digitally, the postcard also acted as a mechanism to invite those audiences we weren’t going to be connecting with via email to provide us with feedback. Image 5 – A Family Business resources postcard, China Plate Theatre. If we are sending a survey to participants who have taken part in or attended one of our Training & Development programmes or events, we have been distributing surveys as part of a follow up email, which may also include session resources and an invitation to sign up to our mailing lists. We’ve also been trialling sharing the survey link in the chat function on Zoom at the end of online workshops, to encourage people to fill it in straight after the session with the aim of improving our response rate and to capture people’s ‘fresh’ reactions. What did the data tell you about your impact in different locations? This is a very interesting question for us to consider. Historically, China Plate has reached over 100 venues a year, touring to studios and community venues with the occasional bigger / midscale event. In recent years, and with the development of our new mission, we have actively looked to both diversify and consolidate our offer. When we say ‘diversify’, we are referring to the sense of opening up outside of theatre partnership and presenting conventions, such as making work with healthcare workers or for bus stops, and consolidating our offer around shows that reach a bigger audience, i.e. aiming for our first 1,000+ seats touring show in 2026. With that in mind, the diversity of locations and contexts into which these surveys are going is very broad indeed. Within the first 18 months of working with the Toolkit, we have done a regional midscale tour, two international / national small-scale tours, a small-scale London run; made and presented audio pieces with a healthcare union and trained producers through longer-term and one-off programmes. The Toolkit has enabled the gathering of much more coherent and quantifiable data in all of those environments, but we are yet to be able to build iterative patterns from it. In snapshot, it is fantastic at telling us how an interaction has gone, and, to an extent, who was there. Long-term, it will allow us to adapt our work and communication strategies in a more targeted way. This is already significantly influencing how we approach building long-term touring relationships, and we are exploring several multi-year programmes with different partners to help us embed our work within communities across the country. Image 4 – Wuthering Heights by Inspector Sands. Photography by Alex Brenner. Benefits and Challenges What have been the biggest challenges of using the Toolkit, and how did you overcome them? As a small team, one challenge has been balancing the significant amount of work involved for our Digital & Data Manager. This includes upholding the Audience Aims flowchart and Toolkit processes across China Plate’s whole programme of work, particularly in tandem with the reporting required from NPOs. Designing and building surveys can take a lot of time. These surveys are generally iterative but are also tailored to suit each specific project and show. As we are currently exempt from some elements of NPO reporting, we are exercising our freedom to use surveys to gather feedback in favour of what we need. 2023-24 was our first year using the Toolkit, so we decided to trial dimension lists for Live Productions and Training & Development. We also trialled using the Illuminate demographic questions. When reviewing our end of year Ambition Progress Report, we found we didn’t have a large enough response rate to be able to benefit from sufficient benchmarking[1]. We’ve decided we need another year of surveying to be able to critically reflect on how our audiences/creative communities are rating our dimensions and form more robust benchmarks[2], as we will be able to compare year on year. We’ve also encountered some challenges with how we design surveys in Culture Counts. One issue is deciding whether to group surveys within a single evaluation or to build them separately in different evaluations. Another challenge is whether to deviate from the prior, peer, and public combination in favour of creating multiple public surveys. Each of these choices affects how effectively we can utilise the reporting options. With guidance from Counting What Counts, we have learnt that we can: Temporarily move surveys into the same evaluation to increase the response rate closer to the threshold before running the end of year reports using the Reporting Dashboard (as it’s unlikely for some projects that we will ever go over the minimum sample size of 25 public responses[1]). Move surveys out to be able to run an Insights Report for only one survey. Move two separate surveys into the same evaluation to be able to compare them using the Analytics Dashboard. What have been the biggest benefits of using the Toolkit? The biggest benefit of using the Toolkit in conjunction with our Audience Aims flowchart has been the improvement to our quantitative and qualitative data collection around our mission and vision. It has allowed us to form a centralised process and collect feedback from a breadth of our ‘creative communities’, including audiences, participants, creatives, cast, guest speakers, facilitators and venues themselves. It has enabled us to work better with venues/partners (and consultants) in terms of including our audience feedback survey as part of their post-show emails, which has historically been difficult as a touring organisation that is not in direct contact with bookers. Using the Toolkit in conjuction with our flowchart has allowed the framework to become embedded as a company-wide process, ensuring the delivery of the business plan is audience-focussed and each project is feeding into that (i.e. the business plan mapping onto a show/project and a show/project mapping onto the business plan). This is bolstered by having clear outputs and outcomes from the flowchart to evaluate against within the Toolkit when choosing dimension statements and designing surveys. Using these tools together also provides longevity of thought for each show or project; we can create different iterations of the Audience Aims flowchart and learn from the responses to previous surveys about the outcomes/impact of future versions of the show or project. Furthermore, data collected via the Toolkit has directly supported writing the final project evaluations for funding applications, as the producer can reflect on whether the objectives, outputs and outcomes have been achieved, and to what degree. This enables us to identify key takeaways and learnings which can be shared with the wider team. Image 6 – Wuthering Heights by Inspector Sands. Photography by Alex Brenner. Closing Messages What is the one message you would like to leave readers with? Take the time to build a trackable framework or process, which will allow you to strategically understand what you need to know and how to ask it. Even if it has taken us several years to develop our Audience Aims Flowchart, we can feel assured that our team is onboard with the processes, as they can see how it benefits their roles and supports the success of each project or show. Counting What Counts’ Reflections Thank you to Sarah, Ed, and the team at China Plate Theatre for their valued contribution to this case study! Their insights offer valuable lessons for other touring companies seeking to enhance their evaluation processes and develop partnerships with venues. By integrating the Toolkit with your internal processes, you can gather meaningful feedback and make informed decisions that drive your mission forwards. Embracing a collaborative approach to these evaluation practices not only enriches your projects but also strengthens your relationships with venues and partners, ensuring mutual benefits from shared insights. We look forward to hearing more from China Plate as their practice evolves and as they continue using the Impact & Insight Toolkit! *By audience, we mean our creative communities and anyone who comes into contact with our brand as a consumer – this might be artists, creatives, producers, participants, members of the public, other organisations, freelancers, community groups or organised groups (like schools and colleges). [1] Within the Reporting Dashboard, dimension results are compared to a benchmark using a statistical significance test. For technical reasons, these tests don’t work very well when there is a very small sample size. So, for that reason, we only run a test for a dimension result if there are at least 25 public responses. [2] A Benchmarking Dataset is now available and therefore benchmarks can be obtained outside of the Reporting Dashboard.