A change in season is the ideal time for reflection, and, with the new Financial Year upon us, there’s no better time to pause and take a fresh look at your evaluations to make sure that they are working for your organisation.

 

Introduction

With Arts Council England’s (ACE) recent decision to extend their current funding cycle from 2023-26 to 2023-27¹, it might be tempting to de-prioritise gathering data to support future funding applications for the time being. However, with lingering uncertainty regarding the future of arts and cultural investment¹, it is increasingly important for organisations to be able to demonstrate their resilience and non-financial value to stakeholders and potential funders.

We especially developed the Impact & Insight Toolkit (Toolkit) to help arts and cultural organisations:

• Learn about the impact of their work
• Implement data-driven decision-making
• Evidence commitment to ACE’s Ambition and Quality Investment Principle (A&Q IP)

The A&Q IP is about demonstrating your commitment to ‘improving the quality of your work’ through a ‘well communicated set of aims and ambitions’²  that are specific to your organisation.

The Toolkit helps you gather a rich set of data that speaks to the quality of your work across different time periods and perspectives. It helps you stay on track with capturing the data you need for your next funding application, while helping you demonstrate your present commitment to the A&Q IP.

 

Demonstrating Ambition and Quality

A good starting point for demonstrating Ambition and Quality is the implementation of a Quality Evaluation Framework. A Quality Evaluation Framework (QEF) enables you to:

• Systematically track your progress against measurable outcomes and ambitions
• Identify any potential issues
• Gain insights into how to improve the quality of your work

At the heart of an effective QEF is a clear definition of what quality looks like for you. As an organisation, you must identify what you believe are markers of high-quality work so that you can measure the extent to which your work meets your individual definition of quality³. It’s important to heed ACE’s guidance that no ‘types or scales of creative activity are inherently better or of greater value than others’³.  ‘Quality’ will look different to everyone, which is why it is so important that you identify what quality looks like for your organisation and the work you deliver.

Establishing a QEF demonstrates to ACE that you actively consider the A&Q IP in your planning and use the data you have collected to progress, in line with the Key Performance Indicators identified in your funding agreement[3]. ACE encourages organisations to utilise an evaluation framework to measure and track the quality of their work[2].

There are many options to choose from when it comes to establishing a QEF. However, the Impact & Insight Toolkit proffers a QEF specifically designed with, and for, arts and cultural organisations.

 

Choosing the ‘right’ questions for you

There are a few different ways to approach creating evaluations in the Impact & Insight Toolkit. However, to get the best out of your evaluations, we recommend that you choose questions that resonate with your strategic goals and ambitions. We call this ‘Articulating Ambitions’.

The Toolkit builds upon a digital evaluation tool, Culture Counts. Culture Counts enables you to create unique evaluations containing surveys with questions that capture your quality indicators. One type of quality indicator encouraged in the Toolkit approach to evaluation is a ‘dimension’. Dimensions are standardised statements included within surveys, allowing respondents to rate their level of agreement. Dimensions capture data relating to specific outcomes which arts and cultural organisations may deem important to determining how successful their work was. You can explore the available dimensions in the Dimensions Framework, developed to help you convert your organisation’s mission and vision into a measurable form.

Revisiting the dimensions chosen in the Articulating Ambitions stage of the Toolkit is best practice. This ensures that your evaluations collect data that helps you learn how your work is perceived; evidences your commitment to the A&Q IP; and helps you stay on track to meet the goals outlined in your Investment Principles Plan.

Consider the following questions:

1. Do the questions included in your surveys indicate what ‘quality’ looks like to you?
2. Does the data you have collected thus far enable you to demonstrate how different works/events/activities contribute towards fulfilling your strategic goals?
3. Does the data you have collected thus far enable you to demonstrate your commitment to improving the quality of your work?

If the answers to these questions are ‘no’, don’t hesitate to speak with us about choosing a different selection of dimensions; there might just be one that needs swapping to make it the ‘right’ set for you.

 

Suggestion:

ACE holds the board accountable for monitoring and reporting upon progress made towards targets and success measures outlined in an organisation’s funding agreement.

With this in mind, we recommend that organisations engage board members and stakeholders in selecting their chosen dimensions. Involving your board in this way from the outset ensures that they feel more connected to the evaluation process and the reports produced, which will aid decision-making and prioritisation.

 

If you have not used the Dimensions Framework recently, then you may be interested to learn that it now features a selection of ‘place-based’ dimensions. The new dimensions have been explicitly developed by Counting What Counts to support the evaluation of place-based work in the cultural sector, following consultation with arts and cultural organisations across the country.

 

Reflecting and Reporting on Ambition and Quality

Once you have collected your quality-focussed data, you should allocate some time to reflect on this. A couple of questions to ask of this data could be:

1. Do the results reflect your ambitions, as an organisation and for this specific work? If not, why might that be?
2. How can you use these results to further our offer and continue to push for greater quality of work?

There is a variety of tools available within the Impact & Insight Toolkit that you can use to explore the data collected. Use these as a starting point to aid conversations regarding the quality of your work:

• Reporting Dashboard
• Analytics Dashboard
• Insights Reports

If you are from an ACE-funded organisation, make the most of the charts and dashboards by discussing them at your board meetings and including them in your board papers, which are subsequently shared with ACE. If you have any questions about the charts you find in your account, or need support interpreting them, get in touch and we will do so gladly.

This provides great evidence that you are actively engaging with the Ambition and Quality Investment Principle, and that you are committed to offering high-quality work.

 

How to approach evaluation for your organisation

With all this in mind, it’s important to acknowledge that there is no singular ‘correct’ way to use the Toolkit. While our guidance offers insights into best practices, each organisation should tailor its evaluation approach to meet its specific needs.

Although ACE has funded the Impact & Insight Toolkit, the data collected through Culture Counts remains under your ownership. Ultimately, you have the freedom to use the Toolkit to track and demonstrate your progress and impact in a way that feels true to your organisation.

 

 

1 https://www.artscouncil.org.uk/blog/time-to-think

2 https://www.artscouncil.org.uk/blog/essential-read-ambition-quality

3 https://www.artscouncil.org.uk/lets-create/strategy-2020-2030/investment-principles

 

Photo by Marcel Strauß on Unsplash