ACE recognises that because of COVID-19 you may not be able to complete the requisite number of evaluations this year.

Arts Council England – Mandatory Requirements for Creative People and Places Using the Impact & Insight Toolkit

The information on this page was written by Arts Council England for CPPs using the Impact & Insight Toolkit. View/download a PDF version of this page here.

  1. Mandatory evaluation expectations for new CPPs
  2. Evaluation expectations for existing CPPs
  3. Activity choice
  4. Dimension Choice
  5. Sampling Guidance

 

Introduction

This Impact and Insight toolkit has been developed from the Quality Metrics pilot in 2016. Counting What Counts successfully tendered to provide the service to National Portfolio Organisations in our 2018-2022 round of funding.

Arts Council England have since, introduced it to be used by New Creative People and Places projects as a pilot, to test whether this model can be effectively used to address our second key CPP Research question:

To what extent is the aspiration for excellence of art and culture and excellence of the process of engaging communities achieved?

We are hoping use of the Toolkit will assist Creative People and Places projects and Arts Council England in their evaluation of how far CPP activity contributes to the aims of the programme, including but not limited to:

  • Communities are empowered to take the lead in shaping local cultural provision;
  • The aspiration for excellence is central to the activity we will support – this covers both excellence of art and culture and excellence of the process of engaging communities;
  • Through these projects, we will demonstrate the power of arts and culture to enrich the lives of individuals and make positive changes in communities;
  • Activity will be radically different from what has happened before in the eligible places prior to the Creative People and Places programme.

The Toolkit can be used to deepen your understanding of how well your intentions for your activity align with the experiences of your peers and your audiences.

 

 

Mandatory evaluation expectations for new Creative People and Places

It is a mandatory requirement for New Creative People and Places projects (those funded from October 2019) to undertake a minimum number of Impact & Insight Toolkit evaluations each year and share findings with their Relationship Manager via an agreed reporting template. There is a set of standards that these mandatory evaluations must adhere to. Existing Creative People and Places Projects that were funded prior to October 2019, that choose to use the Impact & Insight Toolkit should also adhere to these basic standards for any evaluations they agree to share with their Relationship Manager.

In each of the evaluation years (April 2020 to March 2021; April 2021 to March 2022; April 2022 to March 2023 and April 2023 to March 2024), all New CPPs must carry out at least four evaluations using the Impact & Insight Toolkit and, as part of their payment condition in May of each year, which states:

Submission of your annual report summarising your Impact and Insights Toolkit evaluations

Each project should submit an Annual Evaluation Report which shares their findings with their Relationship Manager. Guidance about how to create an evaluation report is available here.

The four evaluations should be representative of your programme across the year and can include:

  • Events: eg performances, festivals, exhibitions;
  • Creative Workshops and participatory experiences: eg craft workshops, singing and or music groups;
  • Decision Making Activities: eg community commissioning panels;
  • Co-created work: eg a large-scale event which has been co-created by the community;

As a minimum each evaluation must consist of:

  • one pre-activity Impact & Insight survey from a relevant member of staff outlining the creative intentions for the activity (we strongly recommend that you encourage more than one member of staff to complete this);
  • post-event Impact & Insight survey completed by a representative sample of the audience (this can be audience in the traditional sense or participants).

You may also wish to include:

  • one post-event Impact & Insight survey completed by relevant peers
  • post-event Impact & Insight survey completed by a relevant member of staff.

CPPs that do not have four events per year available to evaluate, should discuss suitable usage with their Relationship Manager.

 

Evaluation expectations for existing CPPs, funded before October 2019

There is no requirement for Existing CPPs, funded before October 2019 to use the Impact & Insight Toolkit, but if they do choose to use it we expect them to follow the standards set out here. If an existing CPP chooses to use the toolkit, they should discuss how many evaluations it would be appropriate for them to carry out each year with their Relationship Manager and share findings from those evaluations with their Relationship Manager, using the agreed reporting template.

 

Activity choice

A mandatory evaluation should involve an activity that can be evaluated using either the Core CPP Experience Dimensions for Audiences and/or participants (see below) OR the Core CPP Experience Dimensions for Core CPP Experience Dimensions for co-creators and/or community decision makers (See below).

We encourage CPPs to evaluate a range of their work. CPPs will find the toolkit most valuable if they use it to evaluate events where they can learn something interesting about the experiences of their audiences or participants, or where they have a particular hypothesis about programming or marketing that they wish to test. CPPs can discuss their event choices with their Relationship Manager if that would be helpful.

CPPs and NPOs can evaluate the same event, if it was a joint commission, however they must ensure that their joint evaluation meets the mandatory requirements for both an NPO and a CPP evaluation.

Beyond the four mandatory evaluations, all CPPs are welcome to undertake as many additional evaluations as they like.

 

Dimension choice

Core CPP Experience Dimensions for Audiences and/or participants

The following metrics are mandatory when evaluating things like:

  • Events: eg performances, festivals, exhibitions;
  • Creative Workshops and participatory experiences: eg craft workshops, singing and or music groups;

The questions should be asked of a random sample of a public audience. These mandatory questions could also be used in the context of an artist led creative workshop for participants, if they are relevant. If you are un-sure please check with your Relationship Manager. Please note these are the core mandatory questions, projects may choose to add any additional metrics you wish. The first three metrics are also mandatory for Band 2 and 3 National Portfolio Organisations.

Distinctiveness: It was different from things I’ve experienced before

Relevance: It had something to say about the world in which we live

Rigour: It was well thought through and put together

Local Impact: It’s important that it is happening here

Enthusiasm: I would come to something like this again

Cultural Contribution: It provides an important addition to the cultural life of the area.

Core CPP Experience Dimensions for co-creators and/or community decision makers

The following metrics are mandatory when evaluating decision making activities, for example like:

  • Decision Making Activities: eg community commissioning panels or ‘Go and See’ type events;
  • Co-created work: eg a large-scale events which has been co-created by the community / members of the public;

The questions would be asked of members of the panel or participants in a co-created project. Please note these are the core mandatory questions, projects may choose to add any additional metrics you wish.

Intention: I felt able to shape the intention of the project

Voice: My ideas were taken seriously

Experimenting: I felt comfortable trying new things

Intensity: I felt deeply involved in the process

Motivation: I felt motivated to do more things in the future

 

 

Sampling Guidance

Counting What Counts has produced guidance on appropriate sample sizes for different types of event, including participatory events which typically involve smaller groups of people.

 

Go back to Support Materials

 

The information on this page was last updated on 21 April, 2020.

X
X