Arts Council knows that because of COVID-19, the audience data and insights NPOs collect might be limited, so they’ve produced some guidance to help NPOs know their expectations.Please click here to see the guidance for NPOs regarding their use of the Toolkit for the financial year 2020-21
Arts Council England – Mandatory Requirements for National Portfolio Organisations Using the Impact & Insight Toolkit
The information on this page was written by Arts Council England for NPOs using the Impact & Insight Toolkit. View/download a PDF version of this page here.
- Mandatory evaluation expectations for band 2 and 3 NPOs
- Evaluation expectations for band 1 NPOs
- Event / activity choice
- Dimension Choice
- Sampling Guidance
It is a mandatory requirement for band 2 and 3 National Portfolio Organisations (NPOs) to undertake a minimum number of Impact & Insight Toolkit evaluations each year and share findings with their Relationship Manager via an agreed reporting template. There is a set of standards that these mandatory evaluations must adhere to. Band 1 NPOs that choose to use the Impact & Insight Toolkit should also adhere to these basic standards for any evaluations they agree to share with their Relationship Manager.
Mandatory evaluation expectations for band 2 and 3 NPOs
In each of the evaluation years (April 2019 to March 2020; April 2020 to March 2021; and April 2021 to March 2022) of the Impact & Insight Toolkit, all band 2 and 3 NPOs must carry out at least four evaluations using the Impact & Insight Toolkit and share findings with their Relationship Manager. Counting What Counts will work with the Arts Council and NPOs to develop a standard reporting template for mandatory evaluations. As a minimum each evaluation must consist of:
- one pre-event Impact & Insight survey from a relevant member of staff outlining the creative intentions for the work (ideally NPOs should engage three to five staff for each mandatory evaluation where possible)
- one post-event Impact & Insight survey completed by relevant peers (ideally NPOs should engage three to five peers for each mandatory evaluation
- one post-event Impact & Insight survey completed by the same staff members
- one post-event Impact & Insight survey completed by a representative sample of the audience
Band 2 and 3 NPOs that do not have four events per year to evaluate, such as biennials or those with a limited public programme, should discuss suitable usage with their Relationship Manager.
Evaluation expectations for band 1 NPOs
There is no requirement for band 1 NPOs to use the Impact & Insight Toolkit, but if they do choose to use it we expect them to follow the standards set out here. If a band 1 NPO chooses to use the Toolkit, they should discuss how many evaluations it would be appropriate for them to carry out each year with their Relationship Manager and share findings from those evaluations with their Relationship Manager, using the agreed reporting template. As a minimum each evaluation should include:
- one pre-event Impact & Insight survey from a relevant member of staff outlining the creative intentions for the work
- one post-event Impact & Insight survey completed by the same staff
- post-event Impact & Insight survey completed by a representative sample of the audience
As a guide, it may be appropriate for a band 1 NPO to conduct one formal evaluation that uses the Core Cultural Experience or Participatory Dimensions and adheres to the guidance above. Findings would then be shared with their Relationship Manager.
Event / activity choice
A mandatory evaluation should involve a public-facing event that can be evaluated using the Core Cultural Experience Dimensions (see below). Where an organisation is primarily participatory, or has a relevant piece of participatory work it wants to evaluate, the NPO should discuss with their Relationship Manager whether some or all of their mandatory evaluations can be built around the Participatory Dimensions instead.
We encourage NPOs to evaluate a range of their work (e.g. theatre productions in a main house and studio). NPOs will find the toolkit most valuable if they use it to evaluate events where they can learn something interesting about the experiences of their audiences, or where they have a particular hypothesis about programming or marketing that they wish to test. NPOs can discuss their event choices with their Relationship Manager if that would be helpful.
Touring and receiving NPOs can use the Culture Counts platform to carry out a shared evaluation of the same event. Each organisation will need to submit a separate report on the event to their Relationship Manager; this will count towards the quota of mandatory evaluations for both organisations.
Beyond the four mandatory evaluations, all NPOs are welcome to undertake as many additional evaluations as they like. There is no requirement to share these evaluations with the Arts Council, and they can be designed in any way that NPOs choose.
NPOs must use the following Core Cultural Experience Dimensions for any public-facing mandatory evaluation:
Self, Peer and Public
Concept: It was an interesting idea
Distinctiveness: It was different from things I’ve experienced before
Challenge: It was thought-provoking
Captivation: It was absorbing and held my attention
Relevance: It has something to say about the world in which we live
Rigour: It was well thought through and put together
Self and Peer only
Risk: The artists/curators really challenged themselves with this work
Originality: It was ground-breaking
Excellence: It’s one of the best examples of its type that I have seen
Counting What Counts have produced a Toolkit User Guide which offers guidance on appropriate sample sizes for different types of event, including participatory events which typically involve smaller groups of people. As a general rule of thumb NPOs should aim for 70-100 public responses per event.
The information on this page was last updated on 21 March, 2019.