With the end of the 2021-22 evaluation year fast approaching, we wanted to use the time to thank the cultural organisations that have participated in one (or more!) strands of the Impact & Insight Toolkit’s development programme:
- Strategic Development Strand
- Artform & Museum Metric Strand
- Participatory Metric Strand
Your knowledge and expertise have been invaluable in informing the direction of the Toolkit since its launch in 2019 and we hope you continue to collaborate with us in 2022.
Unsurprisingly, the pandemic meant that activity planned for the three strands of the Toolkit’s development programme was significantly scaled back in 2020 and 2021. Moreover, to ensure that the Toolkit was able to support the sector during this turbulent time, Counting What Counts identified and prioritised new and more urgent areas of work, such as collaborating with cultural organisations on:
- creating an evaluation template, guidance and question bank specifically for organisations that suddenly had to shift their creative programme online
- producing a question bank and guidance for evaluating the increasing number of outdoor events
- creating a dimension category for evaluations focussed on placemaking events
Whilst, much like the rest of the country, we needed to rethink activity planned for 2020 and 2021, we’re pleased to confirm that we were able to complete many of the original tasks and areas of improvement identified. So, we thought now would be the perfect time to give you an update on the development programmes’ outputs that you have helped progress. Scroll down to the bottom of this blogpost to check out the completed tasks!
How can you get involved with development work in 2022?
As we approach the start of the evaluation year 2022-23, we remain committed to collaborating with cultural organisations and ensuring that their experience and skills continue to lead the Toolkit. We realise that evaluating your programme and understanding how people experience your work is more important than ever before and we want to ensure that the Toolkit continues to support you with this.
We recognise that the sector is in a very different place now than when the Toolkit launched in 2019 and it is vital that any development activity we undertake in 2022 remains relevant to the sector’s current needs.
We are working on our research plan and many of the remaining tasks identified in 2019 i.e., the Data Analysis tasks, will be folded into that. As the Toolkit is now being used by many more organisations such as Creative People and Places (CPPs), Museum Development Network (MDN) and project grant recipients, we have decided to announce and invite you to participate in development activities through open calls in our monthly newsletter and social media. This will ensure that any user of the Impact & Insight Toolkit will have the opportunity to be involved in developments, refinements and improvements.
As ever, arts and cultural organisations drive the Toolkit and it is imperative that you continue doing so, making it the best it can be!
- Arts and cultural organisations took part in a series of workshops to design and improve the peer survey.This improved survey is ready to use and is included in the template called NPO Toolkit Users: Starting point (inc. recommended peer survey) Read more here.
- Cultural organisations were invited to test and give feedback on the beta version of the peer reviewer database, which has become the Peer Matching Resource.Read more here.
Measuring Equality, Access and Inclusion
- Ju Gosling of Together! 2012 very generously shared their Metrics for Equality, Access and Inclusion so that other cultural organisations could include these in their evaluations. Read more here.
Data Analysis and Interpretation
- Arts and cultural organisations took part in user testing sessions for both the original Insights Report template and the improved Insights Report template.
- Impact & Insight Toolkit users were invited to attend a series of data-focussed webinars; recordings of these are linked to below:
- Counting What Counts began working with a network of arts and cultural organisations on a touring trial.The trial addressed key concerns raised by Touring companies, including:
- How best to get arts and cultural organisations and non-NPO venues working together
- What might a ‘whole tour’ data analysis look like, and how could that create greater buy-in from all the venue stakeholders involved in touring work
- Counting What Counts worked with The Audience Agency and NPOs to understand whether the Audience Finder and core Toolkit surveys could be combined and what effect, if any, this would have on user experience and data quality.The outcome of this trial is an offer, available to some arts and cultural organisations. This offer is presented on the Arts Council England’s website.
Artform and Museum Metrics
- Arts and cultural organisations and Artistic and Quality Assessors (AQA) created artform and museum specific dimensions which have now been added to the platform for you to use. Please read our blogpost here.
- At the Artform & Museum Metric Strand workshops in November 2019, it was clear that arts and cultural organisations wished to include sentiment questions in their evaluations. This proved to be a substantial piece of work and Counting What Counts took some time to thoroughly research and consider existing sentiment frameworks and identify one that would be suitable for the Toolkit. In January 2022 we were pleased to share our work with you and announce the addition of sentiment questions to the platform. To learn more, please read our series of blogposts and guidance.
- Organisations taking part in the Participatory Metric Strand attended workshops in 2019 where they worked to refine existing participatory dimensions down to a core set of 21 dimensions. Read more here. Counting What Counts will now monitor dimension usage with the aim of refining these down further.