FAQs We understand that using a new tool can be confusing, so we’ve provided these FAQs to help you get to grips with it all. Not found your answer? If you’ve got a question you’d like us to answer, please get in touch. Confused by any of the language we use? We try to avoid jargon where we can, but if you need help unpacking any terms, please see our Glossary page. 1. Mandated Use and Arts Council England Requirements Is it mandatory to sign up to the Impact & Insight Toolkit and use Culture Counts? More For the majority of Art Council England (ACE) funded organisations, it is not mandatory to sign up to the Impact & Insight Toolkit (Toolkit) and use Culture Counts. Although registering to the Toolkit is not mandated, it is a requirement of you to develop an evaluation framework and to evidence your commitment to fulfilling the Ambition and Quality Investment Principle (IP). The Toolkit has been developed to directly support ACE-funded organisations in evidencing their commitment to the Ambition and Quality IP and, therefore, the Toolkit will be a very useful resource in fulfilling the IPs for many of you. Please know that this is not the case for those in the Creative People and Places (CPP) portfolio, for whom Toolkit registration and Culture Counts use is a funding requirement. If you are a CPP, please look at the appropriate funding requirements guidance to ensure you are meeting these. If we choose to not participate in the Toolkit project, how do we go about developing an evaluation framework? More Whilst we are not familiar with other approaches or tools that might support you in developing your framework or evidencing ambition and quality, our blogpost on evaluation frameworks would be a good place to start. If use isn’t mandated, does data collected get sent to Arts Council England in some way? More In short, the answer to this question is ‘no’. The new Toolkit is more about engaging your board and Senior Management Team in understanding evaluation and the data that’s collected, than reporting to ACE. If you use Culture Counts, you will be provided with Ambition Progress Reports and will also be able to generate Insights Reports. These reports should fuel board meetings and provide a conversation kick-starter. Minutes taken at your meetings will need to be sent to ACE and it might be that your Relationship Manager also joins a board meeting. Whilst you do not need to send reports or data to ACE, they will expect to see that you are discussing the Investment Principles and utilising data collected. So, who is the data owner? More You are! Culture Counts is a data processor. The user of Culture Counts, you, owns the data. It is not owned by Arts Council England. If we register to the Toolkit project, are we committing to use Culture Counts? More No. Registering to join the Toolkit project will give you access to evaluation support, guidance and resources, as well as the Culture Counts platform. If you choose to not use Culture Counts but do want to engage with other aspects of the Toolkit project, that is fine. 2. Culture Counts and Account Management If we have participated in the ‘previous’ Toolkit project (2019-2023), and have used Culture Counts, do we still need to register to use the new Toolkit? More Yes. The ‘new’ Toolkit project is based on a ‘new’ contract and therefore you will need to sign up. If we have participated in the ‘previous’ Toolkit project (2019-2023), and have used Culture Counts, will my previously collected data still be accessible? What about if I am running an evaluation over the transition period? More Any data collected as part of the ‘19-23 project can still be accessed via your Culture Counts account. No data will be ‘lost’, and your account will still be fully functioning. This means that any evaluations that are currently running will continue to collect data. Please know that if you do not transition to the new Toolkit project, access to Culture Counts will be discontinued from May 31st 2023. In addition to utilising inbuilt dimensions and templates, are we able to add in our own questions to surveys on Culture Counts? More Yes, absolutely. One of the good things about Culture Counts is that it is a flexible tool. Therefore, you can create evaluations based solely on the inbuilt questions, ‘write your own’ questions, or use a combination of both. It really is up to you. If our organisation registers, do we just get one login to Culture Counts? More As part of the new Toolkit project, there is a ‘multiuser’ feature, which means that each organisation can have up to three logins. You can choose to collaborate on evaluations or to work separately. Please know that only one person from an organisation needs to register. Are we able to access results and reports in an automated way through Culture Counts, or do we need to wait for reporting from Counting What Counts? More Counting What Counts (CWC) is going to be providing users with reports on, most likely, a quarterly basis. We do not know exactly what these will look like and are keen to engage with ACE-funded organisations to ensure that these reports are genuinely insightful and helpful. However, you can generate automated reports through Culture Counts at any time. These are called Insights Reports. You can also extract the raw data any time you like. Can we export data collected in Culture Counts and use it outside of the platform? More Yes, there is a CSV download feature. This feature extracts all data relating to a specific evaluation. You can then manipulate the data in a spreadsheet and use it as you see fit. Some people might choose to use their data to power interactive dashboards, such as Power BI; others might choose to upload data to other platforms. Can we import data into Culture Counts that we have collected outside of the platform? More Yes, there is a CSV import feature. After setting up the evaluation in Culture Counts, you download a template. You populate this template with your data collected elsewhere and then import. Once we select and write questions for our evaluations, are we able to change these for different projects or are we wedded to those initially chosen? More You can absolutely change questions if that’s what you choose to do. We recommend that you have different dimension ‘lists’ for different types of work (e.g., you could have a participatory-works list and a presentational-works list). You can then select whichever is most appropriate and, of course, add and/or remove questions as you wish. 3. PwC and Illuminate; The Audience Agency and Audience Finder Does the Toolkit replace The Audience Agency’s Audience Finder? More No. The Toolkit responds to a completely different contract set out by ACE. PwC has been commissioned to offer a data collection service and to produce audience development focussed insights; this service is called Illuminate. What is the difference between Illuminate and the Toolkit? More The objective of both the Toolkit and Illuminate is in providing target data insight to arts & cultural organisations. Whilst both projects involve some similar data collection tools, the purpose behind the data insight is completely different. This means that the way the data is analysed and presented, and the questions that it can help organisations to answer, is unique for the two projects. In a nutshell, Illuminate is about understanding who your work reaches; the Toolkit is about understanding how your works contribute to you achieving your overall ambition. I have more questions about Illuminate! More We would recommend that you: Explore Arts Council England’s webpages on Illuminate Contact PwC (the team delivering Illuminate) via: Email – [email protected] Telephone – +44 800 031 8671 4. Accessibility, Touring and Other Complexities Are we able to use paper surveys? More The Culture Counts platform is an online, digital, evaluation tool. Whilst it is possible to collect survey responses via paper surveys, it is not recommended as transposing and importing the data to Culture Counts can be resource heavy. Are we able to change the wording of inbuilt questions? More If you want to use the inbuilt questions to Culture Counts, the wording is fixed. This is to enable research and analysis further down the line; these will also help fuel your reports. You can add in your own custom questions if the inbuilt ones are unsuitable. How do we use Culture Counts if our audiences have disabilities and/or are neurodiverse? More Firstly, it is important to remember that it is not mandatory for most ACE funded organisations to use this tool. There may be others available that are more appropriate for your organisation and audience. If you do use this tool, you can write your own questions. The platform supports the use of emojis in its surveys which we have found to be quite popular with some audiences. Can we do a mixed-method approach to support our audiences? More Yes, of course! It might be that you want to collect online, digital responses from a certain subset of your audience, whereas a more interactive and/or physical method of collecting feedback might be more appropriate for others. You could then transpose the feedback into a method suitable for Culture Counts, meaning that all data is kept in one place. What accessibility needs have been taken into account with Culture Counts? More The surveys generated through Culture Counts are AA accessibility certified. This means that they are screen reader accessible and that the contrasting used is considered appropriate. The inbuilt questions are written to a reading age of 12, meaning that most young adults and adults can respond to these questions. You can write in your own questions if those inbuilt are not appropriate. The platform also supports the use of emojis, which has proven to be quite popular. Peer review is considered to be a positive thing in evaluation and is encouraged in the Toolkit, but how do we find suitable peers for our work? More You should look in your own professional network and give the Peer Matching Resource a try. Firstly, a peer reviewer can be anybody who is not involved in the curation or creation of your work AND whose professional opinion you would respect on it. Therefore, a suitable peer reviewer for one work might not be suitable for a different work. A peer reviewer does not need to work in the same artform as the work that’s being reviewed; a peer reviewer does not need to work in the arts and cultural sector. What’s important is that you believe that their professional perspective on your work will provide value to your understanding of its effectiveness. The Peer Matching Resource is an inbuilt resource to Culture Counts, helping you to connect with those across the sector that have expressed an interest in being a peer reviewer. Using this resource is optional; you could connect with peers through your pre-existing professional networks. We don’t have access to box office data or Internet access – how do we collect responses? More There is an offline data collection feature, built into Culture Counts. This means that you can digitally collect 1 to 1 interviews in your space that does not have Internet. When you go somewhere that does have Internet access, you can sync the responses collected to your Culture Counts account. As a touring company, how do we approach data collection at host venues? More If the touring company and host venue are both ACE-funded, this can be helpful as both parties will likely be familiar with the Toolkit and, if they were band 2 or 3 NPOs in previous funding rounds, they will already know how to use Culture Counts and understand how the data is saved and presented. There are a few tips we have: Communication – openly explain what you need and why you need it Clarity and understanding – people generally find that having a data sharing agreement in place is useful to ensure that there is clear understanding Compromise – when it comes to choosing the dimensions and other questions to include, you will likely need to compromise to ensure that all parties achieve data they need Be creative in data collection – even if you can’t access box office data, you could do things like sharing QR codes on event programmes or posters; you may be able to conduct 1 to 1 interviews ‘on the ground’ Share evaluations and data – all collected data is anonymised, meaning that there is no reason that data cannot be shared. If both parties have a Culture Counts account, evaluations can be ‘shared’ between parties, meaning that they both have access and can use the data accordingly.