Frequently Asked Questions
These FAQs have been compiled following correspondence with Impact & Insight Toolkit users, as well as questions that have been raised in Q&A sessions.
We have divided questions into several categories, to make it quick and easy to find the answer to your query:
Can I add my own questions to my mandatory evaluation surveys?
Yes, absolutely. We would strongly encourage you to carefully think about what else you might wish to add to the surveys and make those additions. In the Design page of your survey, you can find a large variety of content types including drop down and multiple choice to assist you in capturing what you’re interested in learning. You might wish to peruse the other standardised metric statements available. You can find these in the ‘Dimensions’ section of the Design page.
My organisation puts on less than 4 events per year; how can we carry out 4 evaluations?
This really depends on your organisation and the specifics of the events you are putting on. If your small number of events includes a tour, each location could have its own evaluation. This would enable you to compare how a work was received in different locations. In a festival situation, it could be that you choose to have a separate evaluation for different works. This way you would be able to compare a variety of works and see that they are all achieving their aims.
Are the response options in the gender section standard defaults or can I change them?
The gender question is inbuilt into the Culture Counts platform. The free-text question that follows has been added to enable self-definition when a respondent selects ‘In another way’. The question has been written considering guidance from Stonewall, ‘[a respondent] should be able to describe their gender in their own words. This signals an understanding that gender is a wider spectrum than female and male.’ The Impact & Insight Toolkit has been designed to capture data on the impact that work is having on your audiences, and not about the demographics of your audience – this is where Audience Finder comes in. That said, you are able to add your own questions to your survey, if you wish to. However, it is important to flag that if you choose to create a different question on gender, it won’t appear in the automated report for your Relationship Manager.
Can we use these surveys with children?
Yes, you absolutely can. All the dimensions have been written to a reading age of 11 and therefore it is likely that those above this age won’t require any assistance in understanding the questions. If you are asking younger children, they may need some assistance. We suggest that you refrain from asking any further personally identifiable questions and that you obtain permission from the adult(s) responsible for the children before surveying them.
Our audiences are not comfortable using tablet computers or PCs; can we do this on paper?
In short, the answer is ‘yes’. However, we would strongly encourage you to think about whether you definitely cannot obtain responses via interview or e-survey, or whether it just hasn’t been tried before with your audience. Using the Culture Counts platform directly to gather responses is preferable for a number of reasons, including: your data can contribute to the aggregate data set; you will be able to view visualisations in real-time on your dashboard; you will have time-stamps enabling you to compare people’s reactions over time. If you’re uncertain on which method might be best for you to use, or if you would like to know the process for arranging paper surveys, take a look at our guidance.
Does it matter when my events take place?
Not at all. As long as your 4 events for your Arts Council Toolkit evaluations fall into that year’s funding cycle, there is no problem. It might be that your organisation only puts on events over the festive period, or that you are holding an exhibition which runs for 5 months and it won’t close until February. That is all absolutely fine. As long as the events you’re evaluating have received funding for that financial year, there is no problem.
Does it matter if my events are related?
No. Toolkit evaluations should typically reflect the variety of the work that your organisation does. However, if you’re a touring theatre company and only have one production a year, touring to different locations, then you might choose for your four evaluations to each reflect a different location on the tour. Essentially, it is flexible. We recommend that your event choice reflects your offerings as an organisation in order for you to get the most out of the evaluation.
I have seen guidance that we should aim for a sample of 70-100 responses per event; this is unrealistic for my organisation and our events. Is it okay if we don’t meet these numbers?
Sampling is a very difficult subject to give a definitive answer to. The 70-100 figure is a suggestion that was taken from one school of thought on the matter. However, due to the unique nature of all the arts and cultural events, this figure won’t be attainable for some organisations. This is not a problem; the 70-100 figure is a suggestion and not a requirement. Each of your events is unique and there is no ‘one size fits all’ response. We would encourage you to gather as many responses as you can, aiming for them to be reflective of your audience.
Counting What Counts has produced guidance specifically on appropriate sample sizes for different types of events, including participatory events which typically involve smaller groups of people. The minimum number of public responses required in order to meet the mandatory requirements is one; however, it is advised that NPOs follow the guidance available to ensure maximum insight is gained.
If you would like more specific guidance, please do get in touch with your primary contact at Counting What Counts and we will happily advise you further.
Culture Counts platform
How do I setup another login for a colleague?
This isn’t something you have the facility to implement. We are able to create a new account for your colleagues; however, it is important that you know that their accounts will be entirely separate from yours. There is not a way for multiple email addresses to log into the one account.
Sometimes people choose to share one account across a whole team (we can change the login to a generic email address such as office@ if that makes you feel more comfortable) and other times people choose to have multiple accounts and use the Sharing Evaluation function so that they can share evaluations across the email addresses.
My password isn’t working but I haven’t changed it; what went wrong?
The most likely reason for this is that you are trying to access Culture Counts via the global site – culturecounts.cc. You will be directed to the global site if you Google ‘Culture Counts’ or possibly if you type in the web address from memory. In order to ensure you are on the correct side, you must either go via the Impact & Insight Toolkit website or you can visit the platform directly. Either way, what you will notice is that when you enter your login details, there is ‘uk.’ presented in the address bar, preceding ‘culturecounts’. It is this ‘uk.’ which indicates you are on the UK site, not the global one.
Why can’t I see self and peer responses in my Reporting dashboard?
Once you have collected your data, you will want to take a look at the results in your Reporting dashboard. You might notice that the system says you have no peer or self-assessors, and that the system has categorised them as public responses. This will most likely be because you used the incorrect links for your peer and self-assessors. The system does not know which survey in your evaluation is for which respondent group. It is down to you to ‘tell’ the system by either using the online/interview/display link (this will always record the response as public), or you will need to write the respondents’ email addresses in the boxes shown on the ‘Invite’ tab and then use the links that have been generated, unique to them. For more information on this, please see this video. There is also guidance in Creating an Evaluation.
In the Reporting Dashboard, why does the system say I haven’t gathered enough data to populate the Experiences and Expectations section?
This is probably occurring because either you haven’t ‘linked’ your prior and post surveys in the Configure tab of your surveys, or because you haven’t selected the appropriate time category (pre/post) for your surveys. Again, this can be found in your Configure tab. Just to confirm, the Self Pre-Event Survey should be recorded as ‘pre-event’; the Self and Peer Post Event Survey should be recorded as ‘post-event’; the Public Survey should be recorded as ‘standard’.
I am having problems with the platform in Internet Explorer… Why is this?
The Culture Counts platform functions optimally when accessed via a web browser other than Internet Explorer. Therefore, I would recommend that you use any other web browser (e.g. Firefox, Safari, Chrome…) to create your evaluations. The reason for this is that, whilst the browser is still being maintained, Microsoft is not developing Internet Explorer any further. Thank you for your understanding!
Audience Finder and the Toolkit
My organisation puts on very few events and/or has small audience sizes; we are struggling to meet our quota for Audience Finder. How should we go about adding another survey to the mix?
If you’re already struggling to reach your Audience Finder quota, you might be concerned that you will miss out on achieving your required Audience Finder quota now that you also need to distribute Toolkit surveys. Think carefully about your event choice; for instance, if you have some smaller, more intimate, events, this is where you might want to use Toolkit as your evaluation tool. You might then want to use Audience Finder at your large-scale events. Please see our guidance on managing Audience Finder with the Toolkit here. If you really don’t think you will be able to use the Toolkit without missing your Audience Finder quota, please speak to your Relationship Manager.
Why do I need to speak with both The Audience Agency and Counting What Counts?
The Impact & Insight Toolkit and Audience Finder are two separate surveying platforms, owned by two separate companies. Counting What Counts will support you as you strive to independently use the Impact & Insight Toolkit; The Audience Agency will support you as you set up your Audience Finder survey with them.
Are peers paid to review?
Anyone that has signed up as a peer reviewer as part of the Toolkit is not expecting payment for their review. Acting as a peer reviewer is an opportunity for them to experience a variety of works and to explore how their own practice relates to what others in the field are doing. Of course, it is also good for their professional development and they will know that they are contributing to an Arts Council England project.
What about expenses?
It is between both parties, the receiving organisation and the peer, to determine who will cover expenses. It might be that a peer reviewer contacts the receiving organisation to establish these details before accepting their invitation, in order to confirm their understanding.
As a registered peer reviewer, how do I log into Culture Counts?
When you register as a peer reviewer, you are not registering for a Culture Counts account; you are simply registering your details so that those using the Toolkit can contact you for peer review purposes. Therefore, as a peer reviewer, you do not need to log into the Culture Counts platform at all. If you need to amend or wish to remove your peer reviewer profile, please do get in touch via firstname.lastname@example.org and we will gladly make any alterations necessary.