Frequently Asked Questions

1. Mandatory requirements

What are my funding requirements?

Your funding requirements are based on your Arts Council England (ACE) funding band or organisation type. Click on the relevant band or type below to see your requirements:

National Portfolio Organisation (NPO) – Band 1
National Portfolio Organisation (NPO) – Band 2 and 3
Creative People and Places (CPP)
Sector Support Organisation (SSO)

If you are not sure which band or type you are, please check your ACE funding agreement.

Can I add my own questions to my mandatory evaluation surveys?

Yes, you can add your own questions in the ‘Design’ section of the platform.  Watch our step-by-step Designing your Surveys YouTube video for more information.

2. Mandatory requirements (Band 2 and 3 NPOs)

What is the difference between a ‘Core’ and ‘Flexible’ evaluation?
Can I add or remove demographic questions?

‘Core’ evaluations must include standard demographic questions.  ‘Flexible’ evaluations can have the standard demographic questions removed.  You are welcome to add additional demographic questions to both ‘Core’ and ‘Flexible’ evaluations.

My organisation puts on less than four works/events per year; how can we carry out four evaluations?

This depends on your organisation and the kind of works/events you put on. If your works/events include a tour, each location could have its own evaluation. This would enable you to compare how a work/event was received in different locations.

If one of your works/events is a festival, you could create separate evaluations for individual works. This way you would be able to compare a variety of works happening at the festival and see if they are all achieving their aims.

It’s also worth remembering that a suitable work doesn’t need to be in-person. If you are delivering work online, this can also contribute to your mandatory evaluation count. If you still do not think you will have enough works, please speak with your ACE Relationship Manager.

Does it matter when my works/events take place?

The works you are evaluating need to take place during the current financial year (1st April – 31st March). When they take place within the financial year is up to you, it might be that your organisation only puts on works over the festive period, or that you are holding an exhibition which runs for five months.

How many self-assessors should evaluate a work/event?

To meet your mandatory requirements, you need at least one self (prior) and self (post) assessor response. However, to get the most out of the Toolkit, we recommend aiming for three to five self (prior) and self (post) assessor responses.

What do I need to do once I’ve completed an evaluation?

Once you have completed an evaluation:

3. Accessibility

Can we use paper surveys?

Yes, but please be aware that you will need to input the data manually using our ‘Data Import’ feature.  Watch our step-by-step Using the Data Import Feature video and read our Carrying out a Paper Survey guidance for more information.

Are the response options in the gender section standard defaults or can I change them?

The gender question is inbuilt into the Culture Counts platform.  The free-text question that follows has been added to enable self-definition when a respondent selects ‘In another way’.  The question has been written considering guidance from Stonewall provided shortly before the commencement of the project, ‘[a respondent] should be able to describe their gender in their own words.  This signals an understanding that gender is a wider spectrum than female and male.’[1]  Of course, if, for your specific work or organisation, it is imperative that you make available a long list of genders, you could use the logic feature so that, if someone selects ‘In another way’, they are directed to a follow-up question containing the long list to choose from. [1] Stonewall (2016). Do ask, do tell: capturing data on sexual orientation and gender identity globally.

Can we use these surveys with children?

Yes, the dimensions have been written to a reading age of 11. If you are asking younger children or people with additional needs, they may need some assistance. We suggest that you don’t ask any further personally identifiable questions and that you obtain permission from the adult(s) responsible for the children before surveying them. Depending on the exact configuration of your evaluation, it might be that you choose to restrict the demographic questions asked of those under 18, perhaps only asking age and gender. Depending on your funding requirements, this may only be suitable if you are also collecting data from adults who would be able to provide the location.

4. Work/Event Choice

How many evaluations can I do?

As many as you like! Just make sure, if you are looking to meet ACE funding requirements, you at least meet your mandatory requirements.

Who is responsible for setting up evaluations?

This will span across multiple departments within an organisation.  Often people working in Marketing or Communications manage the account and set up the evaluations.  However, they should involve the Programming/Artistic team as self-assessors, and with any decisions around additional questions being asked if they’re to do with the creative or artistic intentions; for instance, the addition of any dimensions.  Results from the evaluation should be considered across the organisation and reflected upon in team meetings, board meetings etc. For more information on how to approach the results across your organisation, we recommend looking at the blogpost, ‘What’s Your Question?’.

What type of works/events can I evaluate?

This can vary, from a collaborative workshop to live performance, the main point is that there must be intention behind the work. You can evaluate works that are online, outdoors, touring… The list goes on!

Does it matter if my works/events are related?

Toolkit evaluations should typically reflect the variety of the work that your organisation does. However, if you’re a touring company and only have one production a year, touring to different locations, then you might choose for your evaluations to each reflect a different location on the tour. We recommend that your work/event choice reflects your offerings as an organisation in order for you to get the most out of the process.

Can I evaluate a participatory work/event?

Yes, watch our step-by-step Creating a Participatory Evaluation YouTube video for more information.

Can I do a public pre-event survey for my participatory work/event?

Yes, however, comparisons between pre-event and post-event public data are not currently supported by the platform; you will need to download a csv file of the data and compare it using software such as Microsoft Excel. Watch our step-by-step Evaluating Participatory Works series for more information.

Is there a set sample size?

No, your sample size will depend on the nature of the work you are evaluating.  Please read our Sample Size Guidance for more information on appropriate sample sizes for different types of works/events, including participatory works/events which typically involve smaller groups of people.  The minimum number of public responses required in order to meet the mandatory requirements is one; however, it is advised that users follow the guidance available to ensure maximum insight is gained.

5. The Culture Counts Platform

How do I send a survey to self and peer reviewers?

Watch our step-by-step video on Using the Invite Page or read our written Inviting Peer and Self Assessors to the Survey guidance.

When should self-assessors fill in the self (prior) survey?

We recommend completing self (prior) surveys after the work/event has been planned, but before production starts. You might consider completing prior surveys for multiple works/events at the same time, allowing you to easily compare works/events.

Are responses collected if someone doesn’t complete the survey?

Yes, as long as you have an active internet connection, all responses (even incomplete ones) are recorded.

What’s the difference between Insights Reports and the Analytics dashboard?
  • Insights Report – An Insights Report is an auto-generated report that allows you to see the alignment between self (prior), public and peer data.  Depending on your funding requirements, the Insights Reports may need to be generated within one month of completing an evaluation and emailed to your ACE Relationship Manager.  Watch our Generating Your Insights Report video or read our written Creating an Insights Report guidance for more information.
  • Analytics dashboard – The Analytics dashboard allows you to see both your overall evaluation data and individual survey data.  The Analytics dashboard does not need to be used to meet your funding requirements.
Can I download charts from the Analytics dashboard?

Yes, to download individual charts, click the chart’s ‘download’ button.

Download button

To download all the charts, click the ‘export evaluation’ button.

Export Evaluation button

Do I need to link my surveys using the ‘link survey’ feature?

No, the ‘link survey’ feature is an old way of copying questions between prior and post surveys.  We no longer recommend linking surveys.

Can I delete survey responses?

No, you cannot delete survey responses.  If you accidentally answer a live survey, get in contact support@countingwhatcounts.co.uk and we can delete the response for you.

Can I import external data?

Yes, to import external data use our ‘Data Import’ feature.

Can I have multiple people signed up for the Toolkit?

We can create new accounts for your colleagues; however, their accounts will be separate from yours.  Multiple email addresses cannot log into one account.  Some organisations share one account across a whole team (we can change the login to a generic email address such as office@ or info@) and others choose to have multiple accounts and use our ‘share’ feature to share evaluations across the separate accounts.

Why can’t I see self and peer responses in my Insights Report?

The two most common reasons why self and peer data are not appearing in your Insights Report are:

1. You used the incorrect links for your self-assessors and peer reviewers. The system does not know which survey in your evaluation is for which respondent group.  It is down to you to ‘tell’ the system by either using the online/interview/display link (this will always record the response as public), or by using the ‘Invite’ tab (this creates unique links for your self-assessors and peer reviewers).  For more information on how to correctly invite self and peer assessors, watch our step-by-step video on Using the Invite Page or read our written Inviting Peer and Self Assessors to the Survey guidance.

2. You haven’t selected the correct ‘survey type’. There are three survey types: standard, post and prior. You can change a survey’s ‘survey type’ in the ‘Configure’ tab.  Please make sure each survey has the appropriate survey type:

  • Post-Event Survey for Self and Peer Respondents – ‘Post’ survey type
  • Pre-Event Survey for Self Respondents – ‘Prior’ survey type
  • Survey for Public Respondents – ‘Standard’ survey type

Three survey types: prior, post and standard

Why is my password not working?

The most common reason your password is not working, is that you are trying to access Culture Counts via the global site – culturecounts.cc.  You will be directed to the global site if you Google ‘Culture Counts’.  To login to the correct site, either go via the Impact & Insight Toolkit website or visit the Culture Counts platform directly.

I am having problems with the platform in Internet Explorer… Why is this?

We’re phasing out support for Internet Explorer – find out more about recommended browsers here.

6. Audience Finder and the Toolkit

How does the Impact & Insight Toolkit work with Audience Finder?

Impact & Insight Toolkit and Audience Finder are completely separate tools and run by different companies. The Impact & Insight Toolkit measures your creative intentions, audience and peer experience, whereas Audience Finder is helping to research the demographic of your audience.

However, Arts Council England is enabling some organisations to collect Audience Finder results in the Impact & Insight Toolkit.  To find out more, read Arts Council England’s Collecting Audience Finder data in the Impact & Insight Toolkit webpage.

My organisation puts on very few works/events and/or has small audience sizes; we are struggling to meet our quota for Audience Finder. How should we go about adding another survey?

If you’re already struggling to reach your Audience Finder quota, you might be concerned that you will miss out on achieving your required Audience Finder quota now that you also need to distribute Toolkit surveys. Think carefully about your work/event choice; for instance, if you have some smaller, more intimate, works/events, this is where you might want to use Toolkit as your evaluation tool.  You might then want to use Audience Finder at your large-scale works/events. Please see our guidance on Working with Audience Finder and the Impact & Insight Toolkit.  If you really don’t think you will be able to use the Toolkit without missing your Audience Finder quota, please speak to your ACE Relationship Manager.

7. Peer Review

How many peer reviewers should I invite?

We recommend inviting as many as you can because some peer reviewers might not be able to make the day and time of the work/event.  Toolkit users generally receive around two to three responses – the more the better!

Do I have to invite peers through the Peer Matching Resource?

No, use of the Peer Matching Resource is optional; you are welcome to invite peers from your own professional network.

Is the organisation expected to pay for peer expenses?

Paying for peer expenses is optional.  When inviting peer reviewers, we recommend communicating clearly whether you will be covering expenses (e.g., travel, work/event tickets).  Please be mindful when selecting peers, of where they are based, as this may affect costs (e.g., travel).

As a registered peer reviewer, how do I log into Culture Counts?

When you register as a peer reviewer, you are not registering for a Culture Counts account; you are simply registering your details so that those using the Toolkit can contact you for peer review purposes.  Therefore, as a peer reviewer, you do not need to log into the Culture Counts platform at all.

Can I edit or remove my peer data on the Peer Matching Resource?

You are not able to edit or remove your peer review profile yourself.  If you need to amend or wish to remove your profile, please get in touch via support@countingwhatcounts.co.uk and we will make any alterations necessary.

8. Other Resources

Can I benchmark against other arts and cultural organisations?

Yes, we have two tools available that can be used for benchmarking and exploring the dataset:

Where does the data go once it is submitted to ACE?

When you ‘Submit’ the evaluation data, you are agreeing for certain data within this evaluation to contribute towards the public-facing, anonymised, aggregate dataset.  Data that will be submitted comprises: dimensions data, demographic data and data properties. This means that data from custom questions will not contribute to the public-facing, anonymous aggregate dataset.  The anonymised aggregate dataset powers two tools for you to explore, helping you and others to understand more about the impact of different arts and cultural experiences in the UK: the Benchmarking Dashboard and the Insights Dashboard.  All data collected and processed on behalf of Arts Council will be done adhering to their Data Principles.  

Powered by BetterDocs

X
X