Creating Your First Evaluation

Having received your login details from the Counting What Counts team, you will be able to access the Impact & Insight Toolkit. Click ‘Sign In’ at the top of this page to access your account on the Culture Counts platform.

Culture Counts Sign in page

Once you’ve logged in to your account, this guide will take you through the steps required to create a new evaluation and make your first survey.  Jump to:

What is an Evaluation?

Evaluations are folders or containers that contain all the information for a particular event, project or place that you wish to evaluate. If you wish to create several related surveys to evaluate an event, they should all be grouped and organised within a single evaluation. The evaluation will store and present all the data collected through those surveys so that you can quickly see what an event is achieving overall.

Creating an evaluation by copying the evaluation template

After you log in, you will see that we have created an evaluation template to give you some insight into how you can structure your evaluation using the Impact & Insight Toolkit. The template contains the Core Cultural Experience Dimensions that band 2 and 3 NPOs are required to use for Arts Council mandatory evaluations that will be shared with their Relationship Managers. As an easy starting point for your evaluation activity, you can copy and then edit this evaluation template, or of course you can create your own evaluation from scratch.  We recommend that you try both and see what suits you best.

Key Survey Types

There are three primary respondent groups that we suggest you consider when designing your evaluation: public/participant assessors, self-assessors and peer assessors. A detailed explanation of these three categories can be found in the User Guide. Correspondingly there are three types of surveys that can be created within an evaluation: a ‘standard’ survey (for the public to respond to), a ‘prior’ survey (for the self assessor to respond to) and a ‘post’ survey (for both peer and self assessors to respond to). Our evaluation template contains one standard survey for public assessors, one prior survey for self assessors and one post survey for self and peer assessors. We encourage you to browse the surveys in the template to see how they have been set up.

Please note that if you would like to preserve the template to work from in the future, you should not make or save any changes to the surveys within the evaluation template itself. Instead, please copy the surveys before you start editing them.

Copying the Evaluation Template

To copy the evaluation template, first create a new evaluation for your event. This will be a new container where the copied surveys can be stored. To start a new evaluation, click the New evaluation button at the bottom of the Evaluations panel. You will be prompted to give your evaluation a name (normally the name of the event you are evaluating) before being taken to the evaluation overview screen, shown below.

example Culture Counts evaluation

To copy the evaluation template surveys into your own new evaluation, go back to the template so you can see the three example surveys described above. Copy the example public survey by clicking Edit to the right of the survey name and selecting Copy from the dropdown list. Replace the copied survey name with a new name that reflects your event and choose your new evaluation to copy the survey into. We recommend giving each survey a simple name that both people in your organisation and assessors will understand, like the name of the event and the survey type: ‘Exhibition A – Public Survey’ or ‘Exhibition A – Prior Survey for Self Assessors’. You will need to repeat the process above to copy each of the three example surveys into your new evaluation.

From your new evaluation, you can edit these surveys however you wish, but now you have a basic survey to build on. Click the name of the survey that you wish to edit, and you will be taken to the survey Summary page, which shows a brief summary of the survey settings and questions included. From here you can navigate to different pages in the survey builder by clicking the Manage button in the top right corner, or clicking on one of the tabs in the top menu bar (dark grey). The editable pages include:

  • Configure – to change the survey settings
  • Design – to edit, add or remove survey questions
  • Invite – to add or remove self and peer assessors

*Please note, the links above will take you away from the Impact and Insight Toolkit and to YouTube. Please ensure you are happy for this to happen before clicking and have familiarised yourself with their cookie and privacy policies.

 

Creating an Evaluation from Scratch

To start a new evaluation, click the New evaluation button at the bottom of the Evaluations panel. You will be prompted to name your evaluation (normally the name of the event or place you are evaluating), before being taken to the evaluation overview page. From here you can create new surveys, manage existing surveys and download any data that has been collected for the evaluation. To create a survey for your new evaluation, click Create survey. You will then be taken through the three main stages of survey creation:

  • Configure– to change the survey settings
  • Design – to edit, add or remove survey questions
  • Invite– to add or remove self and peer assessor

Configuring Your Survey

The Configure page is where you set up the following options for your survey.

Survey Configure page on Culture Counts

Name

This is the survey name that respondents will see when they access the survey.

Start and Close Dates

The start and close dates will determine when the survey begins and stops taking entries. It is fine to leave the start and close dates blank. If you leave the start date blank, the survey will be available to record responses immediately, and if you leave the close date blank the survey will remain active indefinitely. When inputting the start and close date, we advise that you include an additional one or two days either side of the official start and close dates of your event to ensure that you capture all available survey responses.

Survey Introduction

Here you can write a short optional introduction to your survey that will appear on the front splash page of your survey. This introductory front page will only appear on the Online and Display survey delivery methods, as interviewers delivering the survey by using a Tablet can verbally provide this information to respondents. This introduction is good place to provide a brief explanation to respondents, such as: We’d love to hear what you thought. Please share your views on the quality of this event via our short survey. Your feedback will help us to understand and measure the impact of our events.

Custom Branding

Here you can upload your own logo, which will appear on the front page with the survey introduction text.

Survey Types

There are three survey types: Standard, Prior and Post.

  • Standard Public Survey: The standard survey is for your general public audiences or visitors. Audiences are typically asked to complete a standard survey after experiencing an event or visiting a place, to measure the perceived impact of their experience.
  • Self Prior survey: Prior surveys allow you to record expectations of an event, to later compare with how it was actually perceived. We typically recommend that prior surveys are completed by self assessors to set up their objectives and creative intentions for the event.
  • Self and Peer Post survey: Post surveys are used to aid comparison between actual performance and expected performance. Typically self and peer assessors complet the post survey. Post event data can be compared with public data from the standard survey.

When creating prior and post surveys for the same respondent group within an evaluation, you must link them together so that Culture Counts knows which data to compare. If you create the prior survey first, you can complete this step when creating your post survey. You will see a ‘Survey Link’ box to the right of your survey type input on the Configure page. Select the correct prior survey from the Survey link dropdown list. This will then sync your prior and post surveys so that details from the prior survey are copied across to the post survey. To complete your post survey, click on the ‘Design’ tab and change the tense of the dimension statements from future to past from the dropdown options provided.

Delivery Types

This section allows you to choose how the survey will be delivered to the public respondents. Culture Counts supports the following delivery types:

  • Online Survey: to be taken by respondents online via email, or on their own smart phones or devices. Online surveys are non-resettable so they can only be taken once by respondents.
  • Interviewer: to be facilitated by an interviewer with a tablet computer or iPad. Interview surveys are resettable so that multiple responses can be recorded on one device. See our guidance on using interviewers here.
  • Display: to be displayed on a device at a set location, such as a fixed podium or a library computer. Display surveys are resettable so that multiple responses can be recorded on one device.

Select all of the methods that you wish to use to deliver your survey. You can use all three methods if you choose. A unique survey link will be created for each method and will be displayed clearly on the Summary page at the end of the survey builder. For more information on the various delivery methods, please see the User Guide. In your dashboard you will be presented with advanced options related to the delivery type(s) you have chosen, including:

  • A finish URL, which will redirect respondents to a specific website (e.g. your organisation’s homepage) on completion of the survey
  • A timeout, which will reset the survey if the respondent is inactive for a period of time
  • Access to survey tools, which will enable interviewers to access tools to support the interview process

Please note that all of the methods require internet connection in order to collect data.

*Please note, the links above will take you away from the Impact and Insight Toolkit and to YouTube. Please ensure you are happy for this to happen before clicking and have familiarised yourself with their cookie and privacy policies.

Continue browsing the Platform Guide by going back to the main menu or view/download a PDF version of the complete Platform Guide here.

The information on this page was last updated on 28 March, 2019.

X
X