Creating Your First Evaluation

Having received your login details from the Counting What Counts team, you will be able to access the Impact & Insight Toolkit at this address: https://impactandinsight.co.uk

Once here, click “Sign In” to access the Culture Counts platform.

Culture Counts Sign in page

Once you’ve logged in to your account, this guide will take you through the steps required to create a new evaluation and make your first survey.  Jump to:

What is an Evaluation?

Evaluations are folders that contain all the information for a particular event, project or place that you wish to evaluate, AKA your cultural offering. If you wish to create several related surveys to evaluate an event, they can all be grouped and organised within a single evaluation. In other words, one evaluation folder per cultural offering. The evaluation will store and present all the data collected through those surveys so that you can quickly see what your cultural offering is achieving overall.

After you log in, you will notice that we have provided you with an evaluation template, to give you some insight into how you can structure your evaluation using the Impact & Insight Toolkit (Toolkit). If your use of the Toolkit is part of your funding requirements, the template will be particularly useful as it contains the core questions that your organisation is required to use for your Arts Council mandatory evaluations that will be shared with your Relationship Manager. As an easy starting point for your evaluation activity, you can copy and then edit this evaluation template, or of course you can create your own evaluation from scratch.  We recommend that you try both and see what suits you best. Details of how to copy the evaluation template and how to create an evaluation from scratch can be found below.

Please note you can choose to do evaluations in addition to those that are required. When creating additional evaluations, you do not have to use Arts Council’s core questions; you can choose whichever questions you like.

 

Key Survey Types

There are three primary respondent groups that we suggest you consider when designing your evaluation: public/participant assessors, self-assessors and peer assessors. A detailed explanation of these three categories can be found in the User GuideCorrespondingly there are three types of surveys that must be created within an evaluation:

  • a ‘standard’ survey (for the public to respond to)
  • a ‘prior’ survey (for the self-assessors to respond to)
  • a ‘post’ survey (for both peer and self assessors to respond to)

Our evaluation template contains one standard survey for public assessors, one prior survey for self assessors and one post survey for self and peer assessors. We encourage you to browse the surveys in the template to see how they have been set up.

Please note that if you would like to preserve the template to work from in the future, you should not make or save any changes to the surveys within the evaluation template itselfInstead, please copy the surveys before you start editing them.

Creating an evaluation by copying the evaluation template

To copy the template, first create a new evaluation for your cultural offering.  This will be a new folder where the copied surveys can be stored.  To start a new evaluation, click the New evaluation button at the bottom of the Evaluations panel (bottom left). You will be prompted to give your evaluation a name (we have called ours ACE test in the example below but normally it would be the name of the offering you are evaluating) before being taken to the evaluation overview screen, shown below.

example Culture Counts evaluation

The next step is to copy the evaluation template surveys into your own new evaluation folder. For this you will need to go back to the evaluation template so you can see the different surveys.  Simply click on the evaluation template you wish to copy from.

Copy the example public survey by clicking Edit to the right of the survey name and selecting Copy from the dropdown list.  Replace the copied survey name with a new name that reflects your cultural offering and choose your new evaluation to copy the survey into.  This is basically like copying and pasting.

We recommend giving each survey a simple name that both people in your organisation and assessors will understand, like the name of the offering and the survey type: ‘Exhibition A – Public Survey’ or ‘Exhibition A – Prior Survey for Self Assessors’. You will need to repeat the process above to copy each of the template surveys into your new evaluation.

If you click on your own evaluation, you will see that the surveys have been copied from the template evaluation and inserted into your own.  From here, you can customise these surveys.  Click the name of the survey that you wish to customise and/or use and you will be taken to the survey Summary page, which shows a brief summary of the survey’s settings and inclusions.  From here you can navigate to different pages in the survey builder by clicking the Manage button in the top right corner and selecting the appropriate page, or clicking on one of the tabs in the top menu bar (dark grey). The editable pages include:

  • Configure– to change the survey settings
  • Design – to edit, add or remove survey questions
  • Invite– to add or remove self and peer assessors

If you would like more information on adding more dimensions and custom questions you can find information and a support film in Designing Your Survey below.

*Please note, the links above will take you away from the Impact and Insight Toolkit and to YouTube. Please ensure you are happy for this to happen before clicking and have familiarised yourself with their cookie and privacy policies.

Creating an Evaluation from Scratch

To create a new evaluation without using a template, click the New evaluation button at the bottom of the Evaluations panel. You will be prompted to name your evaluation (normally the name of the cultural offering you are evaluating), before being taken to the evaluation overview page. From here you can create new surveys, manage existing surveys and download any data that has been collected for the evaluation.

To create a survey for your new evaluation, click Create survey. You will then be taken through the three main stages of survey creation:

  • Configure– to change the survey settings
  • Design – to edit, add or remove survey questions
  • Invite– to add or remove self and peer assessor

Configuring Your Survey

The Configure page is where you set up the following options for your survey.

Survey Configure page on Culture Counts

Name

This is the survey name that respondents will see when they access the survey.

Start and Close Dates

The start and close dates will determine when the survey begins and stops taking entries.  It is fine to leave the start and close dates blank.  If you leave the start date blank, the survey will be available to record responses immediately, and if you leave the close date blank the survey will remain active indefinitely.  When inputting the start and close date, we advise that you include an additional one or two days either side of the official start and close dates of your event to ensure that you capture all available survey responses.

Survey Introduction

Here you can write a short optional introduction to your survey that will appear on the front page of your survey.  This introductory front page will only appear on the Online and Display survey delivery methods, as interviewers delivering the survey by using a Tablet  can verbally provide this information to respondents. This introduction is good place to provide a brief explanation to respondents, such as:

We’d love to hear what you thought. Please share your views on this event via our short survey. Your feedback will help us to understand and measure the impact of our events.

Custom Branding

Here you can upload your own logo, which will appear on the front page with the survey introduction text.

Survey Types

There are three survey types: Standard, Prior and Post.

  • Standard: The standard survey is for your general public audiences or visitors. Audiences are typically asked to complete a standard survey after experiencing an event or visiting a place, to measure the perceived impact of their experience.
  • Prior: Prior surveys allow you to record expectations of an event, to later compare with how it was actually perceived. We generally recommend that prior surveys are completed by self assessors to set up their objectives and creative intentions for the event.
  • Post: Post surveys are used to aid comparison between expectations and experience. Typically, self and peer assessors completed the post survey. Post event data can be compared with public data from the standard survey.

When creating prior and post surveys for the same respondent group within an evaluation, you should link them, so that Culture Counts knows which data to compare.

If you create the prior survey first, you can complete this step when creating your post survey.  You will see a ‘Survey Link’ box to the right of your survey type input on the Configure page.  Select the correct prior survey from the Survey link dropdown list.  This will then sync your prior and post surveys so that details from the prior survey are copied across to the post survey.  To complete your post survey, click on the ‘Design’ tab and change the tense of the dimension statements from future to past from the dropdown options provided.

Delivery Types

This section allows you to choose how the survey will be delivered to the public respondents.  Culture Counts supports the following delivery types:

  • Online Survey: to be taken by respondents online via email, or on their own smart phones or devices. An online survey will only accept one response per device.
  • Interviewer: to be facilitated by an interviewer with a tablet computer. Interview surveys are resettable so that multiple responses can be recorded on one device.
  • Display: to be displayed on a device at a set location, such as a fixed podium or a library computer. Display surveys are resettable so that multiple responses can be recorded on one device.

Select all of the methods that you wish to use to deliver your survey. You can choose however many you wish.  A unique survey link will be created for each method and will be displayed clearly on the Summary page at the end of the survey builder.  For more information on the various delivery methods, please see the User Guide.

You will be presented with advanced options related to the delivery type(s) you have chosen, including:

  • A finish URL, which will redirect respondents to a specific website (e.g. your organisation’s homepage) on completion of the survey
  • A timeout, which will reset the survey if the respondent is inactive for a period of time
  • Access to survey tools, which will enable interviewers to access tools to support the interview process

Please note that all of the methods require an internet connection in order to collect data. 

*Please note, the links above will take you away from the Impact and Insight Toolkit and to YouTube. Please ensure you are happy for this to happen before clicking and have familiarised yourself with their cookie and privacy policies.

 

Go back to Support Materials

 

Continue browsing the Platform Guide by going back to the main menu or view/download a PDF version of the complete Platform Guide here.

The information on this page was last updated on 11 March, 2020.

X
X