Carrying Out a Paper Survey

1. Introduction

The Impact & Insight Toolkit uses an online evaluation platform, but there are times when it is not possible to collect survey responses via online delivery methods. The team at Counting What Counts is here to support you in achieving valuable insight into the impact of your work, regardless of whether online surveying methods are suitable. This document lays out the options available to you and explains how to conduct a paper survey using the Impact & Insight Toolkit.

Before we go any further, let’s consider the three online survey delivery options:

  • Online delivery – A URL is generated which you email out to those that you wish to complete your survey; this requires email addresses and permission to contact.
  • Interview delivery – A URL is generated, formatted for intercept interview delivery. Multiple responses can be taken on one Internet enabled device.
  • Display delivery – A URL is generated; the survey is formatted similarly to the one for interview delivery, except that it is designed to be completed autonomously by the respondent.

We strongly encourage online data collection wherever possible due to the below reasons:

  • It is generally far less resource-heavy – no need for manual data entry
  • Timestamp data is collected, meaning that you can analyse your data to explore how views and experiences change over time
  • Useful visualisations are generated and subsequently presented in the reporting dashboard, all updated in real-time
  • The data is of a higher validity and will contribute to the aggregate dataset

However, there are occasions when you may need to collect data using a paper survey, such as:

  • When you do not have access to the email addresses of attendees
  • When there is no Internet connection (WiFi, 3G, 4G) on site
  • When you have no Internet enabled devices (tablet computers, mobile phones) that you can use to collect data
  • When your audience is unlikely to respond well to the use of technology – perhaps due to age[1], disability or language difficulties
  • When the use of online methods does not feel appropriate given the setting, nature or content of the event you are evaluating

For any given event, the flow chart below will help you decide which delivery method(s) would be most suitable for the distribution of your public surveys.

[1] According to Ofcom in 2017, 44% of those aged over 75 use the Internet ; 53% of those aged 3-4 and 79% of those aged 5-7 use the Internet



2. How do I create a paper survey for public respondents?

  • Follow the steps for creating your evaluation in the Culture Counts platform, ensuring that the public survey is set up exactly how you wish (see Creating an Evaluation for more information).
  • Use the ‘sharing’ feature to share your evaluation with a member of the Counting What Counts team.
  • At least one week before the survey is required, contact us at Counting What Counts to request that we transpose your public survey.
  • We will transpose your survey onto the requested paper format (e.g. PDF, Word document) and return it to you.
  • You can print out your paper surveys and distribute them at your event.

Even if you are using a paper survey to collect public data, you should still use the Culture Counts platform to set up and distribute surveys for self and peer assessors. Once your public data is uploaded into the system, you will be able to view and download graphs of the self, peer and public data together. You will also be able to use the reporting template to produce a full report on your evaluation for your Relationship Manager, and to download all the raw data in a CSV file for further analysis

3. What do I do once I have finished collecting survey responses?

Interpreting your dimensions responses:

When audiences respond to the dimensions statements, they mark a spot on the scale to indicate their level of agreement. You may find that respondents vary in the way that they indicate their level of agreement, as shown in the screenshot below.

Like surveys on the Culture Counts platform, responses are given on a scale of 0 – 1. You will need to estimate the numerical value of each response, up to two decimal places. For instance, in the first example above, one would estimate the response to be 0.9. For the second response, one might estimate 0.95. If you are uncertain, we recommend rounding to the nearest half. We do not recommend that you spend a great deal of time trying to pinpoint the exact marking to two decimal places; in most cases the respondent will have tried to mark either on or in the middle of two lines.

In certain cases, respondents may mark their responses in a way that does not correspond with the scale and/or cannot be accurately interpreted to a valid numerical value. These responses cannot be interpreted or used, though their responses to other survey questions can still be kept and recorded.

A few examples of when it isn’t possible to accurately read the response:

1) In the event that a respondent writes ‘strongly agree’ instead of marking a point on the scale provided, one cannot accurately translate this into a number.

2) In the event that a respondent has indicated strong disagreement with the dimensions statements, but their free text responses indicate the opposite, it is likely that they misread the direction of the scale. We cannot make assumptions about what a respondent may have meant to write, therefore please do not record the response you think they intended to write.

Neither of these examples happen frequently but when they do, please know that you should not use these responses.

The responses from each paper survey need to be recorded into a spreadsheet.

Please see our guidance on importing data into Culture Counts here.

A YouTube support video has been made available here.

Please do not hesitate to get in touch with any questions at all:

Powered by BetterDocs