The Impact & Insight Toolkit uses an online evaluation platform, Culture Counts, but there are times when it is not possible to collect survey responses via online delivery methods. The team at Counting What Counts is here to support you in achieving valuable insight into the impact of your work, regardless of whether online surveying methods are suitable. This page lays out the options available to you and explains how to conduct a paper survey using the Impact & Insight Toolkit. View/download a PDF of this page here.
In this section:
- How to decide on the most suitable delivery method for public surveys
- How do I create a paper survey for public respondents?
- What do I do once I have finished collecting survey responses?
Before we go any further, let’s consider the three online survey delivery options:
- Online delivery – A URL is generated which you email out to those that you wish to complete your survey; this requires email addresses and permission to contact.
- Interview delivery – A URL is generated, formatted for intercept interview delivery. Multiple responses can be taken on one Internet enabled device.
- Display delivery – A URL is generated; the survey is formatted similarly to the one for interview delivery, except that it is designed to be completed autonomously by the respondent.
We strongly encourage online data collection wherever possible due to the below reasons:
- It is generally far less resource-heavy – no need for manual data entry
- Timestamp data is collected, meaning that you can analyse your data to explore how views and experiences change over time
- Useful visualisations are generated and subsequently presented in the reporting dashboard, all updated in real-time
- The data is of a higher validity and will contribute to the aggregate dataset
However, there are occasions when you may need to collect data using a paper survey, such as:
- When you do not have access to the email addresses of attendees
- When there is no Internet connection (WiFi, 3G, 4G) on site
- When you have no Internet enabled devices (tablet computers, mobile phones) that you can use to collect data
- When your audience is unlikely to respond well to the use of technology – perhaps due to age, disability or language difficulties
- When the use of online methods does not feel appropriate given the setting, nature or content of the event you are evaluating
For any given event, the flow chart below will help you decide which delivery method(s) would be most suitable for the distribution of your public surveys.
How to Decide on the Most Suitable Delivery Method for Public Surveys
How do I create a paper survey for public respondents?
- Follow the steps for creating your evaluation in the Culture Counts platform, ensuring that the public survey is set up exactly how you wish (see the Impact & Insight Toolkit Platform Guide for more information).
- Use the ‘sharing’ feature to share your evaluation with a member of the Counting What Counts team.
- At least one week before the survey is required, contact us at Counting What Counts to request that we transpose your public survey.
- We will transpose your survey onto the requested paper format (e.g. PDF, Word document) and return it to you.
- You can print out your paper surveys and distribute them at your event.
Even if you are using a paper survey to collect public data, you should still use the Culture Counts platform to set up and distribute surveys for self and peer assessors. Once your public data is uploaded into the system, you will be able to view and download graphs of the self, peer and public data together. You will also be able to use the reporting template to produce a full report on your evaluation for your Relationship Manager, and to download all the raw data in a CSV file for further analysis.
What do I do once I have finished collecting survey responses?
- Interpreting your dimensions responses:
When audiences respond to the dimensions statements, they mark a spot on the scale to indicate their level of agreement. You may find that respondents vary in the way that they indicate their level of agreement, as shown in the screenshots below.
Like surveys on the Culture Counts platform, responses are given on a scale of 0 – 1. You will need to estimate the numerical value of each response, up to two decimal places. For instance, in the first example above, one would estimate the response to be 0.9. For the third response, one might estimate 0.95. If you are uncertain, we recommend rounding to the nearest half. We do not recommend that you spend a great deal of time trying to pinpoint the exact marking to two decimal places; in most cases the respondent will have tried to mark either on or in the middle of two lines.
In certain cases, respondents may mark their responses in a way that does not correspond with the scale and/or cannot be accurately interpreted to a valid numerical value. These responses cannot be interpreted or used, though their responses to other survey questions can still be kept and recorded.
A few examples of when it isn’t possible to accurately read the response:
1) In the event that a respondent writes ‘strongly agree’ instead of marking a point on the scale provided, one cannot accurately quantify this.
2) In the event that a respondent has indicated strong disagreement with the dimensions statements, but their free text responses indicate the opposite, it is likely that they misread the direction of the scale. We cannot make assumptions about what a respondent may have meant to write; therefore, please do not record the response you think they intended to write.
Neither of these examples happen frequently but, when they do, it’s important to interpret them appropriately to ensure data integrity.
- The responses from each paper survey need to be recorded into a spreadsheet.
Please see our guidance on importing data into Culture Counts here.
A YouTube support video has been made available here.
Please do not hesitate to get in touch with any questions at all: firstname.lastname@example.org
 According to Ofcom in 2017, 44% of those aged over 75 use the Internet https://www.ofcom.org.uk/about-ofcom/latest/media/media-releases/2017/rise-social-seniors#6 ; 53% of those aged 3-4 and 79% of those aged 5-7 use the Internet https://www.ofcom.org.uk/__data/assets/pdf_file/0020/108182/children-parents-media-use-attitudes-2017.pdf
The information on this page was last updated on 10 February, 2021.
Image credit: Joanna Kosinska via Unsplash