In order to better support those organisations that are going to be providing outdoor arts (OA) experiences with their evaluation, Counting What Counts has developed a small question bank, containing suggested questions that can be added to surveys containing dimension questions. However, the research and consultation process which resulted in this small question bank also brought up many other suggested topics, which we would like to address:
- Dimension Selection
- Time of Day
- Safe Space
- Length of Stay Influencers
- Party Makeup
- Economic Impact
2. Dimension selection
A dimension is a standardised statement which respondents rate their agreement level to e.g. The statement “It was thought-provoking” is presented to a respondent with a sliding scale; the respondent touches a point on the scale to record their level of agreement with the statement.
In addition to the dimensions that Arts Council has requested their funded organisations to use, there are many others to choose from. In particular, it is important to note that Placemaking dimensions are available and would be worth considering as possible additions to your OA evaluation. You can read about them here.
The often hyper-local nature of OA events lends them to being experiences that can foster a sense of community and belonging. One example of a dimension from the Placemaking category is “It strengthened my cultural pride”. This might be something you would find useful to capture in order to further your understanding of how members of the public, as well as peers, are responding to your OA event.
There are also dimensions relating to Cultural Experience, beyond those that are within the core set, that should be considered; for instance, “I would come to something like this again”. A full list of the dimensions available to you is here and we would suggest that you approach the idea of adding questions with the question ‘What are my intentions for this piece of Outdoor Art?’.
3. Time of day
Will the time of day affect how someone experiences your work? It is very possible that it will! Say you are running an all-day festival and you are going to use the Toolkit to evaluate the impact of the festival as a whole, and therefore you have one survey to use across the festival. The time of day will present a few variables such as: the weather, which specific events are taking place, how dark it is, how hungry someone might be…!
Whilst these might sound obvious points to consider, having this information will make the data you collect more insightful. Why do we suggest asking for a time of visit, as opposed to specifics about the weather etc? Put simply, we are reducing the number of questions you need to ask of your attendees. Instead of asking them what the weather was like during their visit and what specific works they encountered (and more question besides!) you can ask them for the approximate time of day they attended, as you will have easy access to information about the weather and programme etc. You don’t need to ask the attendees these questions when you already have the answers at your disposal. We would suggest that this question is asked as a multiple choice (where the respondent can select more than one option); for instance:
Question: “During which times were you at Festival X?”
Answer options: “10am-12pm; 12pm-2pm; 2pm-4pm; 4pm-6pm”.
This will provide you with enough information to gain additional insight from your data, but won’t be adding too many questions thus reducing survey fatigue for your respondents.
4. Safe space
The ‘safe space’ matter is something that comes up regularly and is of further importance when considering the matter via a COVID-conscious lens. ‘Safe space’ is an interesting term in that it can mean so many different things. Therefore, if you choose to have a question in your survey about feeling safe in a space, we would encourage you to make the ‘type’ of safety explicit. For instance:
Question 1: “Did you feel that this was a safe space to express yourself, without judgement?”
Question 2: “Did you feel safe with the people around you in this space?”
Question 3: “Did the organisers set out and enforce adequate instruction to make you feel COVID-safe?”
In the case of all three of these questions, you should provide the respondent with a free text question to express how they felt unsafe, should they answer ‘no’.
These three questions all nod to the subject of ‘safety’, yet a respondent’s answers might be very different to the three questions. This has been included within this OA evaluation guidance, as there are considerations specific to outdoor works that could materialise such as adequate lighting, uneven ground etc.
5. Length of stay influencers
For instance, if it is a sunny day, one might feel inclined to stay longer at an OA event; if one felt there were too many people and it was too crowded, one might be inclined to leave. There will always be external factors which you have no control over that may cause someone to leave earlier than expected (e.g. work commitments or child tiredness); however, there are other things that you may be able to influence for future occasions (e.g. crowd control or having covered areas in case of rain). You will not know what can be done to encourage someone to stay longer at your event unless you ask why they chose to leave early.
Therefore, if you are running a ‘drop in and drop out’ OA event, the following question schedule could be considered:
Question: “Did you stay at this event for the length of time you expected to?”
Answer options: “Yes – I stuck to my expectations; No – I stayed longer than I expected to; No – I left after a shorter period of time than I expected; I didn’t have any expectations for how long I would stay”
If they select either of the ‘No’ options, you could then ask them why…
If they stayed longer than expected…
Question: “Why did you stay longer than expected?”
Answer options: “I enjoyed myself more than I expected to; It was lovely to be outside in this weather; My external commitments were postponed or cancelled; There was a great vibe”
If they stayed for a shorter time than expected…
Question: “Why did you stay less time than expected?”
Answer options: “I didn’t enjoy myself as much as I expected to; It was unpleasant being outside in this weather; My external commitments (work/care/appointments etc.) were brought forwards”
By asking these questions you will be able to learn if there is something you could do to encourage members of the public to stay for a longer duration of time and why footfall might alter throughout the day.
6. Makeup of party
Something which is often difficult to capture at ‘drop in drop out’ events is the makeup of the parties that have attended, especially if it is unticketed. Asking questions around party makeup can assist you in learning who your work is drawing in: families, groups of friends, solo attendees… You would then be able to tailor or adapt your programming or marketing accordingly.
Surveys are most often completed by adults and, if ticketed, those that purchased the tickets. In order to delve into this a little further, one could ask the following series of questions:
Question: “Did you visit this event with anyone else?”
Answer options: “Yes; No”
Using the logic feature, exposure to further questions can be tailored.
If they say ‘yes’, you can then ask,
Question: “What word best describes your relationship with those you attended with?”
Answer options: “Friend(s); Mixed age family; Adult family; Co-workers; Partner(s)”
If required, this could be explored further with number style questions such as,
Question 1: “How many adults (aged 18+) did you attend with?”
Question 2: “How many children (aged 0-17) did you attend with?”
However, in many situations, this further couple of questions would be unnecessary as the quantity of attendees can be monitored through other means such as clickers or counter applications. It is the type of party that has most potential to impact future programming or marketing strategies.
7. Economic impact
We know that measuring an OA event’s economic impact is incredibly important. However, we also know that, in order to conduct a thorough and effective assessment, a different evaluation strategy and approach is required. That is why we are not including economic impact in the OA question bank. That said, we would like to point you in the direction of some useful information to kick-start your thinking around this: https://www.culturehive.co.uk/wp-content/uploads/2020/10/Practical-Economic-Impact-guide-1.pdf
A thorough economic impact assessment can support future applications to deliver further works in outdoor spaces.
We know that there will be many further things you may have a keen interest in, and we look forward to hearing more about your Outdoor Arts works, evaluations and insight gained!
Please do reach out with any questions and we will be happy to support you.