When experiencing art and culture, we know that there’s often an invitation to ‘feel’ something, whether that’s joy, awe or terror. But how do we describe these feelings and how can we begin to measure them for evaluation in the Impact & Insight Toolkit?

Introduction

In 2019, we conducted a series of webinars and workshops to understand what those using the Impact & Insight Toolkit (Toolkit) think is missing. From these activities we had feedback that the current selection of dimensions in the Toolkit is missing others that measure the feelings and emotions of those experiencing the work.

The Toolkit aims to measure the nature of the experience of those who experience cultural works by converting qualitative opinions into data, which conceptually might include their emotional response. However, the core focus of the Toolkit has been on the qualities of the work itself, rather than the feelings of the people who experience them.

The goal of capturing people’s emotions is a different measurement requirement but one that is clearly an important aspect of their experience. That being said, emotions and feelings can be a challenging thing to measure, which is explored in a short series of blogposts.

In this blogpost, we’ll set the scene for “Sentiment Month” and lay the groundwork for our decisions to select an existing framework by covering the background for our work on sentiment and providing a refresher for anyone who was not involved in the Strategic Development Strand (SDS).

Initial steps

In response to the expression of interest in measuring emotions and feelings, our initial idea was to approach this like the creation of the dimensions on qualities – a co-production process where we engage our users about the qualities they are interested in measuring.

We set about reviewing the existing dimensions in the Culture Counts platform and considering which of these could be considered more emotion-focussed rather than quality-focussed. We also did a review of some relevant literature[1] on the topic of audience experience which resulted in a few additional dimension suggestions.

We took these dimensions to a webinar and a series of feedback sessions with Toolkit users who had registered their interest in this topic. We asked for feedback on these dimensions: their wording; what else needs adding. We also enquired into any existing measures people might use and on the idea of measuring emotions in general.

The set of dimensions which we discussed in those webinars is below:

Existing Dimensions (shown as Dimension – Statement)

Captivation – It was absorbing and held my attention
Aesthetic Experience – It gave me a sense of joy, beauty and wonder
Meaning – It moved and inspired me
Enjoyment – I had a good time

Proposed Dimensions (shown as Dimension – Statement)

Excitement – I felt excited
Peace – I felt peaceful
Tension – I felt tense
Escape – I was transported to another world
Fun – It was a lot of fun
Worthwhile – I was glad I came

There was feedback on the suggested dimensions in those sessions, but the key takeaways that were made clear were not about the specific dimensions which were tabled in the discussions. Instead, the more interesting feedback points were more generally about the challenges of measuring emotions:

● We should try to capture positive and negative emotion
● Metrics should be distinct with limited amount of overlap
● We should try to measure the overall emotional impact
● We should try to distil the metrics into a small set/framework

These pieces of feedback, and the complexity of the problem that they highlight, got us thinking about the problem differently.

Our co-production approach has been effective so far because we were talking to arts and culture experts about the qualities of artistic and cultural works. In the case of trying to capture people’s emotions or feelings, perhaps, instead of talking with the people who are expert in arts and culture, we should look towards the people who have expertise in this area.

Emotion and sentiment frameworks

Psychologists have studied human emotion extensively and many frameworks have been created which aim to describe the full spectrum of emotions.

As well as frameworks for mapping out emotions, psychologists and social scientists have developed frameworks for measuring a person’s emotional state – usually referred to as sentiment. The idea of sentiment is that it describes a mental attitude that has been influenced by primary emotions. Sentiment is more organised and can be more easily expressed than primary emotions[2].

These sentiment measurement frameworks are designed to take a snapshot of a person’s current emotional state in a way which is valid (it measures what it sets out to measure) and consistent (it works the same way each time).

When so many experts have already invested efforts into mapping the human emotional spectrum and in figuring out ways that we can measure them, rather than reinventing the wheel, the right course of action is to conduct a review of what has already been done and to select the parts which best suit our purposes.

Sentiment Month

Having invested time in reviewing what is already available in the field of sentiment measurement, we have learned a lot and our thinking has advanced. We now want to share the results of this process with you. This means not only the framework that we recommend using but also the learning that has been gained, as it is important to understand the what and why of sentiment measurement before we look at the how.

This leads us to Sentiment Month!

This month, we will:

1. Hear from our resident expert in this area on why we need to take this seriously and the dangers of misunderstanding what we can and cannot measure about a person’s feelings.
2. See the results of our investigation into sentiment measurement frameworks, along with our thought process that led us to this conclusion.
3. Release a new set of tools which implement our recommended sentiment measurement framework along with a set of guidance on how to use them.

 

We’re looking forward to sharing this with you and hearing your feedback!

 

Please know that this is the first blogpost in a series of three; see links below:

1. Exploring ‘feelings’ prompted by arts and cultural experiences
2. The uses and misuses of sentiment measurement for cultural experiences
3. Considering frameworks and the selection of LE-PANAVA

Main image: Photo by Tengyart on Unsplash

 

[1] Capturing the Audience Experience, New Economics Foundation. Understanding the value and impacts of cultural experiences, Arts Council England. Value of Arts and Culture to People and Society, Arts Council England. Outcomes Schema, Cultural Development Network. Arts Audience Experience Index AAEI, Radbourne et al. Audience Knowledge Digest: Why people visit museums and galleries, and what can be done to attract them, Morris Hargreaves McIntyre

[2] https://www.differencebetween.com/difference-between-emotion-and-vs-sentiment/