1. Introduction

 

Peer review provides fresh professional perspectives from people that aren’t involved in the internal development of projects and enables organisations to understand how their work is perceived within their artform, specialist field or other considered area. In Culture Counts, the evaluation platform used in the Impact & Insight Toolkit (Toolkit) project, organisations can see how peer responses compare with the views of audience members, participants and self-assessors.

Remember, peer review is not an examination with a pass mark; it is an opportunity to hear from critical friends who share your desire to improve.

 

2. Principles of peer review

Organisations need diverse perspectives

• Peer review enables organisations to understand the perspectives of practitioners from a wide range of backgrounds.
• Peer review creates valuable professional development opportunities for practitioners at different stages of their careers.
• When selecting peers, consider prioritising people whose backgrounds and viewpoints are not well-represented in the cultural sector.

 

Contributions to peer review

• Organisations and peer reviewers contribute to the development and embedding of an effective peer review system within the arts and cultural sector.
• Peer reviewers contribute their time– both to attend events and to provide feedback.
• Organisations generally contribute a ticket to events and may contribute towards other expenses.

 

All expertise is valid

• Organisations are best placed to decide who has the necessary expertise and insight to review their work.
• Peers with considerable experience in their artform or specialist field can expect to receive more invitations to review work; however, organisations often want to hear feedback from emerging as well as established practitioners.
• The opinions of one peer group are no more valid than another; they are simply different perspectives. When interpreting peer feedback, organisations need to reflect on who their reviewers are: why did these peers respond to this work in this way?

 

Feedback to support practice

• The primary aim of the Toolkit is to help organisations improve their practice rather than prove their worth; peer review is only valuable if it offers a fair critique of an organisation’s work.
• Peer review needs to be approached with a spirit of enquiry, with organisations approaching peers who can challenge their thinking and peers committed to providing honest, supportive and constructive feedback.
• A peer review is a snapshot of an organisation’s work as experienced by one individual, and will not have a direct impact on any decisions made about an organisation’s funding.

 

Appropriate use of data

• Peer reviews are ‘pseudonymised’, which means that, when survey responses are downloaded, personal information about peers (e.g. email address) is not shown.
• Organisations cannot see which survey response came from which peer reviewer, unless personally identifiable questions are asked and answered.
• When analysing data, organisations will typically look at the average scores given by their peer reviewers, and compare them with average audience scores and their own self assessment.
• Data from all survey respondents will be anonymised and provided in a large, public dataset that anyone can analyse to better understand the value of publicly funded arts and culture.

 

Enabling great peer review

• The Toolkit is an opportunity for us all to learn more about how to give and receive useful feedback on arts and cultural events.
• Counting What Counts will work with organisations to produce guidance and resources to share learning about the purposes, processes and outcomes of peer review.
• We can learn from existing research, peer review systems in other countries and best practice in other sectors, particularly education.

 

3. Practicalities of peer review

Who can peer review my work?

A peer reviewer can be anybody who is not involved in the curation or creation of your work. They don’t need to work in the arts and cultural sector, or in the same artform as the work that’s being reviewed. What’s important is that you believe their professional perspective on your work will provide value to your understanding of its effectiveness.

Consider the following example:

The work is a theatre piece about wellbeing in young people. It aims to increase awareness of the importance of good mental health, and the support that’s available to those that feel they could benefit from it.

So, who could peer review this work?

A few suitable suggestions might be:

• A theatre professional
• A youth worker
• A teacher
• A counsellor or therapist
• Mental health support staff

Remember, a suitable peer reviewer for one work might not be suitable for a different work.

 

What are the requirements around peer review for Toolkit users?

Peer review is not mandatory for users of the Impact & Insight Toolkit. However, we recommend you engage 3 to 5 peer reviewers, where possible and appropriate.

Use of peer review relates to the level of detail you have access to in your reports. Read our Evaluation Guide for more information.

 

What is the Peer Matching Resource?

The Peer Matching Resource is a database of sector professionals. By signing up, they have agreed for Toolkit users to approach them and request peer reviews. The Peer Matching Resource is hosted on Culture Counts and is only visible to those that are part of the Toolkit project. Access the Peer Matching Resource in your Culture Counts dashboard, by opening an evaluation and navigating to the ‘Invite’ page.

 

Can I only invite peers from the Peer Matching Resource?

No, the Peer Matching Resource is simply a resource you can use to expand your professional network and invite arts and culture professionals that you don’t already know. You do not need to use the resource. If you want to invite someone you already know and think they could provide a valuable and professional perspective, contact them directly to ask if they will provide a review. If they say ‘yes’, you will need to generate a unique link to the survey for them and send it to them, either through Culture Counts or by copying and pasting.

 

Do we need to pay peer reviewers?

Usually, peer reviewers do not expect to be paid for their time or expertise; however, it is likely they will expect to be reimbursed for expenses. It is worth bearing this in mind when considering who you would like to ask to provide a review. For instance, you might want to think twice about inviting a peer reviewer from Anglesey to review your concert in Eastbourne. To make reviewing more appealing, we know of organisations that have offered a peer reviewer a ‘plus one’ to their ticketed event and, in some instances, the offer of a little perk. e.g., a theatre might offer a glass of wine at the interval; an art gallery might offer 10% off a booking to a future exhibition… You see the idea!

 

When should we send a peer reviewer their survey link?

We recommend emailing your peer reviewer their survey link as soon as possible, once they’ve experienced the work. Not only is it best to capture views when it’s fresh in their mind for accuracy, the peer reviewer will likely also respond more quickly to the survey request if they’re feeling connected to the work. i.e., on a high/feeling the buzz/thinking about it.

 

4. Become a peer reviewer

 

The Impact & Insight Toolkit encourages participating organisations to select peers to review their work, providing unique and well-informed perspectives. Being invited to review an organisation’s work means that your professional opinions are highly valued; the host organisation considers you to be an expert in your artform or specialist area and well placed to offer feedback on their work. The peer’s role is to experience the work of an organisation (for example, a show, a concert, an exhibition or a publication) and respond to an Impact & Insight Toolkit survey.

 

Who can register to be a peer reviewer in the Peer Matching Resource?

• Someone with knowledge and expertise of an artform or specialist field.
• Someone able and willing to make fair and honest judgements, and give constructive criticism and feedback to help support an organisation’s development.

 

Why be a peer reviewer?

• Use your skills and knowledge to help develop creative projects and support artform innovation.
• Provide invaluable feedback to a variety of different organisations, helping them achieve their desired creative intentions.
• Help build an ecosystem and community of supportive creatives in your area.

 

How are peer reviewer responses used?

The mean average of peer reviewer responses will likely be viewed alongside public and self-assessment results. This comparison of self, peer, and public results allows an organisation to see how far they have met their ambitions for a piece of work. Your open text answers are also an important part of the peer process, providing Toolkit users with additional interpretative context about your feedback.

Your responses to dimension and demographic questions contribute to an aggregate dataset. The aggregate dataset helps the sector better understand the value, diversity and impact of work being produced by ACE funded organisations.

Your feedback won’t have a direct effect on funding decisions made by Arts Council England (ACE); the insight gathered through using the Impact & Insight Toolkit will inform conversations between organisations and ACE, contributing to shared, well-rounded perspectives on their work.

 

How do I register or nominate a colleague to become peer reviewer?

Organisations can nominate staff members to act as peer reviewers for other organisations, both to provide individual professional development opportunities and to bring learning back into the organisation. To register a staff member, in your Culture Counts dashboard, open an evaluation and go to the ‘Invite’ page. Here you can register yourself or a colleague as a peer reviewer by clicking ‘Register a Peer’. If you would like your staff members to upload their own details, you can copy and share your organisation’s bespoke peer registration link.

If you would like to register as an Impact & Insight Toolkit peer reviewer and are not part of a participating organisation you can sign-up to the Peer Matching Resource as an independent peer reviewer.

 

I’ve registered for the Peer Matching Resource. What happens next?

The information you have provided will now be visible to users of the Impact & Insight Toolkit and you may be invited to attend and review events. When you receive an invitation, you can accept or decline by clicking the ‘Manage response’ button. Invitations are always sent to you via email; please do not try logging in to Culture Counts unless you are an existing user of the platform.

Once you accept an invitation, you may coordinate the logistics of your attendance and direct any questions to the contact person listed in your invitation. After attending the work, you will be asked to complete an online survey. If you need to cancel, you can do so through the ‘Manage response’ button in your invitation. You can also let them know by sending a courtesy email using the contact details provided.

You may have already been invited to attend and review events. If you have not, we recommend that you please add [email protected] to your ‘safe senders’ list within your email client and that you check your spam folder to ensure that no invitations go ignored.

If you need to change any information in your profile, or would like to be removed at any time, please contact [email protected] and one of our team members will gladly assist you.