An Introduction to Peer Review

Jump to a particular section by clicking on its heading below. Or, view/download a PDF version of this page here. If you’re looking for information on becoming a peer reviewer, visit our separate guide here.

blank sheet of paper and writing materials

What is peer review? 

Peer review is a vital element of the Impact & Insight Toolkit. The Toolkit works by asking audiences, participants and peer reviewers what they think of a cultural event or project and comparing this with the organisation’s original objectives for the work.

Peer review provides fresh professional perspectives from people that aren’t involved in the internal development of projects and enables organisations to understand how their work is perceived within their artform or specialist field. Organisations are interested to see how peer responses compare with the views of regular audience members and participants.

The ‘triangulation’ of self, peer and public views gives organisations a rounded appraisal of their work; it helps them to understand whether they are achieving what they set out to achieve, and how they can develop their work in the future.

Peer review is also valuable for peers themselves. It enables them to experience different types of work, and to explore how their own practice relates to what others in their field are doing. We hope that by expanding and diversifying peer review, the Impact & Insight Toolkit will help cultivate stronger professional networks across the sector, creating a supportive, challenging and stimulating environment for arts and culture to thrive.

This page provides guidance to organisations that are looking for peers to review their work and to organisations that would like to nominate staff members to act as peer reviewers. It provides an overview of the peer review process, from start to finish, and concludes with some guiding principles of peer review. If you are an individual who is interested in becoming a peer reviewer, see our separate guide here.

How do I select a peer to review my work? 

If you have an event or project that you are planning to evaluate using the Impact & Insight Toolkit, you will need to identify and invite suitable peers to review the work. You need at least one peer reviewer per event; we normally recommend that you engage three to five peers.

A. Who is a suitable peer reviewer? 

In a nutshell, a peer reviewer is someone whose opinion you value. You are looking for someone who you think can offer useful insight as well as a fair, informed critique of your work. You should choose individuals who are not invested in the specific work you are evaluating, and you may want to look beyond the ‘usual suspects’ to access perspectives on your work that you might not normally hear.

The type of professional who you would like to hear from will vary from event to event. For example, if you are evaluating an education performance or project involving schoolchildren, teachers observing the work may be ideal peer reviewers. A company that is evaluating a piece of physical theatre around environmental themes may want to approach a physical theatre practitioner and/or someone with expertise in communicating environmental issues to the public.

If you decide to approach multiple peer reviewers, try and choose people with a variety of expertise, experience and specialisms. Having a diversity of peers reviewing your event will offer you a well-rounded perspective of your evaluation. It is also worth considering peers’ locations when choosing; if they are far away, you will need to consider the cost of reimbursing them for their travel costs and time.

B. How do I find peer reviewers? 

To find a suitable peer to review an event, the first place we would encourage you to look is within your own professional networks.  Your networks may contain people from local arts organisations, touring companies, independent artists and academic institutions, amongst others.  By engaging with your current contacts, you are acknowledging the value you place on their professional opinions of your work. 

Within the Culture Counts platform, the peer matching resource allows you to search for peers. You may browse and filter a list of creative professionals who have registered their interest in participating as a peer reviewer. They may be working with an Arts Council funded organisation that is registered to use the Impact & Insight Toolkit, or they may have signed up as an external individual interested in being part of the project. For more information about how to find peers using the Peer Matching Resource please see our guide here.

When selecting your peer reviewers, we would recommend that you engage with peers from both your own network and the new resource available to you, in order for you to achieve a well-rounded range of professional perspectives on your work.

C. How do I invite a peer to review an event? 

The Culture Counts platform enables you to select a peer from the Peer Matching Resource and send them an invitation to review your event. For more information about how to do this, visit our guide here.

If you are approaching a potential peer reviewer who is not registered on the Impact & Insight Toolkit Portal, you may need to tell them a bit about the Impact & Insight Toolkit and the peer review process. You might want to share our guidance on Becoming a Peer Reviewer which explains what the Toolkit is, the benefits of being a peer reviewer, what peer review involves and how information provided by peers will be used. You will need to explain what event you are asking them to review and provide some basic information about the event such as date and location. If a peer who is not registered on the Peer Portal agrees to review your event, you can add their email address to the Culture Counts platform using the box towards the bottom of the ‘Invite’ page of your dashboard. This will enable you to later distribute surveys to your peer.

Once a peer has agreed to review your event, you may choose to email the reviewer with further information about the event. We suggest that you provide enough information to enable peers to understand the aims and intentions behind the work, and where you see the work sitting within your overall programme. Some events may need more explanation than others; some peers – particularly those who don’t know your organisation well – may need more information than others.

It is likely that you will need to arrange a time for each peer to come and see the work and make tickets available as required.

Peers on the Peer Portal are under no obligation to accept invitations to review work; if you approach someone who is unable to review your event, we suggest that you look again within your existing networks or the Peer Portal.

D. How do I send a survey to my peer reviewers? 

Once a peer has attended your event, you will need to send a survey to capture their feedback. The ‘Summary’ page of your dashboard will show all the peers in the portal who have accepted the invitation to review your event, and any other peers who you have approached outside of the portal and whose email addresses you provided on the ‘Invite’ page. From the ‘Summary’ page you can email the survey to each peer using a URL specific to that individual peer. Detailed information about how to distribute surveys to peers is provided in the Impact & Insight Toolkit Platform Guide.


Peer Review of Participatory Work

If you are using the Impact & Insight Toolkit to evaluate a participatory project then you will need to consider whether peer review is appropriate. If the participatory process results in a performance or exhibition, you may want that event to be reviewed by a suitable peer. Alternatively, you may find it useful to invite a peer to observe and give feedback on elements of the participatory work. However, in some cases it may not feel appropriate or useful to include an outside observer – for example if you are working with very small groups, over long time periods and/or on sensitive themes. In these instances, your participatory evaluation would include self and participant assessment only. Please contact support@countingwhatcounts.co.uk if you would like more advice on evaluating participatory work.


men working at a table

How do I become a peer reviewer or nominate colleagues to become peer reviewers?

If you are a practitioner working in or with the arts and cultural sector, you are welcome to register as an Impact & Insight Toolkit peer reviewer, regardless of whether you work at an Arts Council funded organisation.To find out more, visit our guide on Becoming a Peer Reviewer.

The Peer Review Process

The chart below provides a summary of the overall peer review process, distinguishing between actions for organisations and actions for peer reviewers.

Figure 1: Summary of Impact & Insight Toolkit peer review process

The Principles of Peer Review

This final section proposes some principles of peer review that relate to everyone involved in the Impact & Insight Toolkit, including the Arts Council as funder; Counting What Counts as provider; NPOs looking for peers to review their work; individuals wishing to contribute as peer reviewers.

1. Organisations need different perspectives

The peer review element of the Impact & Insight Toolkit will work best if it enables organisations to understand the perspectives of practitioners from a wide range of backgrounds

If you are recruiting people to join the database of peer reviewers, can you prioritise people whose backgrounds and viewpoints are not well-represented in the cultural sector at present?

If you are looking for someone to review your work, can you enable more diverse voices to be heard through your selection of peers? Can you create valuable professional development opportunities for practitioners at different stages of their careers

2. Contributions to peer review 

NPOs and individual peers are asked to contribute to the development and embedding of an effective peer review system within the arts and cultural sector

If you are a peer reviewer, you will contribute your time – both to attend events and to provide feedback afterwards via a short online survey

If you are an organisation that has asked a peer to review your work, you will need to contribute a ticket to your event, and you may wish to make a contribution towards their travel expenses.

3. All expertise is valid

  • The Impact & Insight Toolkit works on the assumption that organisations are best placed to decide who has the necessary expertise and insight to review their work
  • Peers with considerable experience in their artform or specialist field can expect to receive more invitations to review work; however, organisations often want to hear feedback from emerging as well as established practitioners
  • The opinions of one peer group are no more valid than another; they are simply different perspectives. When interpreting peer feedback, organisations need to reflect on who their reviewers are: why did these peers respond to this work in this way?

4. Feedback to support practice

  • The primary aim of the Impact & Insight Toolkit is to help organisations to improve their practice rather than prove their worth; peer review is only valuable if it offers a fair critique of an organisation’s work
  • Peer review needs to be approached with a spirit of enquiry, with organisations approaching peers who can challenge their thinking and peers committed to providing honest, supportive and constructive feedback
  • A peer review is a snapshot of an organisation’s work as experienced by one individual, and will not have a direct impact on any decisions made about an organisation’s funding

5. Appropriate use of data

  • Peer reviews are ‘pseudonymised’, which means that when survey responses are downloaded, personal information about peers (e.g. email address) is not shown
  • Organisations cannot see which survey response came from which peer reviewer unless personally identifiable questions are asked and answered
  • When analysing data, organisations will typically look at the average scores given by their peer reviewers, and compare them with average audience scores and their own self-assessment
  • Data from all survey respondents will be anonymised and provided in a large, public dataset that anyone can analyse to better understand the value of publicly funded arts and culture

6. Local peer clusters

  • Peer review can be a useful way of connecting organisations in geographical areas and encouraging collaboration
  • Local peer clusters offer opportunities for peers to learn from each other, and for organisations to develop a collective as well as individual approach to improvement
  • If you are working with NPOs in a region, what can you do to encourage a joined-up approach to peer review?

7. Enabling great peer review

  • The Impact & Insight Toolkit is an opportunity for us all to learn more about how to give and receive useful feedback on arts and cultural events
  • Counting What Counts will work with NPOs to produce guidance and resources to share learning about the purposes, processes and outcomes of peer review
  • We can learn from existing research, peer review systems in other countries and best practice in other sectors, particularly education

 

Go back to Support Materials

 

The information on this page was last updated on 21 January, 2020.

Image Credit: Kelly Sikkema (image 1) and Nik MacMillan (image 2) via Unsplash

CONTACT US

+44 (0) 1223 656 255
support@countingwhatcounts.co.uk

THE IMPACT  & INSIGHT TOOLKIT IS AN INITIATIVE BY

. Logo of Counting What Counts

© 2019 Counting What Counts Limited. All rights reserved.

X
X