Peer Review

Peer review is a large subject and so in order to direct you to the most pertinent information, please click the button below that best describes you or select an item from the table of contents.

1. Peer review for toolkit users

What is peer review?

Peer review is a vital element of the Impact & Insight Toolkit. The Toolkit works by asking audiences, participants and peer reviewers what they think of a cultural event or project and comparing this with the organisation’s original objectives for the work.

Peer review provides fresh professional perspectives from people that aren’t involved in the internal development of projects and enables organisations to understand how their work is perceived within their artform or specialist field. Organisations are interested to see how peer responses compare with the views of regular audience members and participants.

The ‘triangulation’ of self, peer and public views gives organisations a rounded appraisal of their work; it helps them to understand whether they are achieving what they set out to achieve, and how they can develop their work in the future.

Peer review is also valuable for peers themselves. It enables them to experience different types of work, and to explore how their own practice relates to what others in their field are doing. We hope that by expanding and diversifying peer review, the Impact & Insight Toolkit will help cultivate stronger professional networks across the sector, creating a supportive, challenging and stimulating environment for arts and culture to thrive.

This page provides guidance to organisations that are looking for peers to review their work and to organisations that would like to nominate staff members to act as peer reviewers. It also provides an overview of some guiding principles of peer review. For more detailed information about using the Peer Matching Resource please see our the relevant section in Creating an Evaluation guidance. If you are an individual who is interested in becoming a peer reviewer, see our separate guide (below) Peer Review for Professionals.

2. How do I select a peer to review my work?

If you have an event or project that you are planning to evaluate using the Impact & Insight Toolkit, you will need to identify and invite suitable peers to review the work. You need at least one peer reviewer per event; we normally recommend that you engage three to five peers.

2.1. Who is a suitable peer reviewer?

In a nutshell, a peer reviewer is someone whose opinion you value. You are looking for someone who you think can offer useful insight as well as a fair, informed critique of your work. You should choose individuals who are not invested in the specific work you are evaluating, and you may want to look beyond the ‘usual suspects’ to access perspectives on your work that you might not normally hear.

The type of professional who you would like to hear from will vary from event to event. For example, if you are evaluating an education performance or project involving schoolchildren, teachers observing the work may be ideal peer reviewers. A company that is evaluating a piece of physical theatre around environmental themes may want to approach a physical theatre practitioner and/or someone with expertise in communicating environmental issues to the public.

If you decide to approach multiple peer reviewers, try and choose people with a variety of expertise, experience and specialisms. Having a diversity of peers reviewing your event will offer you a well-rounded perspective of your evaluation. It is also worth considering peers’ locations when choosing; if they are far away, you will need to consider the cost of reimbursing them for their travel costs and time.

2.2. How do I find peer reviewers?

To find a suitable peer to review an event, the first place we would encourage you to look is within your own professional networks. Your networks may contain people from local arts organisations, touring companies, independent artists and academic institutions, amongst others. By engaging with your current contacts, you are acknowledging the value you place on their professional opinions of your work.

Within the Culture Counts platform, the peer matching resource allows you to search for peers. You may browse and filter a list of creative professionals who have registered their interest in participating as a peer reviewer. They may be working with an Arts Council funded organisation that is registered to use the Impact & Insight Toolkit, or they may have signed up as an external individual interested in being part of the project.

When selecting your peer reviewers, we would recommend that you engage with peers from both your own network and the new resource available to you, in order for you to achieve a well-rounded range of professional perspectives on your work.

2.3. How do I invite a peer to review an event?

The Culture Counts platform enables you to select a peer from the Peer Matching Resource and send them an invitation to review your event. You can find a step by step guide in our Creating an Evaluation guidance.

If you are approaching a potential peer reviewer who is not registered in the Impact & Insight Toolkit’s Peer Matching Resource, you may need to tell them a bit about the Impact & Insight Toolkit and the peer review process. You might want to share our guidance on Peer Review for Professionals which explains what the Toolkit is, the benefits of being a peer reviewer, what peer review involves and how information provided by peers will be used. You will need to explain what event you are asking them to review and provide some basic information about the event such as date and location. If a peer who is not registered on the Peer Matching Resource agrees to review your event, you can add their email address to the Culture Counts platform using the box towards the bottom of the ‘Invite’ page of your dashboard. This will enable you to later distribute surveys to your peer.

Once a peer has agreed to review your event, you may choose to email the reviewer with further information about the event. We suggest that you provide enough information to enable peers to understand the aims and intentions behind the work, and where you see the work sitting within your overall programme. Some events may need more explanation than others; some peers – particularly those who don’t know your organisation well – may need more information than others.

It is likely that you will need to arrange a time for each peer to come and see the work and make tickets available as required.

Peers in the Peer Matching Resource are under no obligation to accept invitations to review work; if you approach someone who is unable to review your event, we suggest that you look again within your existing networks or the Peer Portal.

2.4. How do I send a survey to my peer reviewers?

Once a peer has attended your event, you will need to send a survey to capture their feedback. The ‘Summary’ page will show all the peers in the portal who have accepted the invitation to review your event, and any other peers who you have approached outside of the portal and whose email addresses you provided on the ‘Invite’ page. From the ‘Summary’ page you can email the survey to each peer using a URL specific to that individual peer.

2.5. Peer review of participatory work

If you are using the Impact & Insight Toolkit to evaluate a participatory project, then you will need to consider whether peer review is appropriate. If the participatory process results in a performance or exhibition, you may want that event to be reviewed by a suitable peer. Alternatively, you may find it useful to invite a peer to observe and give feedback on elements of the participatory work. However, in some cases it may not feel appropriate or useful to include an outside observer – for example if you are working with very small groups, over long time periods and/or on sensitive themes. In these instances, your participatory evaluation would include self and participant assessment only. Please contact support@countingwhatcounts.co.uk if you would like more advice on evaluating participatory work.

3. Principles of peer review

This final section proposes some principles of peer review that relate to everyone involved in the Impact & Insight Toolkit, including the Arts Council as funder; Counting What Counts as provider; organisations looking for peers to review their work; individuals wishing to contribute as peer reviewers.

3.1. Organisations need diverse perspectives

  • The peer review element of the Impact & Insight Toolkit will work best if it enables organisations to understand the perspectives of practitioners from a wide range of backgrounds.
  • If you are recruiting people to join the database of peer reviewers, can you prioritise people whose backgrounds and viewpoints are not well-represented in the cultural sector at present?
  • If you are looking for someone to review your work, can you enable more diverse voices to be heard through your selection of peers? Can you create valuable professional development opportunities for practitioners at different stages of their careers?

3.2. Contributions to peer review

  • Organisations and individual peers are asked to contribute to the development and embedding of an effective peer review system within the arts and cultural sector.
  • If you are a peer reviewer, you will contribute your time – both to attend events and to provide feedback afterwards via a short online survey.
  • If you are an organisation that has asked a peer to review your work, you will need to contribute a ticket to your event, and you may wish to make a contribution towards their travel expenses.

3.3. All expertise is valid

  • The Impact & Insight Toolkit works on the assumption that organisations are best placed to decide who has the necessary expertise and insight to review their work.
  • Peers with considerable experience in their artform or specialist field can expect to receive more invitations to review work; however, organisations often want to hear feedback from emerging as well as established practitioners.
  • The opinions of one peer group are no more valid than another; they are simply different perspectives. When interpreting peer feedback, organisations need to reflect on who their reviewers are: why did these peers respond to this work in this way?

3.4. Feedback to support practice

  • The primary aim of the Impact & Insight Toolkit is to help organisations to improve their practice rather than prove their worth; peer review is only valuable if it offers a fair critique of an organisation’s work.
  • Peer review needs to be approached with a spirit of enquiry, with organisations approaching peers who can challenge their thinking and peers committed to providing honest, supportive and constructive feedback.
  • A peer review is a snapshot of an organisation’s work as experienced by one individual, and will not have a direct impact on any decisions made about an organisation’s funding.

3.5. Appropriate use of data

  • Peer reviews are ‘pseudonymised’, which means that when survey responses are downloaded, personal information about peers (e.g. email address) is not shown.
  • Organisations cannot see which survey response came from which peer reviewer unless personally identifiable questions are asked and answered.
  • When analysing data, organisations will typically look at the average scores given by their peer reviewers, and compare them with average audience scores and their own self assessment.
  • Data from all survey respondents will be anonymised and provided in a large, public dataset that anyone can analyse to better understand the value of publicly funded arts and culture.

3.6. Enabling great peer review 

  • The Impact & Insight Toolkit is an opportunity for us all to learn more about how to give and receive useful feedback on arts and cultural events.
  • Counting What Counts will work with organisations to produce guidance and resources to share learning about the purposes, processes and outcomes of peer review.
  • We can learn from existing research, peer review systems in other countries and best practice in other sectors, particularly education.

4. Peer review for professionals

The Impact & Insight Toolkit encourages participating organisations to select peers to review their work, providing unique and well-informed perspectives. Band 2 and Band 3 NPOs are required to nominate at least one peer reviewer from their organisation to the Peer Matching Resource database so that other participating organisations can invite them to act as peer reviewers. It is not compulsory for other Toolkit users (i.e. Band 1 NPOs, SSOs, etc.); however, they are welcome to register peers if they choose.

Being invited to review an organisation’s work means that your professional opinions are highly valued; the host organisation considers you to be an expert in your artform or specialist area and well placed to offer feedback on their work. The peer’s role is to experience the work of an organisation (for example, a show, a concert, an exhibition or a publication) and respond to an Impact & Insight Toolkit survey.

 

4.1. Why become a peer reviewer?

  • To use your skills and knowledge to help develop creative projects and support artform innovation.
  • To provide invaluable feedback to a variety of different organisations, helping them achieve their desired creative intentions.
  • To help build an ecosystem and community of supportive creatives in your area.
  • To understand what other people are creating and trying to achieve in your creative ecology or further afield.
  • To help you better understand the work you are making.
  • To experience different types of work and expand your knowledge base.
  • To use your experience as evidence of your CPD within the field.

 

4.2. Why would an artistic or cultural organisation want a peer reviewer’s perspective on their work?

  • To embrace a fresh set of professional eyes that aren’t involved in the internal development of a project.
  • To understand whether they are meeting their creative intentions with their work.
  • A selection of peer reviews coming from a variety of backgrounds can provide a well-rounded perspective on their work.
  • To help embed a rich process of critical reflection across the sector.

 

4.3. Who should be a peer reviewer?

  • Someone who has knowledge and expertise, either regarding a specific artform or spanning various artforms.
  • Someone who is able and willing to make fair and honest judgements, and give constructive criticism and feedback to help support another organisation’s development.

 

4.4. How will a peer reviewer’s response be used by the organisation and by Arts Council England?

  • A peer reviewer’s response is invaluable as it contributes to an organisation’s overall understanding of their work.
  • The Impact & Insight Toolkit is a self-evaluation tool, in which the key insights come from the comparison of self, peer and public responses.
  • Using the reporting function of the Impact & Insight Toolkit, the mean average of the results gathered from peer reviewers will be viewed alongside public and self assessment results. This comparison of self, peer, and public results allows an organisation to see how far they have met their creative intentions for a piece of work.
  • Your open text answers are also an important part of the peer process, providing self assessors with additional interpretative context about your feedback.
  • It is crucial you know that your feedback won’t have a direct effect on funding decisions made by Arts Council England (ACE); the insight gathered through using the Impact & Insight Toolkit will inform conversations between organisations and ACE, contributing to shared, well-rounded perspectives on their work.
  • The aggregate data set will help the sector better understand the value, diversity and impact of work being produced by ACE funded organisations.

4.5. What does being a peer reviewer entail? What are the peer reviewer’s responsibilities?

  • After uploading their details to the Peer Portal, individuals may be approached by organisations with invitations to review events.
  • If an organisation asks you to be a peer reviewer and you accept, you will then be invited to attend the event and complete a short survey.
  • Individual peer reviewers are under no obligation to review work; they can choose which invitations to accept based on their availability and interest.
  • If you are interested in reviewing another organisation’s project as a peer, you can also contact an organisation directly.

4.6. How do I register or nominate colleagues to become peer reviewers?

Organisations may be keen to nominate staff members to act as peer reviewers for other organisations, both to provide individual professional development opportunities and to bring learning back into the organisation. If you are a Band 2 or 3 NPO, ACE requires your organisation to be available to provide a minimum of four peer reviews per year. These assessments can be carried out by one staff member or shared out between multiple staff members. There is no limit to the number of people you can nominate from your organisation to act as peer reviewers – you may want to put forward someone from your creative team, say, and someone with education or audience development expertise.

To nominate someone from your organisation, you will need to tell them a bit about the Impact & Insight Toolkit and the peer review process. You might want to share this guide which explains the basics.

If your staff member is happy to become a peer reviewer, they will need to register their details. In your Culture Counts dashboard, open an evaluation and go to the ‘Invite’ page. Here you can register yourself or a colleague as a peer reviewer by clicking Register a Peer. If you would like your staff members to upload their own details, you can copy and share your organisation’s bespoke peer registration link.

Arts Council England suggests:

As a recommendation towards meeting your minimum requirements as an NPO (4 peer evaluations which can be carried out by different nominated peers from your organisation), that you accept peer evaluation requests across each year based on capacity and other commitments. You may choose to balance evaluation visits equally across the year, for example by accepting one each quarter and planning in advance when throughout the 12-month period you are available to accept requests. Provided that you have completed your minimum reviews as stated in your funding agreement, you are under no obligation to accept further requests where capacity and resource are limited. However, where peers do have capacity available, you can accept and complete a number of additional reviews for other organisations to support the initiative.’

If you would like to register as an Impact & Insight Toolkit peer reviewer and are not part of a participating organisation please contact support@countingwhatcounts.co.uk to arrange for your details to be uploaded to the Impact & Insight Toolkit Peer Matching Resource.

4.7. I’ve registered. What happens next?

The information you provided will now be visible to registered users of the Impact & Insight Toolkit. From time to time you may be invited to attend and review events. When you receive an invitation, you can accept or decline at your discretion by clicking on the Manage response button. Invitations are always sent to you via email; please do not try logging in to Culture Counts unless you are an existing user of the platform.

Once you accept an invitation, you may coordinate the logistics of your visit and direct any questions to the contact person listed in your invitation. After attending the event, you will be asked to complete a short online survey. The survey will take about ten minutes to fill in, depending on how much information you wish to provide.

If you need to cancel, you can do so through the Manage response button in your invitation. You can also let them know by sending a courtesy email using the contact details provided.

You may have already been invited to attend and review events. If you have not, we recommend that you please add no-reply@culturecounts.cc to your ‘safe senders’ list within your email client and that you check your spam folder to ensure that no invitations go ignored.

If you need to change any information in your profile, or would like to be removed at any time, please contact support@countingwhatcounts.co.uk and one of our team members will gladly assist you.

Powered by BetterDocs

X
X