We've been doing annual appraisals at Agile Collective for about 4 years (if memory serves).

When we were 6 - 8, we had a 360 peer review with everyone completing an appraisal form about themselves and other members. We had a grading system that had a vague association /correlation with our pay bands.
This was felt to be (understandably) a bit unhelpful and missing the point of an appraisal so we removed that element but kept the form.
The next iteration involved a more customised form for each member, asking for feedback in different areas (project work, managerial/operational, client management etc). We kept the whole team review meeting aspect but it was starting to get unwieldy and taking up a huge amount of time.
This year we have streamlined it (in theory) - each member chooses 3-4 people to apprasie them and also how (one-to-one, group, written). Once this has been done, we arrange a PDP meeting with the appraisee plus two others to discuss their appraisals and talk about training and development for the next year.
Whichever way we've done it, it takes a lot of time to organise and each year we wonder if we should bother!

This is not from first-hand experience, but.... (I recall some useless appraisals in the past, in a university setting, and no doubt others have their tales of woe)

... have you thought of the underlying principles of constructive feedback? To my mind, how useful any appraisal is depends on the underlying approach, attitudes and assumptions. I've just been helping in a minor way with a short (600 word) document called "Giving constructive feedback in work contexts: 10 guidelines" -- if there's interest I could ask the author if she was happy for me to share it here.

Hey Aaron

I'm glad you're asking about this, we think that the people stuff is just as important as the outward facing stuff. Here's a lengthy explanation on how we do it (please note: Outlandish don't do this perfectly but we understand how important it is, and we're having a good crack at it. I've copied a lot of the info below from our Wiki).
- We don't have appraisals, we do quarterly OKRs
- Each Outlander (member and the step before member) has a buddy. The overall purpose of the role is to provide emotional and practical support to the ‘mentee’, and to help them understand the Outlandish way of working. I'm including the buddy stuff to give you a comprehensive overview.

Core Responsibilities

Initial on-boarding. The buddy will (week 1):

Make sure the mentee is enrolled in the on-boarding process.
Introduce the mentee to the office and to other people who can provide more information.
Ensure the mentee is confident in understanding what work they are booked on, how they find out about more work and what projects they might be most interested in working on, how they ensure this is communicated to the project management circle (to aid future scheduling).
Support the mentee throughout the on-boarding process by, for example, helping them find answers to any questions that remain unanswered.

For a collaborator moving towards being an Outlander the buddy will (first 70 days):

Answer any questions the mentee may have, or point them in the right direction to find the answers.
Take mentee and one or two others on lunch at least once (paid by Outlandish) to check progress, answer questions, make friends!
Invite mentee to at least two circle meetings, and attend with them if necessary (e.g. where the buddy is not member of the circle relevant to the individual’s role or interest in Outlandish).
Support the mentee in understanding and meeting the requirements to being an Outlander and help them as necessary through the application process.
Once the mentee is confident they want to actively work towards being an Outlander, to help them set their OKRs and develop their PDP.

Ongoing (for at least the first year):

Ongoing support, including being someone to talk to when things are puzzling or uncertain.
Attending any meetings where the mentee might need additional support (eg OKR/PDP meetings).
Experience

Of having become an Outlander, or a member, themselves.

Skills
Listening.
Asking supportive questions.
** Knowledge **
The Outlandish induction process.
Outlandish OKR and PDP process.
The way in which Outlandish uses sociocratic decision making
The Outlandish competency framework.
General knowledge of how the office works.
General knowledge of who does what – ie who to ask.

This is a link to the OKR description: https://docs.google.com/document/d/1JO4DJu7JUnN8D7FFGIxmVp-8qPjR0LpP6zxhnpnq-l8/edit?usp=sharing

I think this is going relatively well (at the moment we don't have time to do them) but we did have a review and decided that OKRs are better than appraisals, because it's just down to a handful of people to get them done.

We also want to move towards PDPs (so, if the OKR is 'what can you do for Outlandish?, we'll use the PDP as 'what can Outlandish do for you?').

Sorry for the extra long post, I hope you find it helpful!

Hi Aaron

In my view we need to talk about the underlying principle behind appraising someone. For me, 'to appraise' has the connotation that someone knows better how someone should be.

Personally, I think that is wrong-headed, so I much prefer the idea and practice of feedback - freely given and openly received - and moving the responsibility for personal development to the individual rather than suggesting for a moment that it might lie with an 'appraiser'.

I am guessing this is where you are coming from too?

Here's some text on PDPs to complement what Kayleigh sent. Hope it helps.
https://docs.google.com/a/seestep.com/document/d/14weNbJXKN2DACArEaxdk09maUSv5yIEpuNzOx22To7o/edit?usp=sharing

Best
Pete

We have a flat pay structure and are all directors of the company with equal share, so pay doesn't enter into it for us. We don't do appraisals as such, as we work on the basic assumption that we're all competent, motivated and able to see where we fit it, and we use retrospectives to reflect on how the work is going, with the assumption that people will take away anything that they need to address in their own work. We do have reviews, which largely focus on welfare and development.

We have a spreadsheet with the names of all of the staff (there's 11 of us) and once a quarter, each of us will click the 'pick me a random person' button and pick someone to have a review with. It's very informally done, but means that everyone gets covered. They will meet for 30 minutes, with the person who initiated it acting as a facilitator, and making space for the person to talk about:

• What's going well?
• What's not going so well? Specifically:
• How is your stress?
• How is your workload?
• Do you have any health and safety concerns for yourself? For others?

The meeting should also be used to check that the participant is:
- booking and using holiday appropriately
- identifying and undertaking training

Brief notes are taken and are stored in our shared repository, encrypted with the keys of the two people involved so that everyone can see their own notes, and if we did need to recover a person's notes for whatever reason, we could do so but it would have to be a public process - I can't just go and read the notes of a review I wasn't part of.

Wow! who knew appraisals would generate this much interest :-)
@asimong that sounds really interesting, I'd be very interested in reading that.
@kayleighwalsh thanks for sharing, that's a really interesting approach, I will read in more detail later and no doubt pick your brains in more detail this weekend if I may. Making / finding time is always the issue, it seems. I think I recall a blogpost on your website about OKRs, maybe by Harry? We did look at buddying a while back but and even had a lovely "Circle of Love" diagram but it never took off, probably because we didn't have a specific enough vision / purpose for it... and time. So you don't get feedback from everyone you work with but rather a buddy?
@peteburden you are dead right, and what you've described is our approach. And what we / I have always liked is getting feedback from everyone you work with. For me that is really valuable, but there are other (more regular) ways of doing that - which happens on projects etc anyway. However, I get that some people are uncomfortable about giving / receiving feedback in a group environment.
@robredpath we don't have flat pay and the first iteration was not the best (one should never grade each other, even with the best intentions, it won't end well), the grading was not meant to be associated to pay it just ended up that way. We have learnt. I like your approach, especially (as with outlandish) it is quarterly so less likely to be forgotten and more part of your working life, and quite lightweight. What happens if things come out of it that need to be addressed? e.g. difficulties/issues between two people.

Please get back to me maybe in a week or so, as the doc I mentioned about constructive feedback is being revised. Thanks for the interest!

Calverts (also flat pay/consensus management) doesn't do appraisals, 360 or otherwise, on members; it does do a lot in terms of induction for new employees/candidates for membership, including specific trainings (one of which is that a candidate has to have attended at least one external worker coop training or event), buddy mentoring (only we call it parenting - awful, but there you go). So we rely heavily on having an open and solidaristic culture where problems, grievances or underperformance can be picked up and addressed at the watercooler/team/open meeting level, after which we have a conventional formal disciplinary/grievance procedure where 3 members not involved form an ad hoc panel (supported with professional advice if necessary) to resolve the issue. If the subject doesn't like their decision they can appeal, and the appeal panel is the whole coop. Bob Cannell is an expert on this stuff, mine his brain at the Worker Cop (sic) weekend ...

@aaronhirtenstein We don't have a formal process for dealing with things that come out of a supervision/review meeting - that's potentially a weakness, but we wouldn't want to be overly prescriptive in any case. If there is an issue that's raised, then there's at least one other person who then knows about it, and can help the other reflect on what to do about it - and potentially help. There's no specific duty imposed on the facilitator to take up any issue that's raised during the meeting, but we're all generally helpful people so it's likely that the facilitator would at least offer.