Loomio
July 25th, 2012 23:34

Semi-automated group outcomes

Nicolas Wormser
Nicolas Wormser Public Seen by 71
Nicolas Wormser

Nicolas Wormser July 25th, 2012 23:34

This is a discussion to figure out how, if at all, we should automate the attributions of outcomes to the proposals.

The manual process would be letting the author/facilitator/administrator choose the outcome of a proposal once it is closed by using a button-based interface similar to the one used for voting.

The semi-automated way that I propose would be to set rules at the moment when we create a proposal to determine how the results of the vote will impact the outcome. These specific rules could be:
- a minimum acceptance rate: state that the proposal shall not be adopted unless a certain percentage of the audience voted “YES” to it.
- a minimum participation rate (Quorum): states that the proposal shall not be adopted unless at least a certain percentage of the audience casted their vote.
- possibly other options such as “automatically reopen for X days if quorum not reached“ etc…

One the proposal is closed, the author/facilitator/admin would either chose to go with whatever the generated outcome is, or decide to do something else with the proposal:
- reopen it: postpones the due date (and possibly change the outcome rules?)
- amend it: leaves this proposal "as is" and open a new editable copy of it, with no vote attributed.
- abandon the proposal: state that the proposal (and its generated outcome) doesn't make sense anymore

The reason for this shift is to allow more formal decision-making processes (e.g. compatible with law), more transparency for the audience, and less ambiguity amongst the diverse stakeholders.

For more insight on my reflection on this, see this document (which you are welcome to edit/comment on):
https://docs.google.com/document/d/1Zs454ok3kJ6YFPcrxh992fRmo7dudkwd7492pD3Bhi8/edit

What are your thoughts regarding the semi-automated outcome generation?

Paul Smith

Paul Smith July 26th, 2012 00:22

Excellent background material here.

I'm still mulling it over but a couple of points I'm pondering while thinking it through:

How would this approach deal with multiple/consecutive proposals if they're implemented?

Does implementing automation limit the possibilities in any way? Or does this limit the possibilities even if it's not automated?

And will it influence the way in which people vote on a proposal? Eg/ Am I more likely to vote No or Block near the end of a proposal to stop an automated agree if my position would be abstained otherwise?

Danyl Strype

Danyl Strype July 26th, 2012 01:16

For a consensus decision-making model, the discussion management tools in Loomio are more important than the proposal/ voting engine. When using consensus, the pie chart is a straw poll - a way of getting a visual indication of whether consensus has been reached. The actions in response to a significant number of people saying 'no', or even 'abstain', would be to carry on discussion until consensus emerges. Often a proposal can include parts that everyone agrees with, and parts that are under-developed. Breaking off the agreed part into a separate proposal and seeing that there is consensus on it does three things:
* allows people to start implementing that decision
* lifts morale and confidence that the group can find consensus with enough thought and discussion
* allows the group to drill down to specific points of dissensus, which can then be further researched, reflected on and discussed until a clear solution emerges which the group can support

Loomio tools that could help this:
* ability to branch discussions so that the various issues that are causing people to say 'no' or 'abstain' can be explored separately
* ability to split a broad proposal into a number of more specific proposals

From Nicholas' comments, it sounds like you are extending Loomio to support majority-rules groups as well, which is fine, but good to be aware of the difference. Would it be worth having an option, or a configuration engine, that allow a new Loomio instance to be set up for consensus OR majority-rules (and potentially other situations).

Richard D. Bartlett

Richard D. Bartlett July 26th, 2012 06:48

Wow such good input on this thread already, love it. This is rich stuff.

My concern with this proposed feature is not so much about the specifics but with the underlying process for how features get designed and prioritised.

We've been doing some work with Keong on developing a User Centric Design Methodology, which is a coherent iterative process for determining what the users want and then prioritising how we deliver those features to them. We're starting to get our head around the UCDM process, the next step is figuring out how to make it work in a distributed development environment.

There's some notes about what we've been doing here:
https://docs.google.com/document/d/19ABs8EYc5LOJwVGxykEESw0z2TvqwyTK8tujG-Tci5k/edit

From the focus group with the users there was definitely a request for clarifying the outcome of proposals, so that's got to be one of our priorities. The question is how we develop the 'outcome' feature. I love the work you've put into it Nicolas. The problem is, as you've identified with your research, that automating the outcome opens up the can of worms of decision/discussion rule sets. It is likely that this will need to happen at some point, but i'm not sure that designing this huge feature in one block is the way to go.

My suggestion for how to move forward with this is to implement the simplest possible Outcome feature. Perhaps when a Proposal closes the Author is prompted to fill in a brief summary describing the conclusion. The Decision participants would then be notified of the Outcome and it would be attached to the Proposal when it moves down the screen into Past Proposals.

This should be super easy to build and can be deployed immediately, then we can see how the users respond and develop it further one step at a time. It's likely we'll want to add some kind of status (e.g. Passed, Failed, Reopened, etc), it's likely we'll want to be able to stipulate rule-sets to automate the process, and there's plenty of other options we can imagine too.

The thing is we can't guess what the users want so it's a much more efficient use of our resources to deploy something super simple and see what they tell us.

Obviously we've got to do more work on clarifying our process and then communicating it and making it accessible for anyone to contribute to. I feel like we're going to get in a groove though.

Benjamin Knight

Benjamin Knight July 26th, 2012 22:36

Thanks heaps for your background work Nicolas!

I think it's going to be really important for us to develop Loomio to provide a framework for the whole Discussion>Decision>Action process, so I'm very glad we're talking about this.

My worry with automated or semi-automated judgments about the outcome of a decision, is that it feels like it would be placing an artificial framework on an organic decision-making process. I'm much more in favour of a system where the outcome is described qualitatively by participants in the discussion. I think a great first step (which also happens to be potentially the simplest/easiest to implement) is for Loomio to send an email (or for a box to pop up inside Loomio) to the person who made the proposal when it closes, asking them to describe the outcome succinctly (250 characters?), which then gets added below the Proposal. I'd love to see a 'proposal outcome' message that goes out to the other people participating in the discussion (if that's the level of notification they want) that has the outcome summary, and shows the graph.

If other group members disagree with the proposal summary, they can say so in the discussion and ask for it to be modified.

Dynamic and flexible and easy!

That's how I'd love to see the default, anyway. I absolutely agree that there will be some formalised groups who will want very clear formal parameters - would love to see a plugin enabling that sort of thing to be implemented for groups that want it. How would you feel about that Nicolas?

Nicolas Wormser

Nicolas Wormser July 29th, 2012 13:54

Thank you all for your rich reactions here!

@Paul

The automation wouldn't change the way discussions are led, it would just be a way to automatically tag a proposal as "rejected" or whatever.
I don't think it would limit possibilities if the analysis we do is optimum. But I agree with others that we're not ready to do such an analysis and that we would need some experimentation with a simple version before we head this direction.

@Rich & @Ben

I think you've got me changing my opinion here. The best solution for know is to go with a fully manual process, as simple as possible, that might satisfy most of the groups' needs.

Looking this with a long-term perspective, I quite like the idea of an option in the group settings (instead of a "plugin") to tweak the way outcomes are made (notifications, automation etc…).

But for now, I totally agree that we should go as simple and flexible as possible.

Nicolas Wormser

Nicolas Wormser started a proposal July 29th, 2012 14:17

Implement a manual way of setting outcomes for now, while keeping in mind that we might extend it with optional automation at some point. Closed 11:59am - Wednesday 1 Aug 2012

Set outcome automation aside for now.

Being able to specify a set of formal rules from which proposal outcomes are automatically drawn might be useful for certain groups at some point, but it is too early to design it properly now.

Instead, we shall implement a simple manual way of phrasing outcomes for proposals, that might be iterated to handle notifications and outcome statuses (such as "rejected" or others).

The wording of the outcome statuses, as well as the fact that we decide whether or not we handle notifications and outcome statuses is not part of this proposal.

This proposal is just for deciding to keep it simple, flexible and manual for now.

Results
Agree - 12
Abstain - 12
Disagree - 12
Block - 12
12 people have voted (1%)
AM

Allan man
Agree
July 29th, 2012 15:47

Good way to learn what to build if you do decide to automate. Think outcomes is an important feature to have.

Aaron Thornton

Aaron Thornton July 29th, 2012 20:00

Hey Nicolas. we should have a mock up for manually setting an outcome this week. We are just going to put a paper prototype in front of some users for feedback.

Aaron Thornton

Aaron Thornton
Agree
July 29th, 2012 20:02

Nicolas Wormser

Nicolas Wormser
Agree
July 29th, 2012 21:02

Joshua Vial

Joshua Vial
Agree
July 29th, 2012 23:18

Richard D. Bartlett

Richard D. Bartlett
Agree
July 29th, 2012 23:42

Awesome. Happy to see this built once the paper prototype has been in front of a couple users.

 vivien maidaborn

vivien maidaborn
Agree
July 30th, 2012 00:30

seems like perfect example of wait for the UX guidance!

Benjamin Knight

Benjamin Knight
Agree
July 30th, 2012 00:33

Awesome Nicolas, sounds perfect!

Alanna Irving

Alanna Irving
Agree
July 30th, 2012 01:37

Paul Smith

Paul Smith
Agree
July 30th, 2012 02:17

G

guillaume
Agree
July 30th, 2012 10:03

Maz Hermon

Maz Hermon
Agree
July 31st, 2012 01:44

Jon Lemmon

Jon Lemmon
Agree
July 31st, 2012 20:34

Definitely needed.

Jon Lemmon

Jon Lemmon July 31st, 2012 20:37

Awesome! Has a paper prototype been built yet? If so, we should take a photo of it and post it here so Nicolas can see it. I'm thinking if Nicolas has any other ideas for ways it could be implemented (just in terms of design and placement of buttons, etc.) he could mock them up and post them here and we could test that in front of users as well and see how they respond.

Otherwise, Nicolas, if you're happy to wait a couple days, we'll bring some users into the office and test this feature and then send you a mockup to build it on based on their reactions.

Nicolas Wormser

Nicolas Wormser August 3rd, 2012 12:50

Sorry for coming back so late on this. That sounds great Jon, I'd be supra keen to get some users' feedback on a few mockups.

Considering I'm leaving in 2 hours for a 2-week trip, I'd suggest to use the mockups Aaron made, that reflect most of how I feel the feature should be implemented.
You can start by showing them a paper version of this mockup:
https://www.yammer.com/loomio/api/v1/uploaded_files/5986625/version/4544975/preview/Mock%20up%20-%20admin_close_toolbar%20.jpg
In that first mockup, we should probably replace the 3 buttons by a single one that would say "write outcome" or something like that.
When we click on this button we get a textarea to phrase the outcome (either in the Proposal section or in a box that appear in the middle of the screen). We don't have any mockup for this yet but maybe that's something we can draw a 10-second sketch of.

After showing this sketch to the user, show this one, minus the "Approved":
https://www.yammer.com/loomio/api/v1/uploaded_files/5986626/version/4544976/preview/Mock%20up%20-%20Approved_with_msg.jpg

I think that's it for the minimal version of outcomes.
It could also be interesting to show them the version with outcome tags/types. That's just a matter of leaving the first mockup as-is and showing the "Approved" of the second mockup that we would have hidden in the first scenario. We will probably have feedback that will show us that the 3 buttons of the first mockup aren't appropriate, but I would leave it as is for now and see what the users say.

Then when we get feedback, I'd be happy to start implementing from that feedback, as soon as I come back from Scotland (the 19th).

See you guys in two weeks