Loomio

New feature: Score poll

GK
Greg Kan Public Seen by 122

SCORE POLLING IS HERE!

We've been asked for it many times, and now we've happy to announce that Score Poll is available in Loomio.

Score Polling is a simple but expressive voting process which captures the level of preference voters have for each option in the poll.

In practice this means voters choose a "score" from 0 to 9 for each option in the poll.

The wikipedia page on Score voting details the properties of this voting system quite well, if you're interested.

As organisations move away from the traditional consensus vote, the score poll will complement the advice process when it comes to decision making. It can be used to survey the range of preferences without implying or determining an outcome.

Try out our live demo here

https://help.loomio.org/en/user_manual/getting_started/decision_tools/score_poll_form.png

MB

Matthew Bartlett Wed 21 Nov 2018

Just running my first score poll right now! I might be in love.

MN

Matt Noyes Thu 22 Nov 2018

Nice!!

LF

Luke Flegg started a score poll Thu 22 Nov 2018

Who would be best to bank with? Closed Sun 25 Nov 2018

Best could mean: most ethical, no card reader required for online banking, late closing time for branch, modern, nice app and online banking

14 - Metro Bank
8 - Barclays
14 - Nationwide
16 - Cooperative
9 - Triodos
LF

Luke Flegg Thu 22 Nov 2018

8 - Metro Bank
5 - Nationwide
3 - Triodos
2 - Cooperative
0 - Barclays

I think we need to check if Nationwide still do business accounts

LF

Luke Flegg Thu 22 Nov 2018

Very similar to dot vote but dots are per option, not per group of options. I like it. Somewhere between dot vote and proposal. I still think multiple proposals running together in competition is still best for multiple choice where you want to rule out options people have objected to.
All in all, great. Can imagine using this

MB

Matthew Bartlett Thu 22 Nov 2018

I feel like score poll is actually kinda fun, in a way that most of Loomio is not that fun. (Good, serious, worthwhile, but not that fun.)

JW

James Wilson Fri 23 Nov 2018

7 - Barclays
5 - Cooperative
2 - Nationwide
0 - Triodos
0 - Metro Bank
A

anna Sat 24 Nov 2018

9 - Cooperative
7 - Nationwide
6 - Triodos
6 - Metro Bank
1 - Barclays
A

anna Sat 24 Nov 2018

This is excellent news, thank you! Can see using this all the time

C

Connor Sun 25 Nov 2018

Yesss! Range Voting is here!
My only concern is that people will still use the ranked choice option, thinking it's just another valid option, despite the fact that it's actually pathological and should be completely replaced by the Range Voting feature. Or as you've called it, Score voting.

RG

Rob Guthrie Mon 26 Nov 2018

I'm really pleased that you're pleased!

I think the usefulness or correctness of ranked choice depends on the application. That said, I'm not sure how often people use it for an appropriate application, and I'm considering removing it, or making the set of poll types configurable by the group admin.

GC

Greg Cassel Tue 27 Nov 2018

@adroit I'd disagree that ranked choice is inherently pathological, although it has a limited domain of effective usability. Actually I think it's the best way to make some difficult collective decisions, in which the collective must choose one or more options from a larger set of recognized options. For example, this is precisely why ranked voting exists, and I think that ranked voting is deeply preferable to plurality voting.

I'd certainly use ranked choice instead of score polling to make any official collective decisions. Score polling is deeply vulnerable to insincere tactical voting if it's used in official decision process. However, it holds immense potentials for general co-sensing and co-design process, so I'm really glad that Loomio has added it. :)

JD

Josef Davies-Coates Wed 28 Nov 2018

BTW @lukeflegg I don't think Nationwide have ever done business banking, but they are clearly considering it - I've had a couple of surveys from them recently that would suggest as much anyway...

JD

Josef Davies-Coates started a score poll Wed 28 Nov 2018

I've always wanted to score, but on a scale of -2 to +2 not 0-9 (in my expereince with asking people to rank things 1-10 almost no one ever gives less than 5 so hard to capture negative votes) Closed Sat 1 Dec 2018

But this just does 0-9, right?

12 - option 1
10 - option 2
15 - option 3
👤

Anonymous Wed 28 Nov 2018

9 - option 1
6 - option 3
5 - option 2
RG

Rob Guthrie Wed 28 Nov 2018

@jdaviescoates we can introduce the ability to configure the min and max, but wanted to wait until it was requested. Even better, until someone submits a PR for it.

GC

Greg Cassel Thu 29 Nov 2018

Hi Rob, does "submit a PR" in this context mean creating an Issue here? https://github.com/loomio/loomio/issues

[Edit: I realized later that you probably mean pull request. I need to slow down sometimes in my commenting.]

👤

Anonymous Thu 29 Nov 2018

9 - option 3
5 - option 2
3 - option 1

yep for now

LF

Luke Flegg Fri 30 Nov 2018

That's exciting. Yes I think they used to offer one but stopped like 7 years ago. I can't remember why. We went with Metrobank in the end for Dignity platform

C

Connor Tue 4 Dec 2018

Well... no, sorry. There are a lot of myths and misinformation out there about voting systems. You seem to have run into quite a few of them. Since there are others reading this I'll take some time to put context around the links in my response, but you should familiarize yourself with their content if you'd like to discuss further.

Quality Comparison

Range ('score') voting is as good or better than both IRV (what folks here have been calling "ranked choice") and Plurality in every case in terms of the quality of results. It's both superior to IRV as well as superior to Plurality. (In fact, the plurality system is so bad that (computer simulations indicate) replacing it with range voting would improve society by a comparable or greater amount than the entire invention of democracy in the first place.)

this is precisely why ranked voting exists, and I think that ranked voting is deeply preferable to plurality voting

You are generally correct that IRV is intended as an improvement over Plurality, although in fact it's frequently not - since they share many of the same pathologies, as the city of Burlington learned the hard way when they were among the first to try it. (They have since repealed IRV.) Unfortunately, IRV's problems are hidden under its complexity, whereas Plurality's flaws are simpler, familiar, and obvious to voters who can take them into account - meaning socially at least, IRV is in fact still worse than plurality.

On the other hand, Range Voting exhibits no pathological behavior, ever. The results may be less than perfect - but never completely wrong (as in the Burlington case, as well as others such as Peru 2006). Plurality and IRV can't even make it past Arrow's theorem of impossibility, making it mathematically impossible for them to always have sensible results.

Hopefully that covers your comment of

I'd disagree that ranked choice is inherently pathological, although it has a limited domain of effective usability.

Yes, I'd say that domain is quite limited. Namely, limited to elections you don't care about. You seem to want to do the opposite:

I'd certainly use ranked choice instead of score polling to make any official collective decisions.

Yikes. Why use the broken system precisely where good results are needed most? Why introduce the spoiler effect and pick the wrong person for your next secretary? Why allow the chance that you could be making the results worse for yourself just by voting?

However, it holds immense potentials for general co-sensing and co-design process, so I'm really glad that Loomio has added it.

Loomio is a perfect place to pick the right one and stick with it, focusing developer efforts and minimizing complexity. Not only is range voting simpler than IRV, but it can in fact be used as if it were other systems - want to vote Approval-style? Simply vote maximum and minimum ratings only. Want Plurality for some reason? Vote minimum for all but one.

Score polling is deeply vulnerable to insincere tactical voting if it's used in official decision process. However, it holds immense potentials for[..]

Your "however" implies that Range has somehow been disqualified from serious voting due to your previous statement. This is a fallacy. Indeed, more expressive systems (range being the most expressive) allow more room for dishonest voting (by nature). If you really want to limit dishonest voting, you could simply take away some expressiveness - try flattening range to approval voting, and viola, goodbye 'rampant' dishonest voting. The cost? results quality! (though still better than plurality and IRV!) See the graphic attached. Why forfeit that quality prematurely though? You could allow the dishonest votes on the more accurate system and the output will suffer, yet will still be better than any other systems. Do we give a higher priority to making it hard to lie, or to getting the most representative results regardless? It's good to have your priorities straight when filing objections.

(To see the above visually, see the image.)

That said, it's understandable to look for voter honesty. But the Gibbard-Satterthwaite theorem shows that no rank-order ballot voting method – not IRV, not Borda, not Condorcet, not any of the infinite number of rank-order voting methods that nobody has even invented yet – can give you full honesty - for all of them, that feat is impossible. With IRV or any such voting method, there are elections in which the only way for you to get a good election result, is to flat-out lie by pretending A > B when your honest belief about that candidate pair is A < B.

Indeed, the better way to achieve a gamed voting system is... to use anything worse than Range. For more on this, check out the book "Gaming the Vote" (NYT review) by William Poundstone.

This is a tremendously important subject – it is not often-enough realized how important. In that vein, I therefore re-emphasize that we should remove "Ranked Choice" from the decision type options. We just might leave our future users quite pleasantly surprised.

C

Connor Tue 4 Dec 2018

I just noticed this implementation is missing one critical feature - the ability to opt-out of rating a particular option at all. At least, it's not clear that this is possible. There are a ton of reasons why this is a necessary aspect of range voting, and forcing zeroes in the place of "blanks" is giving us biased results. Hope to see this fixed soon.

In the future, it would also be a nice feature to allow voters to rank their choices - for those who prefer to just drag-and-drop in order of preference. This would then be reflected automatically in the ratings - distributing their scores equally across the range in the declared order. :)

C

Connor Tue 4 Dec 2018

Yeah, there a lots of numbering schemes with built-in biases, even the 0-100 scale (since it's used so heavily in school, where more than the whole bottom half of the scale was simply "fail") or the 5-star concept (online product ratings). Using negative minimums is fun, but it turns out you can actually solve this problem while also increasing the speed at which people vote by simply not using numbers at all.

This has the added benefit of allowing infinite precision. You don't think about numbers - just position the cursor where it feels right, and click. To see it in action, try out the smartphone app UpVote. It's pretty cool!

GC

Greg Cassel Tue 4 Dec 2018

Thanks for responding in such depth, @adroit . I have mixed feelings. All the hyperlinks above go to the same rangevoting website, which presumably exists purely to promote range voting. You're clearly an enthusiastic advocate and you're welcome to advocate it. However, that website looks to me like lots of theory with thought experiment examples, but practically no use in real serious elections. If it's deeply sound theory which you want people to trust enough to commit serious elections to, then I'd like to see links to it in peer-reviewed professional journals-- probably mathematics journals-- or heck, on Quora, or at least edit the wikipedia page for Score Voting and see how well that holds up. Better yet, of course, get it used in small but serious elections, and build evidence of satisfied users.

I get that range voting is simpler and more expressive than alternatives, and I actually base some of my key decision process models on practically analogous ratings systems. (By contrast, typical political elections-- and ranked voting-- are tertiary subjects for me.) I don't currently understand why you suggest to focus intensely on the strategic voting risk in ranked voting, without discussing how strategies influence range voting. But hey I'm not here to argue against anyone trying ranged voting in any ways they desire! Loomio has it now (as Score Voting) and I hope you explore its performance in varied groups with varied goals. If its effective use-cases exceed my expectations, it'll make my work easier.

Anyway, this conversation reminds me that my main problem with range voting-- and ratings systems in general-- is not the simple and deeply expressive method of rating subjects according to one or more finely graduated dimensions such as Approve/Disapprove. My problem is that ranged results-- and official decisions, when applicable-- are usually determined by (1) 'averaging' by arithmetic mean and (2) ranking the averages. I'll try to make time to focus on alternatives to that procedure.

GC

Greg Cassel Tue 4 Dec 2018

It's quite correct that you don't need to use numbers, and systems with finely-detailed 'sliders' are desirable. However, personal attitude can definitely be positive, negative or (practically) neutral on any specific concept. I think it's helpful to directly enable positive and negative reports on positive and negative feelings. In fact I also think it's highly desirable to enable ambivalent (both positive and negative) reports on specific concepts, although most persons and groups probably aren't going to want to analyze with that much level of detail.

C

Connor Tue 4 Dec 2018

No problem, Greg. It did take some time.
As I requested, I still hope you can find the time to read some of the material before responding. You reacted to the sources, and your skepticism thereof, but not the contents. I know this because you would not have made many of these comments otherwise, since the answers were provided:

rangevoting website, which presumably exists purely to promote range voting.

Nope - the purpose of the site is explicitly stated on their mission page, and they actually advocate other systems than range voting when appropriate. The main idea is to pick the best tool for the job in each case.

that website looks to me like lots of theory with thought experiment examples, but practically no use in real serious elections.

I'm not sure why it "looks like" that to you - it certainly has lots of theoretical examples (it's written by a mathematician after all), but that doesn't somehow mean there is a lack of practical examples. One need only look, to find plenty of real life examples! Actually, I'm not sure how you made it past the front page without knowing about one major example: the Olympics. That seems "real" and "serious" to me :)

If it's deeply sound theory

One way to find out - try to prove it wrong. You'll find plenty of provably valid criticisms against IRV, so be sure not to have double-standards.

which you want people to trust

No, this is exactly what you don't want to require. Any good, mathematically sound principle can be verified without requiring trust - just like the encryption schemes we use for security on the internet today. No need to trust the system - simply look at the theory (or the source code!) and you can see for yourself that it does what it says it does.

We don't use this encryption math in the wild so that we can prove it (in theory) - that would be circular logic. We use this math in the wild because it's proven theory.

then I'd like to see links to it in peer-reviewed professional journals-- probably mathematics journals

Again, just look - as far as mathematical papers, there are many, long, boring, papers, that I didn't link to, but exist nonetheless. math peers have written independently on the topic as well: here's Clay Shentrup confirming that Range voting "produces the highest voter satisfaction index".

or heck, on Quora, or at least edit the wikipedia page for Score Voting and see how well that holds up

Not sure what you mean. Did you read it? The wikipedia page is just fine. In fact, it highlights other advocacy groups besides rangevoting.org (since you want second opinions), and even has a paragraph that effectively answers your tactical voting objections from earlier:

"[The] validity of this problem is further called into question by a 2009 paper which found that "experimental results support the concept of bias toward unselfish outcomes in large elections."[34] The authors observed what they termed ethical considerations dominating voter behavior as pivot probability decreased. This would imply that larger elections, or those perceived as having a wider margin of victory, would result in fewer tactical voters."

Better yet, of course, get it used in small but serious elections, and build evidence of satisfied users.

Again, it's been used in serious elections already, so I'm not sure why this is a concern. (Side note: It's even preferred by evolution.) There is already plenty of evidence voters want it - especially voters who use IRV. This is nothing new.

I don't currently understand why you suggest to focus intensely on the strategic voting risk

Where is this coming from? Looking back on my post, I don't see any suggestions to focus intensely on anything...

in ranked voting

Huh? It was strategic voting in Range voting that I was responding to, not Ranked. I responded to it because you brought it up. It was the only tangible objection you provided, so it was the only one I could 'intensely focus' on.

without discussing how strategies influence range voting.

But that's precisely what we did! Okay I guess you misread something. Strategic Range voting was in fact the only strategic voting we did discuss. To recap:

  1. It's proven that all systems must allow some strategic voting
  2. Unlike other systems, the proven optimal strategy in range voting is dead simple (exaggerate your preferences), and never has you betraying your favorite
  3. Thus this is one of only systems where you can always be semi-honest, and never need to lie, like in IRV and Plurality
  4. Strategy has a relatively mild effect on Range Voting
  5. When strategic voting is at its worst in Range, the effect is, paradoxically, beneficial vs. lesser amounts.
  6. No matter how much tactical voting is employed, Range Voting produces less voter Regret (higher quality) than Plurality and IRV with no strategic voting. This can be seen visually in the bar chart!

That's a quick bullet list on that topic.

But hey I'm not here to argue against anyone trying ranged voting in any ways they desire! Loomio has it now (as Score Voting) and I hope you explore its performance in varied groups with varied goals.

Again, this isn't some new, unexplored, 'experimental' territory, these things are objectively true whether you try them out or not :)

If its effective use-cases exceed my expectations

Not sure where this keeps coming from - the effective use cases are "all of them", but if you don't believe so, it's not like you'll be able to "realize" that just by trying it...

Anyway, this conversation reminds me that my main problem with range voting-- and ratings systems in general-- is not the simple and deeply expressive method of rating subjects according to one or more finely graduated dimensions such as Approve/Disapprove. My problem is that ranged results-- and official decisions, when applicable-- are usually determined by (1) 'averaging' by arithmetic mean and (2) ranking the averages.

Yup, that's the right way to do it. What's the problem? you seem to have forgotten to include why you don't like this.

I'll try to make time to focus on alternatives to that procedure.

Woah there. You never stated the actual problem, so why work on solutions? I've never heard of this objection before, I'm looking forward to seeing the substance of it's argument :)

GC

Greg Cassel Tue 4 Dec 2018

I don't have nearly enough time to reply properly now, @adroit , but I will respond to a couple of bits.

I did indeed use a significant amount of my valuable time to read various parts of the source website. You publicly declared that I hadn't read it because you think I don't understand some things there. I think that's a pretty aggressive way to argue with a stranger who's trying to find common ground, but yes: I did skim over much of the content. I did however read many sentences on the website which gave me the decided opinion that it exists to promote range voting. Perhaps I shouldn't have stated that it exists purely to promote range voting; however, that's quite different than claiming that it exists to promote only range voting for all situations! But, whatever; I'm not here to argue.

one major example: the Olympics. That seems "real" and "serious" to me :)

Yes, the Olympics is very real and serious in its way. However, it's range voting done by professionally accountable judges who hold their scores up in front of the whole world. That's not nearly equivalent or transferable IMO to anonymous voting for public officials, or even for many community use-cases of Loomio Score Polling. Deeply different use-cases require different usage, and accumulating results.

You've spent a lot of time negatively criticizing parts of my comments above. I hope you don't miss the IMO key point that I'm actually encouraging you to use Score Polling and to build convincing evidence of its effectiveness in varied situations. I may reply to more of your critical commentary later if I find time, when I'm not building practical things or recovering from work.

C

Connor Tue 4 Dec 2018

I don't have nearly enough time to reply properly now, @adroit , but I will respond to a couple of bits.
No rush! It can always wait.

I did indeed use a significant amount of my valuable time to read various parts of the source website.

Okay. I'm sorry if I implied that you didn't read any of it, that's not what I meant.

but yes: I did skim over [..] the content.

That's what I meant.

You publicly declared that I hadn't read it because you think I don't understand some things there. I think that's a pretty aggressive way to argue with a stranger

Not quite - not understanding what you read is fine - but on the surface that looks very different from not reading it. Certain questions would not be asked, etc. I don't think there's anything aggressive about asking you to look through it - that's why I phrased it as a request instead of an accusation. I'm sorry it didn't come across to you as politely as I intended.

I did however read many sentences on the website which gave me the decided opinion that it exists to promote range voting.

Well, that is correct. It's run by the "Center for Range Voting", after all. That said, no need to figure out a site's purpose on your own when it has a mission statement :)

Perhaps I shouldn't have stated that it exists purely to promote range voting; however, that's quite different than claiming that it exists to promote only range voting

How? What else could "purely" mean in that context? There isn't a clear difference.

That's not nearly equivalent or transferable IMO to anonymous voting for public officials, or even for many community use-cases of Loomio Score Polling.

Nor does it mean it is not transferable to other scenarios, either. Let's not imply the converse. The statement was simply to show an example of a real and serious usage of Range, not to prove it's universally applicable.

You've spent a lot of time negatively criticizing parts of my comments above.

I'd stick with just "criticizing", since it's not clear what "positively criticizing" would mean in this case. However, I criticize ideas, not people, and I don't think there's anything wrong with that :)

I hope you don't miss the IMO key point that I'm actually encouraging you to use Score Polling and to build convincing evidence of its effectiveness in varied situations.

I definitely didn't miss that, but the scope is too small, and I think that's your point. It's not about me or my choice to use it, it's about waht's objectively better. You're implying that there isn't sufficient evidence already, and trying to pose it as a compromise. I think the topic deserves better than that, that's all.

RG

Rob Guthrie Tue 4 Dec 2018

Right, good call. Maybe we need an X or something next to the slider to remove the value entirely. Any UI suggestions?

To your second paragraph, how does that differ from ranked choice poll?

GC

Greg Cassel Thu 13 Dec 2018

FWIW this is what I found pretty aggressive @adroit , and mostly inaccurate: "You reacted to the sources, and your skepticism thereof, but not the contents." But hey that's fine; I appreciate you redirecting this discussion in a positive and substantial way.

Perhaps I shouldn't have stated that it exists purely to promote range voting; however, that's quite different than claiming that it exists to promote only range voting
How? What else could "purely" mean in that context? There isn't a clear difference.

You seem to have misunderstood my grammatically correct, but eccentric and vague, meaning. Here's another try: "This site exists only to promote range voting" is deeply different than "This site exists to promote range voting as the universally correct voting process." (And, I did not make or imply that second statement.)

Hope that makes sense now. (It's very tangential to me anyway.)

Let's not imply the converse.

No, I wouldn't waste my time arguing that the value of Olympic style voting is automatically and entirely inapplicable to anonymous voting etcetera. Burden of proof (or more accurately, persuasion) is IMO on the persons who promote specific activities in specific social contexts. This relates to me constructively suggesting that we build evidence of range voting successes, instead of debating theories.

You seem to believe there is already plenty of evidence that range voting is 'objectively better' in many or most contexts. Everyone is certainly welcome to believe you, and to practice range voting with you for any & all reasons.

JC

Jaimie Cosmia Thu 14 May 2020

Doesn’t the STAR implementation take what works so well about score voting and add automatic runoff to discourage dishonest voting? I’d love to see it as an option on Loomio.(https://en.m.wikipedia.org/wiki/STAR_voting)

Regardless, I agree with @Connor that the evidence is pretty clear that score voting is the superior method even when voters are tactical.

On a personal/anecdotal note, when I found out about Loomio recently, what really sent me running toward it was seeing that the platform supports score voting (er, polling).

JC

Jaimie Cosmia Thu 14 May 2020

I’m not as convinced as @Connor that ranked-choice shouldn’t even be an option available on the platform, but what I find much, much more problematic is that groups cannot set their own default decision-making method. Why is approval voting the immutable default? Why are the others mere “polls”? I understand that a group could informally agree on any of the polls as their decision-making standard, but I think there’s a lack of psychological weight/seriousness to not setting the default in the group’s code that could lead to a lack of clarity on what counts as a legitimate and final decision.

Furthermore, consider that as the platform expands, decisions might need to interact with smart contracts or some other form of machine code, in which it will be absolutely vital for a default method to be determined so that it can be acted on by the code.

JC

Jaimie Cosmia Thu 14 May 2020

And ime “no opinion” should be the default selection on every option until changed. This helps to ensure that voters aren’t overwhelmed by a long list of options/candidates if they are unfamiliar with many of them. They can simply skip to those they do have an opinion about and ignore the rest. Though, of course, we can hope that with sufficient time voters will research as many of the options as they are able, this isn’t always possible or desirable.

JC

Jaimie Cosmia Thu 14 May 2020

UI suggestion: a button that is selected by default. Having a button helps to prevent the misperception that this is mathematically calculated as “less than zero”, and having it selected by default allows voters to quickly skip through to options they do have an opinion about, without being overwhelmed by those they don’t. As soon as the user moves a slider the “no opinion” button is toggled off. In case of accident, it can be pressed again.

JC

Jaimie Cosmia Thu 14 May 2020

I think @Connor ‘s suggestion, if I may interpret, is that it differs from a ranked-choice poll because it still gives the option of finer granularity to voters who want it, while making the process a tad simpler for those who don’t.

RG

Rob Guthrie Thu 14 May 2020

Thanks for your input @Jaimie Cosmia. In the future the polls will all be more configurable, we'll call this poll templates, and group admins will be able to select which poll templates (or voting processes) are available within the group. They'll also have room for a link to policy/documentation to define what a proposal means in the context of the group.

I'll keep the no opinion option in mind. I get what you're saying and agree.