Loomio

The Open Organization Maturity Model

I
Ilja Public Seen by 414

In the lab of 26/08/2017 it was decided to start a project to help asses the «openness» of the pirate party BE using The Open Organization Maturity Model. This thread is here to share ideas and discuss about this project.

I

Ilja Mon 28 Aug 2017

I've 'rearranged' the file into something that looks a bit more like a questionnaire[1] :)
Under 'Inclusivity', I had to merge the 4th and 5th point of level 2, for the rest there was a very clear 1 to 1 relationship between the several levels.

[1] https://wiki.pirateparty.be/File:The_Open_Organization_Maturity_Questionaire.odt

I

Ilja Mon 28 Aug 2017

On the wiki Limesurey and Framadate are mentioned as possible tools to create an automated survey. I'll check them out, if anybody has any other idea's (preferably libre), do let us know!

I

Ilja Mon 28 Aug 2017

Hmm... Framadate (which is basically OpenSondage) only does polls (one question max), while we need surveys. Limesurvey is open source, but I don't immediately find anything that is free in price. There appears to be a package for Yunohost though[1], but when I try to install, the installation fails… :( On github I already see two issues from people that had issues installing. There are other gratis tools, but I'd rather waitwith using those until we have tried the survey manually.

[1] https://github.com/YunoHost-Apps/limesurvey_ynh

TF

Thierry Fenasse Mon 28 Aug 2017

We may have to try Tellform, a FOSS version of Typeform.com

H

HgO Mon 28 Aug 2017

I tried Framaforms a bit, but I'm afraid it's too buggy :(

TF

Thierry Fenasse Mon 28 Aug 2017

I've took the liberty to invite people to help on translation through the Babel sub group.

H

HgO Tue 29 Aug 2017

Some details about Framaforms : the edition of the form is very difficult because of bugs. For instance, if I want to give several time the same title "In my organization" to groups of radio buttons, the software will give the same ID "in_my_organization". Thus, when I save the form, only one group of radio buttons is kept, which can be very frustrating >.< If I try to change those IDs manually, I can't unless I try some tricks...

Anyways, here is the result (for transparency) : https://framaforms.org/open-organization-maturity-model-1503955871

Also, I'm wondering two things:
* Are the different "questions" mandatory or optional ?
* Wouldn't be better to start each group with a common sentence (e.g. "Materials that are part of decision-making practices..."), so that you don't repeat the sentence three times ?

TF

Thierry Fenasse Tue 29 Aug 2017

  • Wouldn't be better to start each group with a common sentence (e.g. "Materials that are part of decision-making practices..."), so that you don't repeat the sentence three times ?

Yes it would be really better and easier to answer because it splits the common content and gives more attention to the difference between Level 1, 2 and 3.

It can help the reader to better «feel» the difference about «something» regarding Transparency for instance.

  • Are the different "questions" mandatory or optional ?

If we consider the whole document as a single form, those questions about Transparency, Inclusivity, Adaptability, Collaboration, Community can represent a «big questionnaire» and may suggest that the questions are not mandatory to «jump» to something else without being stuck in a required field.

Then it makes me think ... that it would be better to consider 5 questionnaires regarding the 5 topics of «openness» (Transparency, Inclusivity, Adaptability, Collaboration, Community). This can help to treat them as different aspects about an organization, a crew or a project (squad) if we consider the pirates.

I

Ilja Tue 29 Aug 2017

And does it work then eventually? Otherwise, the Tellform Thierry suggested also seems quite nice :o
I've also been thinking, I don't know how exactly you can see the results, but maybe asking what group you are filling the form in for might be a good start question as well :)
I think I'll play with the forms a bit(this week or this weekend I think) to see and feel their behaviour :)

I

Ilja Tue 29 Aug 2017

Hah :D, I didn't see the 'see more' button, so in reply to the rest of the post :p

The questions really talk about sepperate things, so I would think optional? Although we'll have to figure out how to interpret the results then... I hadn't actually tought about this before :/

The common sentence part would indeed be better ^ I'll change it in the .odt file (sometime this week or so)

I

Ilja Tue 29 Aug 2017

I'm not sure about splitting up. There are different topics, so they can be clearly sepperated in the form as far as I'm concerned. But having five different forms seems like overkill. After all, we do want the whole picture, don't we? Unless we want to give smaller questionnaires (with different people getting different topics), but to be representative, we can only do something like that with larger groups (which is not the case ATM)

TF

Thierry Fenasse Tue 29 Aug 2017

Just a question ... why an .odt instead of a wiki page? (other reasons that you did it on the train and that it was easier to work off-line :p )

I

Ilja Wed 30 Aug 2017

Huh, you're right :/ A wiki page makes more sense in about every way, doesn't it? :/
I fixed it :p https://wiki.pirateparty.be/index.php?title=Open_Organization_Maturity_Model/Questionaire
This is now the content of the .odt, but I'll change for the 'common sentence' thing, and I'll also see where they speak about 'leaders', so I can make it a bit better towards PPBE. I'll also change the links on the project page to refer to this page instead of the .odt

I

Ilja Tue 29 Aug 2017

Ohw, and should we already try it out manually? We could already ask a crew or squad or wathever if they want to fill it in internally for their own crew/squad. This could help them think and give feedback about their own openness, and it would give us information on how good the document actually is for the pirate party.
* I think the IT-squad are swell guys who'd be open for something like this ;)

I

Ilja Wed 30 Aug 2017

Common sentences is done on the wiki page. Some points didn't have common sentences. I've left those as they where. (BTW, links on the project page for the questionaire are also refering to this page now.)

H

HgO Wed 30 Aug 2017

@iljabaert Yeah, the result is quite nice, but the creation is too cumbersome in my opinion.

For the score, we can do something like that : level 1 = one point, level 2 = two points, and level 3 = ... three points :D Then, we count the number of questions answered, and we divide by this number. To get a percentage, we should also divide by 3. For instance, imagine I answered four questions : 1 level 1, 2 level 2, and 1 level 3. Then I would obtain : (1*1 + 2*2 + 3*1)/(4*3) = 8/12 = 67 %

Yes, I noticed there isn't a common sentence every time... I'm wondering if it would possible to rearrange the sentences in order to have a common sentence ? Would it be cheating ? :/

Yes, we could try it on a squad or crew. Maybe ITSquad, or WikiSquad, or Internal Democracy ^ It doesn't really matter :)

Ah, and I don't understand your changes made to the questionnaire, on the wiki :sweat_smile:

I

Ilja Wed 30 Aug 2017

Ah :D perfect for the score! And simple actually :p

I don't think rearanging is necessary, it seems ok now I think.

What is it that you don't understand? :/ I may have made some weird comments when I was trying out how to get rid of the 'leader' things, if that's the problem :p (and maybe should have done it all in one swoop instead of sepperate per section while still trying to figure out how to do it :/ )

H

HgO Wed 30 Aug 2017

The only problem I see is that we could never obtain 0% with that method... For instance, if I answered four questions with level 1, I would obtain 4/12 = 33%... Soo if we do level 1 = 0 point, level 2 = 1 point, etc. the same example as before would give : (1*0 +2*1+1*2)/(4*2) = 5/8 = 62.5 %. That's better :D

I didn't see that you wanted to get rid of the "leaders", that's why I got confuse. Now I understand, although I must check in the document if it's that simple to get rid of them :p

I

Ilja Wed 30 Aug 2017

Yes! please do check!

And yes level 1 = 0 points is indeed even better ;)

I

Ilja Wed 30 Aug 2017

Hih :p level 1 = 1 point, because you get 33% for effort ;)

EDIT: I'm just joking of course ;)

I

Ilja Wed 30 Aug 2017

Changing the 'leaders' things is done :) I left 'leaders' under the 2nd section of transparency, because it felt like it was fitting there. I also left it in the 'Inclusivity' description. For the rest I did my best :) The last point in the 'Inclusivity' is a bit weird now though :/ There's no real difference between 2 and 3 I think. Not sure what to do with it :/

H

HgO Wed 30 Aug 2017

For the form, I found https://surveyjs.io which does a great job :D It is also available as a js library, so we could tweak it a bit if we want to. For instance, we could combine the results with a chart (such as D3js). In my mind, we could display a radar chart at the end of the questionnaire :)

Here is the form produced with surveyjs : https://dxsurvey.com/published?id=0cd46408-1894-4649-96c5-bd464d665d4f

Note : I changed "the organization" by "we", because it sounded strange to have "In my organization, the organization...". What do you think of this solution ? :) I will update the wiki page with my changes tomorrow if it's okay for you ;)

I

Ilja Thu 31 Aug 2017

Looks great indeed :o

Yes, please feel free to make the changes you think are necessary :p

I

Ilja Thu 31 Aug 2017

How exactly do you see the radar chart? Would that be different colors for different sections with the radars being the groups/crews? or how do you see it?
Anyhow, I'm all for visualization as long as it gives a clear and, most importantly, a correct impression of reality :) (and I think this radar chart does this)

H

HgO Thu 31 Aug 2017

Hey ! Here is what I was thinking about : https://github.com/PiratePartyBE/open-org-survey
Just clone this repository, and then do

npm update

to install the js dependencies. Finally, open the index.html file in your favorite web browser (i.e. Firefox :sunglasses: :fox:)

Of course, design can be improved, it's just a proof-of-concept ;)

For the changes on the wiki, I've been really tired all the day, so I'm not sure I'll be able to do it this evening :/

EDIT: Attached is an example of radar chart that is displayed when the survey is finished.

TF

Thierry Fenasse Sat 2 Sep 2017

You are a machine! I would like to share the results of my personal answers about Pirate Party Belgium, but I cannot attached à document to this comment.

H

HgO Thu 31 Aug 2017

For testing purpose, I created a github page to show what the questionnaire looks like : https://piratepartybe.github.io/open-org-survey/ ;) You can play around with that and see what should be improved :)

I

Ilja Mon 4 Sep 2017

I tried for PPBE in general, it's clear to me now that different group can behave quite differently :p

Maybe a good thing to have would be to also mention the scores. You can see them by hoovering over the points, but that's not very 'accesseble'. I've also been thinking about the chart and I believe it breaks the 'good practices' for visual representation[1][2]. For example, you'll never be able to get the circles full, so even at 100% the chart won't 'look' like a 100% score. A second thing that may not be okey about this type of chart is that there seems to be 'something' between, for example, 'transparancy' and 'inclusivity', while there clearly is not. And then a third, you can't see the small differences between different subjects. I like the way it looks, but I do believe a simple bar-chart would be more appropriate and more clear (even though it looks boring and dull :p ).

I also assume that these are induvidual scores? This might be OK for smaller groups to just give an indication. If we'd want to bring this to the whole party, we'd need to store the results, calculate general results from that , and we should be able to see the answers that were given so that we can analyse a bit deeper were the problems lay. Is there an easy way to do that? (Or is this already possible?)

BTW, My scores are (for ppbe):

transparency: 67%

inclusivity: 42%

adaptability: 58%

collaboration: 33%

community: 38%

(They are quite low :/ Most I think is because of lack of clear procedures.)

[1] http://callingbullshit.org/tools/tools_proportional_ink.html
[2] http://callingbullshit.org/tools/tools_misleading_axes.html

I

Ilja Mon 4 Sep 2017

I see there is a function sendDataToServer(survey). As far as I understand it, this logs the data and puts it on the chart? I assume to store data... that we'd also need a PHP or Java (or wathever) backend (and modify this function)?

H

HgO Mon 4 Sep 2017

I'm in holiday, so I'm writing just a "quick" reply:
* Just in case, the example that I gave is just an example. I didn't fulfilled the questionnaire yet.
* For me, the graph was great, so I'm not really motivated to choose another one... :( Maybe you can have a look at https://github.com/d3/d3/wiki/Gallery to see if another graph would be better ?
* I agree that the percentage should be more visible. This was the default behavior of the graph ;)
* I agree that nobody would get a 100% score... Is that a problem, though ? We could just make a notice saying that those numbers are just an indication of the organization's openness ?
* Yes, we could quite easily store those results on a server, using a REST api, etc. At the moment, the test doesn't send or store anything.
* Indeed, those are individual scores. Gathering results is another issue, and could be done in many ways... I don't know which one would be the easiest :p For instance:
* Let people tell for which organization they are taking the survey
* Let organizations indicate an e-mail to which send the results
* Let organizations gather themselves the results
* ...
* Yes, the function sendDataToServer was used by surveyjs to store data on their server. This function is called once the questionnaire is completed. Right now, it just displays the graph and doesn't do anything else. I just forgot to rename the function :p So, we could do whatever we want there, storing data, displaying them in a copyable and sharable way, etc.

I

Ilja Mon 4 Sep 2017

Well of course someone could get a 100% score :p That's what I always aim for ^

For the graph I don't think it's actually missleading, so I'm not against the graph, I'm just wondering. And now that I think more about it, you are right that the numbers are just an indication, this would then imply that the difference in score between several subjects isn't really relevant either, which kinda beats my argument :p Pff :/ maybe I'm just overthinking this (it wouldn't be the first time ;) )

Yes, I was thinking REST api as well :) This will then also give us a way of gathering and storing results automatically :) I'll see if I can get something done for that :)

For the rest the survey definitely looks great! It will be awesome once this is done :o

And of course, Have a happy holiday! ;)