Loomio
August 28th, 2017 08:32

The Open Organization Maturity Model

Ilja Baert
Ilja Baert Public Seen by 414

In the lab of 26/08/2017 it was decided to start a project to help asses the «openness» of the pirate party BE using The Open Organization Maturity Model. This thread is here to share ideas and discuss about this project.

Ilja Baert

Ilja Baert August 28th, 2017 08:59

I've 'rearranged' the file into something that looks a bit more like a questionnaire[1] :)
Under 'Inclusivity', I had to merge the 4th and 5th point of level 2, for the rest there was a very clear 1 to 1 relationship between the several levels.

[1] https://wiki.pirateparty.be/File:The_Open_Organization_Maturity_Questionaire.odt

Ilja Baert

Ilja Baert August 28th, 2017 09:02

On the wiki Limesurey and Framadate are mentioned as possible tools to create an automated survey. I'll check them out, if anybody has any other idea's (preferably libre), do let us know!

Ilja Baert

Ilja Baert August 28th, 2017 10:06

Hmm... Framadate (which is basically OpenSondage) only does polls (one question max), while we need surveys. Limesurvey is open source, but I don't immediately find anything that is free in price. There appears to be a package for Yunohost though[1], but when I try to install, the installation fails… :( On github I already see two issues from people that had issues installing. There are other gratis tools, but I'd rather waitwith using those until we have tried the survey manually.

[1] https://github.com/YunoHost-Apps/limesurvey_ynh

Thierry Fenasse

Thierry Fenasse August 28th, 2017 19:41

We may have to try Tellform, a FOSS version of Typeform.com

Thierry Fenasse

Thierry Fenasse August 28th, 2017 19:48

I've took the liberty to invite people to help on translation through the Babel sub group.

HgO

HgO August 28th, 2017 22:04

I tried Framaforms a bit, but I'm afraid it's too buggy :(

HgO

HgO August 29th, 2017 08:32

Some details about Framaforms : the edition of the form is very difficult because of bugs. For instance, if I want to give several time the same title "In my organization" to groups of radio buttons, the software will give the same ID "in_my_organization". Thus, when I save the form, only one group of radio buttons is kept, which can be very frustrating >.< If I try to change those IDs manually, I can't unless I try some tricks...

Anyways, here is the result (for transparency) : https://framaforms.org/open-organization-maturity-model-1503955871

Also, I'm wondering two things:
* Are the different "questions" mandatory or optional ?
* Wouldn't be better to start each group with a common sentence (e.g. "Materials that are part of decision-making practices..."), so that you don't repeat the sentence three times ?

Thierry Fenasse

Thierry Fenasse August 29th, 2017 10:13

  • Wouldn't be better to start each group with a common sentence (e.g. "Materials that are part of decision-making practices..."), so that you don't repeat the sentence three times ?

Yes it would be really better and easier to answer because it splits the common content and gives more attention to the difference between Level 1, 2 and 3.

It can help the reader to better «feel» the difference about «something» regarding Transparency for instance.

  • Are the different "questions" mandatory or optional ?

If we consider the whole document as a single form, those questions about Transparency, Inclusivity, Adaptability, Collaboration, Community can represent a «big questionnaire» and may suggest that the questions are not mandatory to «jump» to something else without being stuck in a required field.

Then it makes me think ... that it would be better to consider 5 questionnaires regarding the 5 topics of «openness» (Transparency, Inclusivity, Adaptability, Collaboration, Community). This can help to treat them as different aspects about an organization, a crew or a project (squad) if we consider the pirates.

Ilja Baert

Ilja Baert August 29th, 2017 17:14

And does it work then eventually? Otherwise, the Tellform Thierry suggested also seems quite nice :o
I've also been thinking, I don't know how exactly you can see the results, but maybe asking what group you are filling the form in for might be a good start question as well :)
I think I'll play with the forms a bit(this week or this weekend I think) to see and feel their behaviour :)

Ilja Baert

Ilja Baert August 29th, 2017 17:18

Hah :D, I didn't see the 'see more' button, so in reply to the rest of the post :p

The questions really talk about sepperate things, so I would think optional? Although we'll have to figure out how to interpret the results then... I hadn't actually tought about this before :/

The common sentence part would indeed be better ^ I'll change it in the .odt file (sometime this week or so)

Ilja Baert

Ilja Baert August 29th, 2017 17:25

I'm not sure about splitting up. There are different topics, so they can be clearly sepperated in the form as far as I'm concerned. But having five different forms seems like overkill. After all, we do want the whole picture, don't we? Unless we want to give smaller questionnaires (with different people getting different topics), but to be representative, we can only do something like that with larger groups (which is not the case ATM)

Ilja Baert

Ilja Baert August 29th, 2017 17:32

Ohw, and should we already try it out manually? We could already ask a crew or squad or wathever if they want to fill it in internally for their own crew/squad. This could help them think and give feedback about their own openness, and it would give us information on how good the document actually is for the pirate party.
* I think the IT-squad are swell guys who'd be open for something like this ;)

Thierry Fenasse

Thierry Fenasse August 29th, 2017 17:43

Just a question ... why an .odt instead of a wiki page? (other reasons that you did it on the train and that it was easier to work off-line :p )

Ilja Baert

Ilja Baert August 30th, 2017 17:18

Huh, you're right :/ A wiki page makes more sense in about every way, doesn't it? :/
I fixed it :p https://wiki.pirateparty.be/index.php?title=Open_Organization_Maturity_Model/Questionaire
This is now the content of the .odt, but I'll change for the 'common sentence' thing, and I'll also see where they speak about 'leaders', so I can make it a bit better towards PPBE. I'll also change the links on the project page to refer to this page instead of the .odt

Ilja Baert

Ilja Baert August 30th, 2017 17:50

Common sentences is done on the wiki page. Some points didn't have common sentences. I've left those as they where. (BTW, links on the project page for the questionaire are also refering to this page now.)

HgO

HgO August 30th, 2017 18:31

@iljabaert Yeah, the result is quite nice, but the creation is too cumbersome in my opinion.

For the score, we can do something like that : level 1 = one point, level 2 = two points, and level 3 = ... three points :D Then, we count the number of questions answered, and we divide by this number. To get a percentage, we should also divide by 3. For instance, imagine I answered four questions : 1 level 1, 2 level 2, and 1 level 3. Then I would obtain : (1*1 + 2*2 + 3*1)/(4*3) = 8/12 = 67 %

Yes, I noticed there isn't a common sentence every time... I'm wondering if it would possible to rearrange the sentences in order to have a common sentence ? Would it be cheating ? :/

Yes, we could try it on a squad or crew. Maybe ITSquad, or WikiSquad, or Internal Democracy ^ It doesn't really matter :)

Ah, and I don't understand your changes made to the questionnaire, on the wiki :sweat_smile:

Ilja Baert

Ilja Baert August 30th, 2017 18:45

Changing the 'leaders' things is done :) I left 'leaders' under the 2nd section of transparency, because it felt like it was fitting there. I also left it in the 'Inclusivity' description. For the rest I did my best :) The last point in the 'Inclusivity' is a bit weird now though :/ There's no real difference between 2 and 3 I think. Not sure what to do with it :/

Ilja Baert

Ilja Baert August 30th, 2017 18:53

Ah :D perfect for the score! And simple actually :p

I don't think rearanging is necessary, it seems ok now I think.

What is it that you don't understand? :/ I may have made some weird comments when I was trying out how to get rid of the 'leader' things, if that's the problem :p (and maybe should have done it all in one swoop instead of sepperate per section while still trying to figure out how to do it :/ )

HgO

HgO August 30th, 2017 19:02

The only problem I see is that we could never obtain 0% with that method... For instance, if I answered four questions with level 1, I would obtain 4/12 = 33%... Soo if we do level 1 = 0 point, level 2 = 1 point, etc. the same example as before would give : (1*0 +2*1+1*2)/(4*2) = 5/8 = 62.5 %. That's better :D

I didn't see that you wanted to get rid of the "leaders", that's why I got confuse. Now I understand, although I must check in the document if it's that simple to get rid of them :p

Ilja Baert

Ilja Baert August 30th, 2017 19:14

Yes! please do check!

And yes level 1 = 0 points is indeed even better ;)

Ilja Baert

Ilja Baert August 30th, 2017 19:18

Hih :p level 1 = 1 point, because you get 33% for effort ;)

EDIT: I'm just joking of course ;)

HgO

HgO August 30th, 2017 21:56

For the form, I found https://surveyjs.io which does a great job :D It is also available as a js library, so we could tweak it a bit if we want to. For instance, we could combine the results with a chart (such as D3js). In my mind, we could display a radar chart at the end of the questionnaire :)

Here is the form produced with surveyjs : https://dxsurvey.com/published?id=0cd46408-1894-4649-96c5-bd464d665d4f

Note : I changed "the organization" by "we", because it sounded strange to have "In my organization, the organization...". What do you think of this solution ? :) I will update the wiki page with my changes tomorrow if it's okay for you ;)

Ilja Baert

Ilja Baert August 31st, 2017 09:04

Looks great indeed :o

Yes, please feel free to make the changes you think are necessary :p

Ilja Baert

Ilja Baert August 31st, 2017 09:23

How exactly do you see the radar chart? Would that be different colors for different sections with the radars being the groups/crews? or how do you see it?
Anyhow, I'm all for visualization as long as it gives a clear and, most importantly, a correct impression of reality :) (and I think this radar chart does this)

HgO

HgO August 31st, 2017 17:25

Hey ! Here is what I was thinking about : https://github.com/PiratePartyBE/open-org-survey
Just clone this repository, and then do

npm update

to install the js dependencies. Finally, open the index.html file in your favorite web browser (i.e. Firefox :sunglasses: :fox:)

Of course, design can be improved, it's just a proof-of-concept ;)

For the changes on the wiki, I've been really tired all the day, so I'm not sure I'll be able to do it this evening :/

EDIT: Attached is an example of radar chart that is displayed when the survey is finished.

HgO

HgO August 31st, 2017 22:52

For testing purpose, I created a github page to show what the questionnaire looks like : https://piratepartybe.github.io/open-org-survey/ ;) You can play around with that and see what should be improved :)

Thierry Fenasse

Thierry Fenasse September 2nd, 2017 11:32

You are a machine! I would like to share the results of my personal answers about Pirate Party Belgium, but I cannot attached à document to this comment.

Ilja Baert

Ilja Baert September 4th, 2017 09:00

I tried for PPBE in general, it's clear to me now that different group can behave quite differently :p

Maybe a good thing to have would be to also mention the scores. You can see them by hoovering over the points, but that's not very 'accesseble'. I've also been thinking about the chart and I believe it breaks the 'good practices' for visual representation[1][2]. For example, you'll never be able to get the circles full, so even at 100% the chart won't 'look' like a 100% score. A second thing that may not be okey about this type of chart is that there seems to be 'something' between, for example, 'transparancy' and 'inclusivity', while there clearly is not. And then a third, you can't see the small differences between different subjects. I like the way it looks, but I do believe a simple bar-chart would be more appropriate and more clear (even though it looks boring and dull :p ).

I also assume that these are induvidual scores? This might be OK for smaller groups to just give an indication. If we'd want to bring this to the whole party, we'd need to store the results, calculate general results from that , and we should be able to see the answers that were given so that we can analyse a bit deeper were the problems lay. Is there an easy way to do that? (Or is this already possible?)

BTW, My scores are (for ppbe):

transparency: 67%

inclusivity: 42%

adaptability: 58%

collaboration: 33%

community: 38%

(They are quite low :/ Most I think is because of lack of clear procedures.)

[1] http://callingbullshit.org/tools/tools_proportional_ink.html
[2] http://callingbullshit.org/tools/tools_misleading_axes.html

Ilja Baert

Ilja Baert September 4th, 2017 09:12

I see there is a function sendDataToServer(survey). As far as I understand it, this logs the data and puts it on the chart? I assume to store data... that we'd also need a PHP or Java (or wathever) backend (and modify this function)?

HgO

HgO September 4th, 2017 12:05

I'm in holiday, so I'm writing just a "quick" reply:
* Just in case, the example that I gave is just an example. I didn't fulfilled the questionnaire yet.
* For me, the graph was great, so I'm not really motivated to choose another one... :( Maybe you can have a look at https://github.com/d3/d3/wiki/Gallery to see if another graph would be better ?
* I agree that the percentage should be more visible. This was the default behavior of the graph ;)
* I agree that nobody would get a 100% score... Is that a problem, though ? We could just make a notice saying that those numbers are just an indication of the organization's openness ?
* Yes, we could quite easily store those results on a server, using a REST api, etc. At the moment, the test doesn't send or store anything.
* Indeed, those are individual scores. Gathering results is another issue, and could be done in many ways... I don't know which one would be the easiest :p For instance:
* Let people tell for which organization they are taking the survey
* Let organizations indicate an e-mail to which send the results
* Let organizations gather themselves the results
* ...
* Yes, the function sendDataToServer was used by surveyjs to store data on their server. This function is called once the questionnaire is completed. Right now, it just displays the graph and doesn't do anything else. I just forgot to rename the function :p So, we could do whatever we want there, storing data, displaying them in a copyable and sharable way, etc.

Ilja Baert

Ilja Baert September 4th, 2017 14:43

Well of course someone could get a 100% score :p That's what I always aim for ^

For the graph I don't think it's actually missleading, so I'm not against the graph, I'm just wondering. And now that I think more about it, you are right that the numbers are just an indication, this would then imply that the difference in score between several subjects isn't really relevant either, which kinda beats my argument :p Pff :/ maybe I'm just overthinking this (it wouldn't be the first time ;) )

Yes, I was thinking REST api as well :) This will then also give us a way of gathering and storing results automatically :) I'll see if I can get something done for that :)

For the rest the survey definitely looks great! It will be awesome once this is done :o

And of course, Have a happy holiday! ;)