Ecosystem technical architecture

@asimong how do you think those technical architecture approaches fit with organizational approaches?

To the extent that organizational forms have been discussed among OAE participants, we usually assumed that components would be developed independently by different people in different organizations, who would collaborate and make their components work together as it suited them. Pretty light weight.

A stack approach would require a different form of organization, I think, which could still collaborate with other orgs with different forms for API or message-based components.

A stack approach would require a different form of organization, I think, which could still collaborate with other orgs with different forms for API or message-based components.

Interesting question. I tend to think that we need to support both a loosely coupled stack approach (such as the one that popped up just now in Bob's last comment), and a message-based approach.

I'm not sure the technical architecture is that different between the loose stack and the message structures. But that might be because I'm not super technical in the architecture department.

I do think this discussion is really important to have though, the sooner the better. And I'll be very interested in how we end up thinking about organization in relation to those decisions.

We have a lot more experience with stack or API architectures than message-based. While Kamasi uses the VF vocabulary, it is not message based, it uses the API in a client-server mode where the client is always the client, and the server is always the server. And at this stage of Kamasi, the client is always talking to the same server (OCP), although that may change over time. I know @ivan116 wants it to be able work on any stack.

I think in a message-based architecture, the components would be much more loosely connected, might be separately hosted on different domains, and if the components were carrying on conversations, they would alternate between "client" and "server" roles, as I sketched out in this github issue.

@matslats Ads API, described in that long set of messages above, would work almost like a message-passing setup, in that separately-hosted components would POST and GET offers and needs via his API.

I hope I am communicating clearly here. Please let me know if I'm not.

Huge appreciation to Bob @bobhaugen for giving us the detail and summarising succinctly!

Some disclaimers are due from me... I know next to nothing on the technical architecture front. I have some amateur coding experience only. My background, and my approach, is very human-centred. I've been fairly deeply into HCI, soft systems, socio-technical, semantics, ontology, that kind of thing.

First, on the API/vocab issue. I think I follow you, Bob. APIs, yes, but if APIs use elements with different, incompatible semantics, there's going to be a quite possibly intractable translation challenge. So, maybe, APIs based on common vocabularies and semantics.

Now to the human interface point of view: whether a set of apps presents to the user as one unified set or a set of separate apps linked behind the scenes, I don't think one can answer in the abstract. What does seem to matter, a lot, is the apparent complexity at any one point in the interaction. So: structure any interaction with clearly delineated decision points, and present all the information relevant to any decision within easy reach of the place where the decision is made. That's how to support humans making decisions, in my mind at least.

While there are purely technical matters that are best hidden (for the above reason) from the end user, I'm a firm believer that the concepts that are used to represent things in the ontology of relevance to end users should be comprehensible to those end users, at least in outline. That way, it will be much easier to build a set of services that actually make sense to users.

Or take it from a different angle... the technical architecture should be built on top of a conceptual architecture that models the common understanding of common users ("commoners"). Added to that, sure, we may need further elements of the conceptual framework that are relevant to developers, and not to users. But that's OK. Though it's liable to change as the technology changes in the future.

And, in my view, the kind of conceptual model diagrams that are used in the above discussion, and the kind that are used by REA people in explaining REA, sit in this good space of models that are intended for common agreement — some, OK, are more technically oriented, but anyway.

I'd really be interested to know if there is disagreement on any of the above issues. What's plain to me might be obscure to others, or just plain mistaken!

There are a couple of architectural memes that I have picked up over recent years. One is "small pieces loosely joined". Another is the general concept of linked data. Those two are not enough, by any means, and I wonder if there are other common memes — more commonly understood memes — that we might use as principles for the technical architecture underlying the ecosystem?

At this point, I'd like to end simply by saying, that's where I'm coming from in suggesting that we look at a conceptual model that can be shared with users first, and ensure that we build the more technical conceptual model on top of that. Having said that, I fully recognise that I am talking in too abstract a way to be really useful, and would value engaging in the actual job of building such models. With the aim of providing a good foundation for a technical architecture.

Like I've been saying elsewhere, the nature of the task of building common conceptual models means that it is almost impossible for one person alone to do it successfully. The coordinators of a model need to be listening, carefully, to how end users think; how they conceive of, how they construe, the tasks that we are trying to facilitate. If different users themselves think incompatibly, then I'd say it is the time for a campaign of education, thought of in a constructivist way, not in a transmissive way, of course. :smiley:

@asimong thanks for very clear comments and questions.

if APIs use elements with different, incompatible semantics, there's going to be a quite possibly intractable translation challenge. So, maybe, APIs based on common vocabularies and semantics.

Yes and yes. That was the reason that the original OAE very loosely decided that a common vocabulary was critical.

I'm a firm believer that the concepts that are used to represent things in the ontology of relevance to end users should be comprehensible to those end users, at least in outline. That way, it will be much easier to build a set of services that actually make sense to users.

We agree, but it's a challenge to do.

the nature of the task of building common conceptual models means that it is almost impossible for one person alone to do it successfully. The coordinators of a model need to be listening, carefully, to how end users think; how they conceive of, how they construe, the tasks that we are trying to facilitate.

I don't know if you've followed on we have worked in Value Flows, and I think many of the OAE participants do things in very similar ways.

We always develop software for live groups that will collaborate in the process: talk to us about requirements, how they do things, and test the system as we develop it incrementally.

We work in a spirals, between deep dives into particular communities, and spiraling back to Value Flows to summarize what we have learned and improve the vocabulary.

Over several years, some of us have developed concepts and components that we re-use in the next engagement. But we never develop software with no live users in the mix.

Now, sometimes this all works better than others, and we still have lots of communication problems and misfits between the software and the users, but it does get better. (I think.) Sometimes the set of components and concepts that we have developed over time does not work so well for the next group, for example. One of the goals of OAE was to develop a more flexible set of components that could be assembled by a community themselves.

The Kamasi project from @ivan116 is an interesting hybrid. It has a stack, and an API, and the components in the stack communicate using the VF vocabulary. And organizationally, there's a small group of devs who work very closely with each other. If anybody is interested, either I or Lynn or Ivan could go into more detail about how it works and why it works that way.

There are a couple of architectural memes that I have picked up over recent years. One is "small pieces loosely joined". Another is the general concept of linked data. Those two are not enough, by any means, and I wonder if there are other common memes — more commonly understood memes — that we might use as principles for the technical architecture underlying the ecosystem?

Those two principles were shared by the original OAE gang, along with some particular concepts about linked data vocabularies (REA, Semantic Web formats, etc). This new revival of OAE brings other vocabularies (ontologies) into the mix: for example, Communecter pivot vocab, the PAIR vocab from Virtual Assembly, and the Food Data Consortium vocab. Thus the need to translate between vocabularies.

Great to hear that we've been on the same page. Yes, I agree with your statement of need.

Can we (and if so, how can we) map out the different vocabularies that we want to translate between? I would find it useful if, in every case, we can get in close touch with at least one of the authors of the vocab. That enables deeper one-to-one dialogue and mutual human understanding; a good precursor to writing translations. I guess it's similar to having at least one native speaker of each language on hand when doing natural language translation.

Can we (and if so, how can we) map out the different vocabularies that we want to translate between?

Yes, we are starting next week to map between the VF vocab and the Communecter pivot vocab. We are in contact with people from Virtual Assembly, and also the Data Food Consortium.

For natural language translation, the Fair Coop community translated all of their OCP software using Transifex. We're looking into how to translate Kamasi. I don't have personal experience in doing so; one of the Fair Coop devs led the process so far.

P.S. maybe we need to add "translatable" to the Open App requirements somewhere? Like, Fair Coop was able to translate the OCP user interface from English to Spanish and allow each user to select their preferred language (of those two choices) because the base OCP software offered features for translating the UI. Otherwise, it would have been really hard.