Changes to the diaspora protocol

As you should be aware of if you want to take place in this discussion, diaspora never had or has a fixed, well defined protocol. Because of that, stuff is sometimes broken and writing actual documentation is impossible. In the past, other networks started implementing our protocol by basically reverse engineering our stuff. While this is fine, we obviously are not able to support this in any way. In fact, our wiki page about the protocols used clearly states

Implementers of this protocol should be advised, though, that Diaspora is in Alpha, and as such, this document is not authoritative, and may lag behind the reference implementation

This is a pretty clear disclaimer. Because of that, no core member of the project ever stated the communication with other networks is a thing we support and that we can/will assure this stuff does not break. In fact, I am not even aware of any of the third party networks devs (except Michael Vogel for some debugging) ever talked to members of the core team to discuss about best practices here.

In our ongoing endeavor to move parts of the federation code inside the new gem, some changes to the protocol have to be made. At the moment, all changes were made to get closer to the originating standards like Salmon to ensure even more compatibility in the future. Even though those changes have been made, we (as in Benjamin, Jonne and myself) ensured that no older diaspora pods communication breaks. The newer 0.5.0.0 releases are much closer to the originating standards but still compatible with very old diaspora releases.

Third party networks have implemented old diaspora federation parts in the past, but they have done a pretty bad job at it. The stuff Friendica sent in never was even close to match the stuff diaspora sends, but it has worked with some hacks on both sides.

This discussion is getting kinda funny here because you are talking about a "diaspora protocol" as if this is something that even exists. diaspora never had a fixed protocol and even the third party networks never did the same we did. So apparently, not even third party implementers were interested in keeping the same standards. Also, at this state, I would like to point out that YOU, Jason, even liked our plans which we are realizing at the moment three years ago. The first discussions about moving the federation inside a separate module were started 2 years ago and even some code was written by Florian. Think about that for a second. I consider three years a pretty good time.

Yes, there were changes made in the new Gem which is actually already included in the diaspora core. You even commented on some of Benjamins pull request and never raised issues and now you act like you never had the chance to participate. I am a bit confused now.

All changes made to the current gem are made in consideration of keeping old diaspora instances alive and at the moment, no one actively involved in those steps has plans on removing said compatibility, because that would be pretty much stupid. In addition, every third party network implementing exactly those old implementations on old pods will still work fine.

If someone added dirty hacks (that were never used by us) to their protocol and now it breaks? Well. Too bad. We keep our old stuff but we are not able to support hacks made by everyone. This would be pretty insane.

Next, the term "stakeholders" which actually drives me insane. Maybe you should read the definition first. We are the development team of diaspora and this is our project. The diaspora users are the stakeholders and not some third party projects developers who are not involved in our development. This is not the W3C and this is not the W3C Social WG. If you want to create a protocol that can be used with all systems, you have to take everyones needs into account and that cannot and will never be done by a single project. You are part of the Social WG and you should know that.

I am a bit angry right now. What you are just trying to do is basically blocking our work on a more stable federation inside the diaspora network and bringing our stuff back to already defined and widely used standards. We have discussed about the importance of the federation in the past and you were on the same side and now you're basically saying we should not be able to make changes without asking everyone on this planet.

So, just to summarize: We (as in the people who have actively participated in the development of the federation modules in the past) will continue to move the stuff into an own gem while keeping compatibility with old diaspora pods. Some time in the future (and this is going to take quite some time), we will gradually deprecate old federation implementations, thus forcing administrators of old diaspora pods to update. This will be done after a long pre-announcement, but again, we are not even close to think about that.

I would like to add something personal. Right now, we are moving from a pretty crappy implementation to a well-defined and well-developed ecosystem. If we decide to move to just another randomized implementation because this is what the community decides, fine, but then the project has to progress without my support, I fear.

“YOLO code”

ROFL.

I agree. Maybe it could be time to write down a formal protocol not tied to diaspora and invite other participants at the table to work with us.

As you should be aware of if you want to take place in this discussion, diaspora never had or has a fixed, well defined protocol. Because of that, stuff is sometimes broken and writing actual documentation is impossible.

I'm sorry to say, but that's the worst excuse I've ever read for not writing a documentation. The actual lack of documentation is probably the reason why there has been so few people working on the project for 3 years. This really is a problem and this really is the main recurrent criticism I've heard since I've been contributing. And I find it pretty accurate.

The fact that de federation protocol has flaws and is not guarantied to work is not to my eyes a good reason for not documenting, not communicated and not trying to be more open towards other projects. Otherwise, I don't think diaspora can be called an open-source project anymore.

I'd be less strong on the Do not merge in changes before listening to all the stakeholders and noting their concerns with implementation and timetable part than Jason. But stil, that doesn't mean we can't communicate. We are not autists, are we!?

@jasonrobinson Before I'm able to vote you will have to define "changes" and "diaspora protocol" first.

Since there is no formal definition of the protocol right now, for me the protocol is defined by our own implementation. Do you have a different definition? Would you define the protocol as the "union" of all protocols used by social network implementations that claim to be compatible with diaspora? (friendica, redmatrix, hubzilla, pyaspora, ...) Would you define the protocol as the "union" of all protocols that are accepted (but not automatically spoken) by the current federation implementation of diaspora?

To define "changes" let's take the example you mentioned. Benjamin started to fix some bugs in the federation code (like #2440) but kept the federation code compatible with the old one. Pods using some old d* code are still able to communicate with pods using the latest code. He extended the protocol to allow communication between pods which is closer to the standards we use. Is the extension of the protocol already a protocol change or do we change to protocol whenever some old implementation becomes incompatible?

Benjamin also added some validations that were fine for our own implementation but not for others. Depending on the definition of "diaspora protocol" he made the protocol incompatible with other protocols or with other "flavors" of the d* protocol which were different from the one we use.

@dennisschubert

Also, at this state, I would like to point out that YOU, Jason, even liked our plans which we are realizing at the moment three years ago.

Sure, exciting stuff, still support it. All I'm asking is to respect other adopters and talk with them - not do one sided work with a "fix yours if you want to" attitude. This is the attitude which still you can see as negative towards the whole diaspora project that was caused by the original founders not promising to respect any other software stacks.

You even commented on some of Benjamins pull request and never raised issues and now you act like you never had the chance to participate. I am a bit confused now.

I don't implement the protocol. As stakeholders here I am talking of the other software stacks.

Maybe you should read the definition first.

‘an individual, group, or organization, who may affect, be affected by, or perceive itself to be affected by a decision, activity, or outcome of a project

Sounds like Friendica and Redmatrix.

This is not the W3C and this is not the W3C Social WG. If you want to create a protocol that can be used with all systems, you have to take everyones needs into account and that cannot and will never be done by a single project. You are part of the Social WG and you should know that.

Yes, being a part of the SocialWG is exactly why I feel that diaspora is the real hope of decentralizing the social web. Tbh, I'm not too sure what the result of the SocialWG will be. Really, the diaspora protocol, or "not protocol" as you want it, is already unifying three different projects and with improvements hey maybe it could be a real unifying thing. I heard even the XMPP networks are planning on federating over some gateway using the diaspora protocol (no, sorry, not a protocol, a "thing"). I know some people from GNU Social have had thoughts at least on this. I know a wip python library that a certain image hosting site was planning to use to federate profiles on their platform.

None of this can happen if we don't collaborate. You guys, the three who are working on improving the protocol, are in the position to do awesome stuff - but instead today I hear that the plan is to just move thinking only of the diaspora Rails based software stack with no care about any other implementers.

You find this justified so you can iterate faster and talk with less people. I find this the death of the dream of growing the decentralized social web around diaspora. It's the death of The Federation.

What you are just trying to do is basically blocking our work on a more stable federation inside the diaspora network and bringing our stuff back to already defined and widely used standards.

No, I'm saying respect is needed and dialogue.

Right now, we are moving from a pretty crappy implementation to a well-defined and well-developed ecosystem.

Great, why not do it right then?

@steffenvanbergerem

No, the protocol (or non-protocol thing) is not so well set in stone that one can look at a change in code and say for sure "this will or wont break something".

But you're not seeing my point. I learnt today that the 3 people working on the improvements will not consult, hear or care about changes that affect software outside of the Rails based Diaspora software stack. There will be no thought on whether Friendica or Redmatrix federation breaks because "hey we didn't declare support for it".

I never said in the OP and sorry if it sounded like that that the changes recently done by Benjamin would have even been possible to see whether they break the integration. The fellows from Redmatrix didn't come blaming the diaspora developers and quickly Benjamin also submitted a patch to revert some of the changes. Even not doing a hotfix to restore breakage is ok I guess because only a part of the pods are affected.

But the idea that in the future, changes will be made to redesign parts of the diaspora non-protocol thingy with only diaspora in mind and not even slightly giving a damn thought whether someone else might be interested is just disgusting.

Are you @jhass and @dennisschubert really going to go so far as to say yes to the above statement, that you have absolutely no desire for collaboration with other networks, that your only interest is diaspora and only diaspora?

If that is so, I think this project is not for me. I don't spend as much time on diaspora as at least Jonne does, but I spend enough that I want to make sure it is worth it. I care a lot for diaspora, but I care more about working towards collaborating and networking together. That is the real future not some single software stack. Alone diaspora is not worth the effort.

I do not want to spend time helping other projects adopt our broken "protocol". Instead I want to move to a protocol that's worth of being adopted by other implementations, asap. And then help other projects adopt it. The current one simply isn't, and I'm sure anyone doing it is simply looking to reach our userbase, not because it's in any way neatly designed, standards compliant, reliable, flexible or full featured. So yes, in the current state I do not want to collaborate with other networks in the aspect of helping them to support our current implementation, I think it'd be toxic for them and wasting both our time.

If you remember NodeInfo I very well wanted to collaborate on things that are actually properly designed. But all I got was interest after there was an implementation to copy, that's not really collaboration and certainly not the kind of collaboration your proposal demands from us before moving forward.