Changes to the diaspora protocol

If you remember NodeInfo I very well wanted to collaborate on things that are actually properly designed. But all I got was interest after there was an implementation to copy, that’s not really collaboration and certainly not the kind of collaboration your proposal demands from us before moving forward.

Yeah and once you actually got feedback all you did was complain that "it's too late you should have given suggestions earlier". Sorry, if you can't take feedback then you shouldn't work on stuff that you want others to implement. The same is I guess going to be with the future diaspora protocol then?

Well, my faith in diaspora is dying fast. If you really want to exchange your technical excellence quest to kill the user base it would be great if you did it with some other project than diaspora. If you really feel this then I fear we've fallen too far apart. Well, at least it means you don't consider my contribution to be worth much to even consider changing your opinion.

I got angry not because of the feedback, but because I asked long long before for feedback, got none, then spend time on an implementation and then got feedback that would require me to spend time to at least half rewrite my implementation whereas if I had gotten that feedback earlier I wouldn't have to spend that extra time. In short I got my time wasted, and yes that makes me angry. If you lack the empathy to understand that then we maybe really should just say goodbye.

This is not a new problem or one that is even limited to Diaspora. Communication protocols (of many kinds) get deployed and improved, and even radically altered all the time. The way we traditionally do this is to provide backward compatibility for a period of time, while providing some kind of indicator within the protocol that shows what revision or expectations this server supports. It's unrealistic in any decentralised platform to require all stakeholders to upgrade en masse. This argument would be valid in either a homogenous or heterogenous network. Even Diaspora has outlying nodes that would like to upgrade on their own schedule - not yours. "Be conservative in what you generate and liberal in what you accept" and with mechanisms for backward compatibility to the extent that this is possible is the way good software has always been written.

As an example I recall the transition from SMTP to MIME and to ESMTP. We've gone through several flavours of IMAP and LDAP and TLS and HTML and yadda yadda. All went smoothly. Some protocols have even gone through practically "overnight" change, like SMTP and XMPP. I'm just suggesting there are ways of managing these changes. Just quietly changing something in an incompatible way and without notifying other stakeholders has never been particularly well received.

As one of the mentioned stakeholders I will not participate in this vote.

I'm fine with Diaspora changing their protocol. There are a number of things that need work. You certainly don't need my approval. A warning and a basic idea of what things are changing would be nice so we know what to look for. Otherwise it's just rude to wake up one day with all your members screaming at you that everything is broken - because of some decision another project implemented and didn't even warn you about.

I had no idea five years ago that Diaspora would want to be a walled garden even though there have been plenty of indications and warning signs. You can have your walls if you want them. My sincere apologies for trying to break down the darn things.

I can't add much here. We have currently no "protocol". With my work at the federation-gem I try to change this and move to a more defined protocol with better documentation. And all this without breaking changes.

But I can't continue my work if I need to test against (and support) 4 crappy implementations instead of only one. And if I would, the result would be crap again, because I need to add even more hacks to support 3 more implementations (which we currently don't support). At the moment, the federation does not work really well and I want to change that as soon as possible (it was broken long enough). Or do you think it is more important to "try" to support friendica, redmatrix and hubzilla than a better federation for all diaspora users? And I think at the end also friendica, redmatrix and hubzilla will profit from this.

The federation-gem will be (and currently is) backward compatible, but only with the old diaspora-implementation. I can't guarantee that friendica, redmatrix or hubzilla doesn't break if they do something different than diaspora.

The only planned changes are not new: #2440 and #5963. And as I said, these will be added backward compatible. I will notify friendica, redmatrix and hubzilla before I remove the old implementation of these issues.

After we have a real definition, better implementation and documentation of the protocol, there will be a better basis to discuss changes or a new protocol. And I can't wait 8 months for every little change, only to get no feedback ... (see nodeinfo).

TL;DR: I am OK-ish with "Changes to the protocol must be pre-communicated with ALL stakeholders" (if communicate means "notify them, before we remove something old") and the currently planned changes are already communicated on GitHub (#2440 and #5963). So I don't understand why this proposal is needed right now? But I am not OK with this from the OP "Even if it is "YOLO code", coming from one developer, that doesn't mean it doesn't have to be supported, maintained and more importantly, not broken for ALL stakeholders.". I can only guarantee that old diaspora pods don't break (and not even that with 100%, because the old implementation is already broken). Otherwise, I can no longer work at the federation.

@dennisschubert The current implementation of the Diaspora protocol in Friendica was done in cooperation with Ilya Zhitomirskiy and @mikemacgirvin. I never even knew that there were made some changes on the Diaspora side to support Friendica. I thought the implementation in Friendica was a (perfect) copy of the part in Diaspora.

For me this "We are the development team of diaspora and this is our project. The diaspora users are the stakeholders and not some third party projects developers who are not involved in our development." sounds like:

"We will change stuff without any pre-warning and we won't even tell this other persons. Maybe they will be aware of the changes when the federations breaks. But even then we won't tell them what we changed because we really don't care about anyone else. Maybe we lose some thousand contacts but we don't mind as long as they aren't using our software."

All I'm asking for is a short notification like what is done in the issue 6405 where there was told what was changed and how to cope with that.

This really shouldn't be a blocker for a further development of the protocol. I really would like to do all necessary changes on the Friendica side - I only need to know what to change. Please give me some time and some piece of information.

@jhass sorry that I replied so late to your nodeinfo proposal. I was really busy in the last months (not only because of Friendica but also at work) so I wasn't able to do everything in time.

To be honest, it looks like neither @augier nor @mikemacgirvin and @michaelvogel have actually read the comments and the original proposal, which makes me angry as hell. I wrote that long comment for a reason. Thanks for voting and commenting without knowing what is actually going on.

I think we made pretty clear that we will be compatible with older diaspora pods (and thus with your old implementation, if it is 100% compatible). I also made clear that we of course will issue announcements if we ever will remove old compatibility layers, which - again - we do not have actual plans right now.

Jason basically proposed that we should not be able to do any changes without discussion it with third parties first. This is pretty clear by stating:

Do not merge in changes before listening to all the stakeholders and noting their concerns with implementation and timetable

And I am absolutely not fine with that. We are all very busy and I have no interest in working on a project where we need to wait 8 months just to make a little change. That's killing everyone.

Again, we will stay compatible with older diaspora versions but as much as I like that other networks have implemented the current state, if something is not able to keep up with the enhancements because the implementations differ, then too bad. We're simply not able to keep everything in mind.

@michaelvogel: I think you mix two different things here:

1. The planned changes which are still backward compatible (the guid and public-key: #2440) and which I planned to tell you, before the backward compatibility will be removed and after I wrote a proper documentation.

2. #6405: the stuff that broke here was not planned, so I couldn't tell you before it broke. I added some more validations, and didn't know that friendica and redmatrix wasn't valid against them. I removed some constraints now (because diaspora doesn't use these fields at the moment) to make friendica and redmatrix compatible again.

There's nothing wrong with informing other projects about changes in the federation, but when the devs have to wait for confirmation from other projects, it becomes a problem. It slows down development, and it may hinder improvements because other projects don't have the time to adapt to the changes.

Last but not least. Either you officially support third parties or you don't. Keeping crappy implementations just

This isn't even about compatibility within Diaspora but about compatibility to third parties; however, when Diaspora developers don't want to wait for approval from other projects they might start a fork, and then we'll have the problem of making different forks compatible to each other. Friendica support is nice, but it's not essential; and without a nicely working federation, it's not particularly useful.

All I’m asking for is a short notification like what is done in the issue 6405 where there was told what was changed and how to cope with that.

I'm pretty sure we could all agree that a notification is nice. In that case, a meaningful commit message and well commented code should be enough to do that. Also (I'm not sure if Diaspora already has that) building documentation from comments may be the perfect way to do this. That way, third parties would have docs they could consult.

Last but not least: Either you officially support third parties or you don't. Keeping crappy code because you don't want to break compatibility and don't want to deal with actually helping those third parties adapt to the new code doesn't help anyone. That's called being lazy, and it's a disservice to third parties and Diaspora itself.

If someone really wants to make sure that third parties can support Diaspora, he or she should help document changes made to Diaspora and help implement them in third party clients.

@dennisschubert I read all comments here. You wrote "we will be compatible with older diaspora pods (and thus with your old implementation, if it is 100% compatible)" but you also wrote "Third party networks have implemented old diaspora federation parts in the past, but they have done a pretty bad job at it. The stuff Friendica sent in never was even close to match the stuff diaspora sends, but it has worked with some hacks on both sides."

So it seems that Friendica isn't 100% compatible. Like I said: The only thing I'm asking for, is a notice when a part is removed that only concerned Friendica. I really would like to make Friendica 100% compatible. But since I don't know Ruby or the project structure, I will not be able to find all quirks on my own.

I do not want to veto on changes. I only want a fair chance to make a change before stuff breaks.