Backup and restore

Pods should schedule backups of user data once a week to the backup pods.

I think it is not frequent enough. Some people post frequently, and one week would be a huge loss for them. How about making that adjustable according to person's activity level? Or simply backup every day if there were changes. If there weren't, don't backup at all.

I also beleive we must support some kind of limitation for pods backup capacity. For example a pod wants to store backups, but no more than for 100 people. Then it shows that it doesn't receive backups anymore, but previous still work.

So we have User and Person models. On migration we must create a new user but do we preserve the same Person model which was known for that person? I suppose we should.

Yes, we should keep the same Person as it is the same identity, just moved local instead of remote.

I don’t think posts and comments can be ignored. It is possible that the new pod doesn’t contain some posts (especially private). I believe it must not be dropped and we should fetch them on restore also.

Well, it's not feasible to import posts or comments imho. What purpose would that even solve? Your contacts would not see any of the uploaded private posts unless you also push them out to them - which would be bad because they already have them from before.

The main point is to save the identity, not every crumb of data related to it.

Where do we store signature? Don’t we have to have a separate field in the schema for that?

This would be the same signing method we use for delivering content - ie the receiver only accepts it after verifying the signature in the payload against the public key of the person. Whether this should live in the diaspora federation gem where the code is readily available is up to question. I'd say no, but of course it would be easier to implement that way. I don't consider this part part of the protocol as such, but then that is not in any way under specification anyway.

I think it is not frequent enough. Some people post frequently, and one week would be a huge loss for them. How about making that adjustable according to person’s activity level? Or simply backup every day if there were changes. If there weren’t, don’t backup at all.

I believe this can be left as detail when the thing actually works. If we leave out posts and comments, which I think is the only real way to do it, then there is absolutely no sense in sending backups daily.

I also beleive we must support some kind of limitation for pods backup capacity. For example a pod wants to store backups, but no more than for 100 people. Then it shows that it doesn’t receive backups anymore, but previous still work.

Again, if posts and comments are not part of the archive, it will always be very small. Additionally, backup pods would be randomized, guaranteeing some balancing of load between pods. Of course we can introduce a setting like this, but personally it doesn't sound super useful, considering how much data pods store normally compared to how much this would add.

One interesting case is what happens if the pod has turned off sign-ups in the event that someone wants to restore? I think we should bypass the sign-up not being on and let the user restore. This would speak towards the setting you mention, so that pods don't gather too many possible identities that could activate suddenly.

Great comments, thanks!

Well, private posts may contain some valueable information for the user herself. I don't have extra copies of texts I posted, and I don't like to lose these texts. Moreover it would be definitely nice to preserve a possibility to continue conversation on some private post that was going on before the move. I don't see why we should push them again. A guid of the post is preserved, so all new comments will be federated well to the right place.

Not to say about public posts which might be wanted to be shared with some new contacts.

So I think posts, the content is extremely important part of the network. That's why people are in it. And the restore feature without posts restore would look unfinished.

Maybe if we push posts to the restore pod as if there were some subscriber on it could make restore easier, since we won't need frequent backup then and any special posts restore feature. At least we could do that for public posts, since for private posts that would imply trust for restore server even before a password was entered. That is not really acceptable.

https://wiki.diasporafoundation.org/Account_Backup_And_Restore#Backup_delivery_message

“backup” is signed and encrypted using the user private key. Once opened, it should contain the following schema...

It's not very clear the structure of the backup field before we "open" it. It must be some data array containing signature and encrypted data, right?

BTW, does any other software in the federation (friendica, redmatrix) do some sort of backup restore?

So here are my changes to the spec according to what we've discussed

https://wiki.diasporafoundation.org/index.php?title=Account_Backup_And_Restore&diff=4454&oldid=4418

@comradesenya unfortunately I'll unlikely have much time for going through these before friday - I'll reply then. Until then, it would be nice if the wiki could be left alone before accepting changes here in Loomio. I don't agree (still) to some things like including posts and comments in the backup archive and I'm not prepared to change my opinion unless some others join in and support that idea and also give valid technical ways to do it sanely which at the moment is missing.

Anyway, lets continue discussion on that, for a few days I'll have to pass on comments. I'll clean up the wiki page to reflect those items that have been mutually agreed upon then.

Once we have a clear understanding we can do a proposal.

TBH, I don't see any technical problems on posts restore. Everything seems to me pretty straightforward. We just add the posts to the database of the backup pod if they aren't there yet. Maybe I miss something?

On contrary, not restoring posts would lead to weird situations, like when after you've moved some of your contacts does comment on an old post of you. Comment gets federated to your new pod, but parent post is not there! Because we haven't merged it.