Loomio

Expiring Old Accounts

MS maxwell salzberg Public Seen by 59

One of the reasons that Joindiaspora is slow/requires lots of resources to run (and possibly other old pods), is there are tons of old, dead accounts in the system. I don't have the exact numbers in front of me, even if we expired accounts that have not signed in in the last three years, that would drastically reduce the database size, which means messages would federate much faster, page loads be more reliable, and would be more cost effective to run. It would drastically improve the entire experience for people using using the Pod, and any new users that try Diaspora out.

There are two tables specifically which really bog down performance (Person and PostVisibilities), and drive the cost and memory of running a large pod up. I'd like to optimize Diaspora for the community who actively use it, so I'd love this discussion to turn into a plan of action to improve this scenario.

There is a few things that we would be a good idea to expire.

1) Local User accounts that have not signed up in a period of time.
as such, we could also expire...
local person objects
all post visibilities from this account (things that they can see)
contacts (both sides)

2) Expired Pods
One thing we don't keep track of if a pod goes down
a) we don't want to send messages to this pod
b) if we have not have had contact for it for some period of time, we should expire all data related to said pod (Person, contacts, post visibilities for local users)

3) Empty accounts just following dhq

4) Any other ideas?

The goal here is that if we can actually expire a proper amount of the data, JD.com (and most likely other pods) can have small data sets, and require less resources to run, which makes them more sustainable for the future. I've been paying for JD.com out of my own pocket, but its starting to become a burden, so I wanted to make sure we found a solution that people found acceptible (and share that process with others).

I'd love all of your thoughts.

R

Ravenbird Sun 3 Aug 2014 8:36PM

Don't like deleting old accounts while there is no way to migrate accounts and date from one pod to another. I get a new account some weeks ago, but my old account have more then 3000 posts and it wouldn't be nice to loose all of them.

JH

Jonne Haß Sun 3 Aug 2014 8:42PM

One interesting approach could be implementing the move account by building an exportable archive that can fully restore an account on any pod. If we have that, one could build a mechanism that automatically builds such an archive for old accounts, uploads that to some location and then deletes the accounts data. When the user then signs in again he could download that archive or it's even automatically restored at that pod.

No matter what approach we settle on here, the big question is who is going to implement it.

F

Flaburgan Sun 3 Aug 2014 8:50PM

@maxwellsalzberg you cannot imagine how happy I am to see you opening this discussion. I opened #4183 about that a while ago but unfortunately we saw no progress on this.

IMO we can safely delete :
* Users who never signed up (they were invited but never clicked on the link)
* Pods which didn't respond for a long time (one year?)

We can also delete users who didn't sign in for more than (period to define), but to send them an email first would be done.

My opinion about this is to propose these actions in the admin panel of the pod:
*Clean pods which didn't respond for more than [period selectable by the podmin, min 6 months]
*Clean users who didn't sign in for more than [period selectable by the podmin, min 6 months]
*Mail users who didn't sign in for more than [period selectable by the podmin]

The mail can be to alert and ask to connect, something like "hey, diaspora* was really improved since the last time. wanna have a look again? If you're not going to use our service again, please delete your account to improve our performance".
Or a mail to warn that the accounts would be deleted "hey, you didn't sign in for more than a year, your account will be deleted if you don't log in before 15 days"

A

Adrenalin Sun 3 Aug 2014 9:17PM

As Ravenbird said…

desperately waiting since more than 1 year for a tool to move the postings from my old account into my new one (thousands of postings). It would be sad to loose them. Wish I could help working on such tool but I have no clue about programming :(

F

Flaburgan Sun 3 Aug 2014 9:34PM

Well, maybe we could add a criteria: "accounts who never posted anything".

MS

maxwell salzberg Sun 3 Aug 2014 10:31PM

certainly we would not delete any info from people who wanted their data... all they would have to do is to just log in to JD.com (or whatever pod) ONCE in the last 3 years. (or whatever timeframe we agree on)

This way we could still dogear accounts to be migrated.

A

aj Mon 4 Aug 2014 2:02AM

in the case of a private pod where there aren't a lot of users, it is often the size of the participations table that bloats the DB

it would be handy to have a function to clear participations more than a year old, they are too far down in the activity feed to ever view them anyway

T

Theatre-X Mon 4 Aug 2014 5:16AM

One of the things I've noticed is that some of the pods (e.g. Diasporg) gives an authentication error even though the login creds are correct. That happened with my first account and has happened approx. 3 times to a friend of mine who eventually gave up on Diaspora because of it.

I would contact some of the account owners somehow. In my case, delete my old account on Diasporg and my account on joindiaspora. I really don't give a shit.

DU

Rich Mon 4 Aug 2014 10:13AM

Confirmed what @aj says re smaller DBs blowing up on the "Likes" etc, it's completely out of all proportion. Did I see a cleanup script for this somewhere?

JR

Jason Robinson Mon 4 Aug 2014 12:48PM

@maxwellsalzberg thank you for this initiative - absolute it is very important to start optimizing the network structure as it's starting to get old enough to suffer from problems of no optimization done.

Unlike commercial providers, the diaspora* network is afaik fully powered by private people from their own pockets. Thus we should absolutely offer tools for podmins to clean data and make sure that they can run their pod on a small as possible hosting plan. This will not only ensure pods run for longer, but also make it more easier for new podmins to start their journey.

My suggestion for user deletions;

  • Make two rake jobs. The first rake job would send a warning email to accounts that would be closed.
  • The date the warning was sent would be stored to the Users table as a timestamp.
  • The second rake job would then delete these account where "warning sent timestamp + configurable period < current timestamp".

Why send a warning email? Two reasons. Firstly, the obvious, to warn the user. In some cases like indicated by @saschamorr - the user could be interested to keep the account.
But the second reason is more important IMHO. By sending the warning email, we are also contacting the user and saying "Hey! We're still here". From the management work I do with the diaspora* project social media accounts, it seems likely that 90% of those 1 million created accounts probably think diaspora* has died.

Whatever way this is done, it would be great to lessen the burden of running a pod. Optimizing joindiaspora.com is a perfect task to achieve that for other pods too ;)

Load More