Loomio
January 16th, 2017 17:27

[it-squad] - about emails @pirateparty.be (mailboxes, forwards, mailing-lists)

Thierry Fenasse
Thierry Fenasse Public Seen by 671

The situation

Our mails are hosted at Infomaniak and it costs now 120€ / year. They invites us to upgrade because their old tools are evolving. It is not mandatory but for instance, the antispam filter is not working nicely.

Until now they provided a single fee for multiple services and they are evolving to a «on demand price/service».
(I suppose that one day or another we will have to upgrade or change)

We use them mainly for our emails @pirateparty.be.

The website, the wiki and some other old tools are hosted by Hetzner.

The Upgrade Proposal

Is to agree to the upgrade and optimize our usage but it will cost us probably more then 120€ / year.

The Do It Yourself Proposal

Is to implement our own mail server+webmail+imap+antispam+antivirus+backup and ask our current @piratepaty.be mail users to migrate their mailboxes or keep a copy for and by themslef. This may ask more human resources, time and discussions and probably not less money neither.

  • For an it-squad it is a nice chalenge and probably a long discussion about the tools. :construction_worker_tone1:

The idea is to evolve to one of those two Proposals and find a concensus about it if possible.

Patrick Installé

Patrick Installé January 16th, 2017 19:21

Is those option not over-engineering ?
Do https://www.ovh.com/fr/produits/mxplan.xml (100 mx) not enough ?

Thierry Fenasse

Thierry Fenasse January 16th, 2017 20:55

And its a one time fee ??? (see the last line of the joinel table) :astonished:

If we change the provider, we still have to tell every users
* to backup their current mailbox,
* use new server settings (Incoming / Outgoing),
* use a new password (that we will have to send and ask them to change)

But for the soooo small fee and if we have POPs / IMAPs / SMTPs and an HTTPs webmail, why not *(except that it's Ovh :tongue: )?

I'll test on one of my domains in the next days.

Pascal Dk

Pascal Dk January 16th, 2017 21:34

If Web4All can be a solution, I'm ok to pay for one year (the "zen" offer).
If we can have more people on this fundrising, I'm defintely ok to participate in the "Premium" offer.

Web4All offers :
https://www.web4all.fr/hebergements.html

(the "Premium" offer at Web4All is 50% cheaper than the new rates on Infomaniak - just saying)

Thierry Fenasse

Thierry Fenasse January 17th, 2017 07:06

I like the idea of Web4All for the association profile.
Zimbra proposed as a webmail is something I like too. More if we can use sharing functionalities between users. Compared to OVH, we are sure to have a nice webmail, pops, imaps, and probably smartbidules synchronization thanks to the Zimbra proposal.
Compared to the current mail hosting, this would be an improvement.

Vincent

Vincent January 17th, 2017 20:32

Sounds good.

HgO

HgO January 17th, 2017 20:50

I'm good with the web4all solution as well. My personal websites are hosted by them, and I'm very happy with their technical support. They are also very transparent about the state of their network, and I can't complain about technical issues (e.g. servers down) since they are very rare.

However, if we are going towards this solution, we should consider using the servers as well ;) This could host simple services, such as wordpress websites or something.

Migration would be simple in my mind : we send an email to every user, explaining that we will change our host in x days, with all the technical details. That's all, I don't see what could go wrong :D

HgO

HgO January 24th, 2017 09:43

Avant de changer d'hébergeur de mails, je pense qu'il vaudrait mieux trier les mailing lists inutilisées.

Tout d'abord, je propose de supprimer l'adresse coreteam@ppbe, puisque celle-ci n'existe plus. Il faudra donc changer les alias qui pointaient vers coreteam@ppbe, soit en les supprimant à leur tour, soit en les faisant pointer vers une autre adresse. Je vois sur le wiki que les alias suivants devront être changés: discussion, get-me-in, international, legal, membership, merchandising, press, mailchimp, twitter, wikileaks et welcome.

  • press devrait devenir une mailing list à part entière, et non un alias. Néanmoins, comme la press squad est pour le moment vide, je propose de faire un alias vers contact@ppbe (en effet, pour le moment c'est via cette adresse que les journalistes nous contactent).
  • international devrait disparaître, mais sinon elle pourrait pointer vers contact@ppbe
  • mailchimp, twitter, facebook (qui pointe vers twitter pour l'instant), trello et vimeo devraient tous pointer vers une adresse commune. Cela pourrait être service@ppbe ou web-service@ppbe, je m'en fiche un peu.
  • merchandising (et merchandise) peuvent selon moi disparaître, sauf si vous y voyez un intérêt à garder cette adresse ?
  • discussion, get-me-in et legal devront être supprimées
  • wikileaks devrait être supprimé, sauf si vous voyez à quoi cela servait ?
  • welcome et membership peuvent également être supprimées (je ne vois pas trop à quoi sert membership ? et welcome fait double emploi avec contact)

Notons également qu'il faudra penser à changer l'adresse mail utilisée pour infomaniak, hetzner, etc. (laquelle est coreteam@ppbe) pour une adresse plus adéquate (du style service ou web-service ?)

Lorsqu'on aura nettoyé tout ce bazar, je propose d'envoyer un mail automatique à toutes les mailing lists. Ce mail demandera à la liste une réponse pour savoir si elle est toujours utilisée. La réponse pourra très bien être un mail vide, peu importe. Les utilisateurs de la liste auront 15 jours pour donner leur réponse. Après ce délai, un second mail servant de rappel sera envoyé. Si après 15 autres jours, nous n'obtenons pas de réponse, la liste sera supprimée.

Voilà, qu'en pensez-vous ? :)

Let me know if you need a translation in English ;)

Renaud Van Eeckhout

Renaud Van Eeckhout January 24th, 2017 19:24

Je prends le train en marche : quand tu parles de vérifier si les mailing-lists sont actives, ça inclut celles des crews ? Perso je dirais bien de supprimer la nôtre vu qu'on ne l'utilise plus depuis 1.000 ans mais faut que je vois avec ma crew alors. Ou alors elle est pas concernée ?

HgO

HgO January 24th, 2017 21:30

Oui, y compris celles des crews (surtout celles des crews en fait, car je pense qu'il y en a très peu d'actives). Quand je parle des mailing lists, ce sont toutes les adresses mails @ppbe qui ne sont ni des alias, ni des adresses personnelles.

Sandrine Debatty

Sandrine Debatty January 25th, 2017 06:21

Je suis ok avec ta proposition HgO.

Pascal Dk

Pascal Dk February 4th, 2017 17:31

La question du changement d'hébergeur est exclusivement liée à une question de thunes.

En gros : est-ce qu'on veut payer 3x plus cher pour la même chose ou pas ?
Sachant que chez Web4All on retrouve les mêmes prix (à peu près) que ce qu'on paye déjà pour les mêmes services (et que Infomaniak gonfle solidement ses tarifs)

La question des ML sera la même quelque soit l'hébergeur et n'a donc pas vraiment sa place ici...

#my2cents

HgO

HgO March 1st, 2017 13:18

Pour info : https://blog.web4all.fr/web4all-cest-termine/

Summary: Web4All (the ASBL/VZW) will be abandonned. Some of the founders want to start a new project, which would be a startup named yulPa. Nothing change for the members, at the moment.

What's important for us: It won't be an ASBL/VZW anymore, and their structure and finances might be less stable in the short run.

Pascal Dk

Pascal Dk March 2nd, 2017 05:50

Still the same team behind the project thou
(not only the founders, the whole team).

Thierry Fenasse

Thierry Fenasse March 24th, 2017 09:24

English

Knowing that our IT costs where reduced by 50% (explained here),

Recieving this morning from Infomaniak, the information about the auto-upgrade on the 4th of April

Meaning that the new admin panel will probably be more confortable to use and manage, but the new costs may raise from 120€/year to ±350€ regarding the number of mailboxes we keep.

Having a small active it-squad.

The upgrade will be done automatically.

Français

Sachant que les frais informatiques viennent d'être réduits de moitier (expliqué ici),

Ayant reçu de Infomaniak l'information que la Console Évoluera le 4 avril

Ce qui veut dire que la migration vers leur nouveaux outils (plus confortable?), et leurs nouveaux tarifs (probablement passant de 120€ / ans à ±350€ en fonction du nombre de boîtes mails qu'on gardera).

N'ayant qu'une petite it-squad active.

La mise à jour se fera automatiquement.

Nederlands

(bijgestaan vertaalmachine)

Wetende dat IT-uitgaven worden 50% verminderd (hier uitgelegd),

Een Infomaniak bericht vanochtend aangekomen uit te leggen dat de console zal worden bijgewerkt, op 4 april

Dit betekent dat de nieuwe admin panel waarschijnlijk meer comfortabel te gebruiken en te beheren zal zijn, maar de nieuwe kosten kunnen verhogen van 120 € / jaar tot ±350 € met betrekking tot het aantal mailboxen we houden.

We hebben een kleine actieve it-squad.

De upgrade zal automatisch worden gedaan.

Thierry Fenasse

Thierry Fenasse April 10th, 2017 13:14

If we (it-squad / Belgian Pirates) want, have time, ressources (more human than anyting else :sweat_smile: ) ... we can also set-up our own mail server.

Knowing that :

Setting up our own mail server is doable, but from personal experience is a real PITA:
* Microsoft mail services blacklist whole netblocks (Hetzner, OVH)
* Google is very picky and inconsistent (sometimes silently end up in the spam folder)

In addition to just installing the SMTP server, the following MUST be met to hope reaching the mailbox:

  • Proper forward DNS records (A, AAAA) and reverse (PTR)
  • Proper SPF records in the DNS
  • Working DKIM setup (both on mail server and in DNS records)
  • Bonus: DMARC records for reporting
  • Bonus: TLS on mail server
  • Proper MX record (mail reception only)

And we can use this bundle of FOSS tools : https://hub.docker.com/r/tvial/docker-mailserver/

DR

Damiens ROBERT April 11th, 2017 20:22

Just a quick note that is important, https://hub.docker.com/r/tvial/docker-mailserver/ seems very complete, well made and very easy to install but the tools installed are command line tools. Meaning that to manage them we'll have to use SSH and the linux terminal.

I am sure there are web applications that allow you to configure the services installed by this project but integrating the web applications with this project will require work and testing.

Finally, we will have to configure our own webmail client (probably roundcube) to use the services installed by the project.

On other words, it seems very professional but it will require quite some work. I think it would be a good solution to avoid to depend on providers who can stop or change the condition of their services and really controlling our infrastructure.

Note that as it is a open source project and that it seems very popular with over 500K downloads in the docker hub (1). It means we can probably have some traction to have other open source project to integrate nicely with it. At least, we can try to open issues on the web applications bugtrackers that we will need to use to administer the mail not using the command line in order to ask them to support the docker mail server. They might answer positively and I would not bury the idea before trying.

(1) https://hub.docker.com/search/?isAutomated=0&isOfficial=0&page=1&pullCount=0&q=tvial%2Fdocker-mailserver&starCount=0

ZeFredz

ZeFredz April 16th, 2017 11:37

This docker image seems to have everything described in the 4 articles posted by @tierce

ZeFredz

ZeFredz April 16th, 2017 15:15

We (@tierce, @damiensrobert, @iljabaert and I) decided to install a test mail server under the domain parley.be (a domain we already have) using docker and the docker-mailserver fullstack.

At the end of the day, we have a partly working mail server (imap, smtp, tls) :)

What's left to do for next time : understand why ssl does not work properly (certificate error) and why clamav is failing to scan emails.

ZeFredz

ZeFredz April 16th, 2017 15:20

we also have some issues with gmail and yahoo which are putting the emails from parley.be in the spam

Thierry Fenasse

Thierry Fenasse April 16th, 2017 16:02

Regarding those the must have we forget to setup a PTR record ... now it's done.

I used mail.parley.be ... but I do not know if parley.be is better or not.

HgO

HgO April 26th, 2017 13:22

I know we are going towards our own mail server, but for your information yulPa (formerly Web4all) propose now a stand-alone mail service : https://yulpa.io/messagerie-zimbra-ne-mutu.html :)

GT

Gilberto Torres May 19th, 2017 16:30

@5euro per account... WE can do it better ;)

HgO

HgO May 19th, 2017 17:05

That's not what I understand ' It's 5€ HT for a mail service with 25Go storage space.

Anyways, I guess we will discuss all that on Sunday (see https://wiki.pirateparty.be/ITSquad/Meeting/21_05_2017). We'll be glad to see you there ;)

Thierry Fenasse

Thierry Fenasse May 21st, 2017 17:28

Because the IPs of Hetzner are blacklisted «by default» by Microsoft (as a anti-spam solution), once we have a new mail server hosted at Hetzner, and if we want to be able to send emails to live.be, outlook.com, hotmail.com and maybe everyone hosted on office365 ...

We also have to consider to have a Microsoft Live Account

  • Do we create one for the IT-Squad?

Subscribing to Microsoft JMRP and SNDS to «register the IP address of our future mail server from Hetzner as a workaround for the Hetzner blocking by Microsoft.

Thanks to @gilbertotorres for the share.

In addition of what we wrote before

Setting up our own mail server is doable, but a real PITA (Pain In The Ass):

  • Microsoft mail services blacklist whole netblocks (Hetzner, OVH)
  • Google is very picky and inconsistent (sometimes silently end up in the spam folder)

In addition to just installing the SMTP server, the following MUST be met to hope reaching the mailbox:

  • Proper forward DNS records (A, AAAA) and reverse (PTR)
  • Proper SPF records in the DNS
  • Working DKIM setup (both on mail server and in DNS records)
  • Bonus: DMARC records for reporting
  • Bonus: TLS on mail server
  • Proper MX record (mail reception only)

As a reminder and for our Mastodon

We have had to register a Mailgun account to have a «clean smtp» to send mail (notifications, password recovery, ...).

The account was registered using those required informations:

  • An email: we used the personnal mail of @damiensrobert
  • A visa card: we used mine.
  • A billing address: Mermaid vsw/asbl
GT

Gilberto Torres May 21st, 2017 18:12

Hey everyone reading,

The "pita" mentioned on the previous post is a mecessary evil on every mail server running...

Best regards,Gilberto Torres

Thierry Fenasse

Thierry Fenasse June 5th, 2017 08:54

During our latest IT meeting we have installed a YuNoHost server under our parley.be domain name.

The main goal (correct me if I'm wrong) is to replace the Infomaniak provider as an SMTP server to

  • send our Newsletters (like the last one nl, fr, en),
  • send notifications / lost password from our Wiki,
  • send notifications / lost password from our WordPress,
  • send notifications / lost password from our Mastodon,

We are happy with a 10/10 result :fireworks: regarding some of the PITA/basic requierments to avoid being considered as spammers when we send mails.

Another step (requiring some contracts to be signed) is to add our mail server IP address to the Microsoft JMRP and SNDS. I do not know if it will be necessary to do the same for Google, Yahoo, Apple or any other of those corporation networks. :robot:

A secondary goal may be to replace Infomaniak for our users having a pirateparty.be mailbox.

Patrick Installé

Patrick Installé June 5th, 2017 09:33

Great job. I love your feedbacks. That's the pirates way of life.

HgO

HgO July 25th, 2017 09:02

So, I made this diagram to help me have a better view on what is the current situation in this mail mess ! :) I also made another diagram to show you what I propose as refactoring. You can see on the bottom-left things I wamt to remove ;)

I'm not sure about:
* treasure: should it be connected to finance, or a standalone ?
* merchandising and shop: do we keep them ? purpose ? connected to finance ?
* legal: purpose ?!
* having a mail per web service: overkill ?
* postmaster: what's the purpose ? difference with webmaster ?
* ahoy: should it stay empty or not ?
* get-me-in: does anybody knows what it was used for ????

Questions, remarks, objections, etc. ? :D Thanks !

Renaud Van Eeckhout

Renaud Van Eeckhout July 25th, 2017 10:19

I think the get-me-in was the form that people could fill on the Drupal website.

Shop I believe was for the webstore we had.

The "web" is a new mail you propose?

I'd also delete legal (if no particular reason to keep it) and coreteam.

HgO

HgO July 25th, 2017 10:42

Thanks ! I didn't even know we had a shop :o No, "web" exists yet, but as you can see it forwards to only one guy, and I'm not sure he is in the party anymore...

I'm ok for deleting "legal", but I would keep coreteam for now as it is used on our DNS details :(

Thierry Fenasse

Thierry Fenasse July 25th, 2017 12:23

That's a clear diagram! Thank you soo much for having done this!

PB

Paul Bossu August 3rd, 2017 07:39

shop and merchandising was indeed for all the promotion kit.. originally on the website (drupal jurgen then wordpress marouan, then.drupal Tom, ... ) all the stock was in à box at the office but it seem we lost it when the German.pirates closed the office

Legal was an.idea of marouan for all the legal stuff (asbl but also laws proposal)

HgO

HgO August 3rd, 2017 08:41

Thanks @paulbossu ! :) Yes, I was there when we were supposed to get back the merchandising, but the office was clean and empty... :( I think @pascaldk tried to contact them, but I guess it didn't work ?

I've removed mail accounts of those who gave me their agreement so far, and I've updated the current schema, you can see it on the wiki (because there is a versioning system there) : https://wiki.pirateparty.be/File:Emails_Structure.png :)

PB

Paul Bossu August 3rd, 2017 10:01

Je dois encore regarder pour paul@pirateparty

HgO

HgO August 28th, 2017 13:01

Sooo a quick update :
* We switched to the new console, which means that everything is pretty but will be more expensive (as this was expected)
* We have till 17th January 2018 to reduce the number of pirate mail accounts and mailing lists (because our current contract ends at this date)
* On the 50 personal accounts, I received 13 "yes I want to keep my pirate address" and 19 "no, you can remove it". I don't know for the 18 remaining people...
* In the new console, if I want to renew the contract now (to see how much it would cost us), they say "450€ HT", while it should be 410€ TTC (at the moment). I suppose this is a bug, meaning that we will have to contact infomaniak...

Also, I will do the re-structuration of the mailing lists (see the schemas above) probably in September / October :)

@paulbossu For now, I'm keeping your mail address, so you have all the time you want to decide whether you want to keep it or not ;) As I told you, it's not a problem to keep a few "uncertain" pirate address. The problem was that we had dozens of old and unused pirate accounts waiting to be removed :p

HgO

HgO October 16th, 2017 10:29

I removed the pirate e-mails account of those who never answered my "warnings".

As the address gent@ppbe was left empty after the removals, I decided to delete this account... Let me know if this was a bad idea :sweat_smile:

In the end, we have 63 e-mails remaining. This should costs us 387€/year. Next step is to restructure the e-mails (see the schema above). Ideally, I would like to have an ITSquad meeting in order to work on this and to think further on this new structure. I think that most addresses can be "merged" through aliases.

We should also contact infomaniak in order to have some clarifications on their pricings...

HgO

HgO October 23rd, 2017 21:16

Sooo @tierce and I contacted infomaniak in order to clarify their pricings stuff. It appears that due to a kind of "legal bug", we will keep the old pricing, that is 120€/year regardless of the number of mails :confetti_ball: :tada:

For the details, they cannot force us to switch to their new pricing. So, as long as we keep renewing our contract each year, we will keep benefiting of this "special" pricing (cc @patrickinstalle ).

This also means that we have a web hosting that we could use (we don't use it yet) ;)

Finally, I think that the mailing lists still need a better structure, and I hope that we will be able to work on that during the incoming ITSquad meeting (see this poll for deciding on the date).

HgO

HgO November 4th, 2017 17:08

Ok, I've made several changes in the e-mail structure. Here isthe result :

https://wiki.pirateparty.be/File:Emails_Structure.png

The most important thing is that all web services are now aliases of web@ppbe. Thus, this mailbox can be used for password recovery. I also changed passwords for most of those services (with the help of @pascaldk) and stored them in a place where ITSquad can find the credentials easily (on our encrypted tiddlywiki, to be precise).

I removed a lot of mailing lists, too (legal, bin, listener, agenda, get-me-in, wikileaks, web4all, lqfb, opinion, ixelles, etc.) !

I also removed the redirections on newsletter@ppbe, and now people who send a mail to this address will receive an automatic message, telling them to contact contact@ppbe instead :)

Now, I'm unsure about the following mailing list :
* info@ppbe and international@ppbe : They seem to collect tons of spam, and I tempted to delete them. I think that they are useless, since we have already contact@ppbe.
* redactie@ppbe Not sure what's the purpose of this one, and it collects also a lot of spam... I'd remove this one.
* shop@ppbe and merchandising@ppbe I'd like to remove these mail accounts, but there seems to have some information about previous orders, Is this information important ? I'm afraid to lose a bit of our memory here, but maybe I'm too careful.
* coreteam@ppbe Aha the funny one :) So, we should remove it, but I think our DNS, bills, and other technical stuff are linked to this mail address...

Valerie D.

Valerie D. November 5th, 2017 21:05

@hgo redactie was the writing squad, for articles on our website, knack and co.

HgO

HgO November 7th, 2017 13:46

Then I guess I can delete this mailing list, as we use loomio at the moment ? If we need a mail address for this purpose, we will just have to create it again.