What is the best open source built-in encryption email server solution to be installed in a vps

The best solution for your problem is to use Google first at all times 😉

google "open source email server +github"

I agree but it won't tell you based on experiences of others, i found cowmail dockerized through youtube and that was never found on a google query, cowmail seems to be promising but how about built-in encryption so even if the vps provider tried to spy on your emails they should not be able to get through your emails

@dIREsTRAITS yes use that, some devRanters recommend cowmail too.
And maybe google "what alternatives to cowmail are there?"

@heyheni hmm why not :D

By the way, it is 2020 and every mailserver* supports encryption (if you mean transport encryption, as E2E is done on clients). ;)

*Server for sending or receiving mails? These are (usually) different programs (though e.g. cowmails puts those two in a container)! Though you of course need both if you want to send and receive mails.

@sbiewald what about if the vps provider tried to spy on you and read your emails, does cowmail encrypt everything? i couldn't find something about it through their documentation

@dIREsTRAITS
You can't do secure computing on hardware what isn't yours. Period.

You can get close, but if your provider will do a RAM snapshot every 100ms (or even manipulates memory), he can likely read your mails.
The problem is, what do you want to protect against? Define your threats beforehand, and then you can look for specific solutions.

If you assume your VPS provider does ocassional snapshots of your virtual *harddrive*, use full disk encryption (or store your mails on an encrypted filesystem).
If you assume your VPS provider does occasional memory dumps, you can setup gpgmail (together with postfix) to encrypt all mails, so you can only read them on your computer; it is unlikely your VPS provider will do a snapshot while you encrypt your mails.

But if you really assume your VPS provider is evil and uses its power against you, noone can help you (because you cannot theoretically even trust your installer to really encrypt the disk).

@sbiewald Thanks for your reply, I agree with you when you mentioned the fact that if you don't own the server physically then you are out of full privacy, when you asked about what I'm trying to protect i would say everything even the least silly emails between me and my friends, imagine if the drive was replaced by the admin, even if it is formatted, data can be restored with proper ways such data recovery solutions so you people that are looking for an opportunity to find something useful from your data can use them even against you.
Also full disk encryption is not something i would consider secure, if you try to access the server via vnc that would be unencrypted and if as you said there was server snapshot it would be useless as well specially with rogue admins, the only solution i would think of is to get a static ip address vpn linked with your router behind a firewall and host your server where you can see it :) other than that you would play dangerous games