So I recently started discarding Proxmox for Arch on my experimental server.. new skill acquired 🙃

Kvm or xen?

@PerfectAsshole just LXC for now (essentially a more secure version of chroot, with similarly trivial overhead), but I guess that I could employ KVM in the future. For now I don't really have a need for it though, as I'm not running any non-Linux VM's in my network right now.

@Condor yeah i see that. I wouldn't mind setting up a xen cluster myself but i know short of testing different distros it wouldn't be used more than a kubernetes cluster would be

@PerfectAsshole hehe, I can see how either KVM or Xen wouldn't be used much. Personally I think that LXC is the future, given its extreme efficiency and the ability to run on any architecture that doesn't have hardware virtualization, such as ARM (which has proven superior to x86 in simplicity, as well as efficiency and consequently power consumption already). That and RAM constraints made me convert everything from KVM into LXC earlier. And I have to say, it's surprisingly good! With it, I can easily fit more than 20 containers into 8GB of RAM, and that was while accounting for the RAM consumption by ZFS ARC on the host. Now that I can use btrfs instead, that's eliminated as well. I wonder how many containers I'd be able to fit that way 😁

The whole point of lxc/lxd (lxc manager) and docker is to not rely on virtualization. Virtualization relies on both hardware and the dynamic translation at runtime.

Gentoo > Arch. Just my preference. Build and harden the distro. Then switch to a hardened os, like clips os. That way you are familiar with linux internals and won’t mess it up.

@tamusjroyce I've tried Gentoo before and I didn't really like it that much. Its idea of compiling everything from source for limited performance improvements strikes me as an unnecessary thing to do. That said, I do compile my own kernels and that was one of the reasons why I got quite a bad taste from Proxmox, because apparently custom kernels don't work very well with ZoL. So yeah...

I've also tried Kubernetes a while ago after hearing about it at FOSDEM 2016. At the time it had a bug introduced into it in the Raspberry Pi release, which disabled me from deploying it onto a Raspberry Pi cluster. Nonetheless, it was a very interesting thing indeed. I'll be sure to look into it as a Proxmox replacement, and if it supports btrfs, I very well might consider it as a replacement for Proxmox instead.

All three have a purpose. I can’t see any going away. Maybe a manager like kubernetes tying your intent with the container model for you.

Docker container: short living services that are spun up and shut down...think videos on Netflix.

Lxc container: Long running with more OS feel to it. For programs that require stability like databases or proxies.

Cloud: rented container. To load balance with. But don’t rely on it alone.

VM to run legacy OS or contain programs that may have security flaws. Harder to break out of jail if it is backed by hardware/dynamic compilation.

@Condor yep! Gentoo is a tutorial. Not a production or day-to-day tool. When using servers, a hardened distro is the way to go. For containers I mean.

Desktop, arch is great!

@tamusjroyce I wholeheartedly agree. My current infrastructure reflects this paradigm very closely actually. LXC for anything that supports it, with guest OS's being whatever my given server is easiest to deploy on (usually Ubuntu or Arch). My mailers are on VPS's, but with a backup of the vmail directories on local storage in case I lose access to the servers (I've had a very bad experience with that in the past, and at the time lost all my emails). In the future I'll also link that local storage to the mailers directly to allow either mailer to see all the emails without a need for periodic synchronization. And for everything that doesn't support LXC (mostly Windows I guess?) I guess that VM's would be the way to go.

That's the best sysadmin right there!

Da great @Condor

I always though special things needed done for the binaries that run on a hardened os. But mb Arch and Debian can be hardened. Never tried it.

Hardening a system makes it harder to deploy to. But way more secure. Worth checking out. You can run a hardened lxc guest., too.

I usually use a cloud service for email services. Their tools help greatly with IP reputation. Ever have a spam kill your IP rating for 50k+ users 200+ employees, it isn’t fun to migrate. Trust me. :)

technically a follow-up rant - if you're interested, have a look:

https://devrant.com/rants/1911672/

https://devrant.com/rants/1920353/

@git-gud thanks for linking them! Indeed, it's a follow-up to those posts 🙂