homelab

· erock's devlog


This post is a living document for my homelab.

status (2026-07-01) #

I ditched proxmox. Ultimately I didn't have a good enough reason to create multiple vms and when I did it was kind of tedious to setup. I went from managing a single server to managing (3) plus proxmox. So I decided to switch back to archlinux, which I've used previously for years. I just find pacman and the system to be a nice balance between linux support, gaming, and package managment. If arch doesn't have a package then the AUR will. I don't bother with an AUR package manager since it's easy enough to git clone + makepkg -si.

I did spend some time setting up qemu in order to get home assistant working. I don't know what it is but I have zero desire to use home assistant. I bought the voice device hoping I could run a small voice model to add items to my todo list but ultimately it was too far behind the mainstream "home" devices. So I spun that VM down and now I really don't have much need for any VMs.

I did spend some time using quadlet which is the podman + systemd integration system so all my containers are managed by systemd. That worked for awhile but unfortunately when working on cross-platform containers podman is just not a great experience. This made me shift away from podman and back to docker.

So now I have archlinux, systemd services where it makes sense, and then containers where systemd services don't make sense. I don't like that I have two separate supervisor systems but they are in their own little boxes which seems to work fine for me. I do share an off-prem rack with Antonio which is running kubernetes. This is where I'm spinning up linux distros to experiement with them. Right now I have alpine and nixos running in a VM there. I love the idea behind nixos but everytime I start using it there's something about it I don't like. I don't know if it's the nix/flake split that I find frustrating when reading docs or the fact that everything feels so tedious. Not only that, but even with this fully declarative system it seems like there are exceptions, especially when it comes to hardware configuration like setting up networkmanager and my network interfaces. If I cannot 1-command spin up a VM with my entire dev environment setup then it feels like the declarative goal has failed. I think it doesn't help that I tend to keep my computers minimal in terms of setup and tooling. It's very easy for me to 1-line install a bunch of packages and be very close to setup. I have a dotfiles repo, a services repo, and then I need to shuttle my GPG key around for passwords. That's pretty much it.

Even my dotfiles are minimal: https://erock-git-dotfiles.pgs.sh/

I'm able to replicate what I need from stow/home-manager/chezmoi in 80 loc. I can go from zero-to-dev pretty fast in arch on a fresh install. I think I'm going to try to build out a couple of shell scripts to help me manage my pacman dependencies so I can get something a little more declarative.

status (2024-07-31) #

I'm finally in a place where I think things are mostly working properly. Let's start from the beginning.

To start, I built a new services VM, cpu, where all of my containers live for various web services. It's also a place where I'm going to be doing some self-hosting of public facing services.

I was having a bunch of issues getting LXC to connect to my nas VM and tried bunch of techniques and eventually got something working. However, in doing that I realized that it was kind of silly to have a nas vm -- that's just running zfs -- when I could just have my proxmox host -- that's also running zfs -- do it instead. They were both running zfs+debian so it felt silly to deal with a nas VM and hdd passthrough.

So I exported my nas zfs pool and imported it into my proxmox host, which worked great! Except I then had another problem: ashift was misconfigured. Darn, that means I need to rebuild my zfs pool.

So I got my old synology out and did data exiltration / infiltration on a fresh zfs pool. There, I think I have everything setup, until I rebooted proxmox after a kernel update: zfs datasets are empty. wtf? After hours messing around with it -- even rebuilding the zfs pool again from scratch -- I figured out what was wrong: I encrypted the disks and didn't mount with the correct flag zfs mount -l -a. Yea, I felt like an idiot.

Now everything is working. I'm still noticing occassional high load / high io when saving a bunch of files to the fs (e.g. package updates). Let's hope I can figure it out. My current suspicion it might be using proxmox with zfs in a 2-disk mirror pool? Not sure.

status (2024-06-25) #

Everything is setup and working really well. I knew GPU passthrough was going to be challenging but I eventually got it to work after about 8-hours of trial-and-error.

Overall really excited about having a system and UI dedicated to virtualization. I plan on distro-hopping a lot more now that it's trivial for me to spin up new VMs.

I'll keep updating this post with more status updates.

Here's a pic of my rack:

homelab

current (gaming) #

next (virtualization) #

last updated:

We're all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde

Subscribe to my rss feed to read more posts.