Running K3s on Bare Metal with Ceph — Notes from WoNQstaq_v3

WoNQstaq_v3 is a three-node Hetzner K3s + Ceph yard with GitOps, storage, registry, CI/CD, and observability stacked on owned hardware.

WoNQstaq_v3 artwork

WoNQstaq_v3 is not a fantasy whitepaper cluster. It is a real three-node Hetzner yard built around owned metal, replicated storage, and the stubborn belief that a lot of things are more fun when you actually operate them yourself.

The current setup is three dedicated servers spread across different rooms in the same datacenter. That is deliberate: close enough to keep east-west latency stupidly low, far enough apart to avoid one local power drama wiping the whole thing at once.

The Physical Shape

At the current state, the cluster is built around:

3 dedicated servers

K3s for the Kubernetes layer

Ceph RBD across all three nodes

replica_size 2 for the current storage profile

96TB raw bulk HDD + 3TB raw NVMe underneath the cluster

roughly ~48TB net bulk and ~1.5TB net fast replicated capacity after redundancy math does its thing

That is the kind of setup that gives you real room for archives, builds, registries, sites, media workloads, and all the side systems that pile up once your yard starts getting busy.

The Control Stack on Top

The attached tarball also confirms that this is not just “Kubernetes and vibes.” The cluster carries real operator tooling on top:

Forgejo for Git

Zot for image registry duties

Drone for CI

ArgoCD for CD / GitOps reconciliation

Prometheus-based observability in the monitoring layer

Cilium in the networking / egress-policy story

That mix is exactly what makes the yard useful: source, images, deploys, storage, and visibility all living in the same self-owned ecosystem.

What the Manifests Make Obvious

The attached material shows workload directories for multiple public and private services riding on top of the cluster, plus ingress resources, Cilium egress policies, PVC usage, and service manifests wired around the stack.

That is the big difference between a lab note and a living infra yard: you can see real workloads hanging off it.

Why Bare Metal Still Wins for This Kind of Work

Control. Density. Predictable resources. And no managed-service tax for the privilege of being abstracted away from your own failures.

Bare metal means you deal with the sharp edges yourself, sure. But it also means the platform can be shaped around the actual workloads instead of being bent to fit a provider’s menu.

Ceph Is Still Ceph

Ceph remains powerful, moody, and extremely educational in the “you will learn this one way or another” sense.

That is not a complaint. It is part of why a setup like this teaches so much. Once you run your own storage layer, you stop talking about infra like a slide deck and start respecting it like a living machine.

For Rikketik, the honest project story is this: WoNQstaq_v3 is the infrastructure backbone that the rest of the yard leans on. It is not just a cool logo and some cluster words. It is the actual operator ground beneath the visible projects.