Architecture Overview

This is the big picture of how the lab is put together. For the reasoning behind each choice see design decisions, for the full component list see the technology stack, and for how requests reach an app see traffic flow.

Built in layers

The lab is built bottom-up. Each layer assumes the one below it. Lower layers rarely change once they work; the top layer changes almost daily.

flowchart TB
  L0["Layer 0: Hardware"] --> L1["Layer 1: Debian"]
  L1 --> L2["Layer 2: Proxmox VE"]
  L2 --> L3["Layer 3: TrueNAS VM"]
  L2 --> L4["Layer 4: Talos and Kubernetes"]

Each layer has its own tutorial.

Physical layout

A handful of mini PCs form a Proxmox cluster. Proxmox then runs two kinds of VM: a TrueNAS box for storage, and the Talos nodes that make up the Kubernetes cluster.

flowchart TB
  switch["LAN switch"] --> nodes
  subgraph nodes["Proxmox cluster"]
    n1["Node 1"]
    n2["Node 2"]
    n3["Node 3"]
  end
  nodes --> tn["TrueNAS VM (storage)"]
  nodes --> talos["Talos VMs (Kubernetes)"]

Everything through GitOps

Argo CD is the only thing that writes application state to the cluster. It reconciles the cluster to match Git, and ApplicationSets generate the individual apps from the directory tree. A rollback is a git revert.

flowchart LR
  git["Git (GitHub)"] --> argo["Argo CD"]
  argo --> cluster["Cluster state"]
  argo -. detects and corrects drift .-> cluster

Two ways in

Traffic reaches an app through one of two planes, chosen per service:

• Public services on ${FQDN_01} are routed with the Gateway API, entering through the home router. Envoy Gateway is the controller behind it, terminating TLS and running a WAF.
• Private services on ${FQDN_02} are published onto the Tailnet with Tailscale Operator, and are only reachable from devices on the Tailnet.

The full path for each is in traffic flow.

Storage and secrets

• Storage: Longhorn provides replicated block volumes for application state. Bulk data and media live on the TrueNAS VM, mounted into pods over NFS and SMB.
• Secrets: nothing sensitive is committed to Git. The External Secrets Operator pulls secrets from Bitwarden Secrets Manager at runtime; the only secret kept locally is the bootstrap token. See Bitwarden access tokens.

Resilience

Three control-plane nodes let Kubernetes tolerate losing one. Because the cluster is defined in Git, rebuilding it is a matter of re-running the layers and letting Argo CD sync; the data that matters lives on TrueNAS and is backed up separately.