A home lab can be a fun playground to explore different software packages, connect a few smart lights and generally tinker around. But once your ‘lab’ grade Home Assistant platform becomes something the house must have to function, when Node Red automation is depended for scenes to occur, when Zigbee2mqtt is a necessity to integrate your smart locks… your home ‘lab’ just became a home ‘production grade’ system.
The new names of the game are stability, resiliency, availability. You start asking questions. So you installed some hypervisor and have multiple VMs, but you built everyone on a single server. What if you have a hardware failure? Sure you have backups, but the time required to restore service means your smart switches don’t work, your lights are stuck on and your bathroom vent won’t turn on because the bluetooth humidity sensors aren’t triggering automations.
This series describes one way to solve this problem. Is it the only way? Not at all. Is it the absolute best way? Probably not. Is it a better way than a single point of hardware failure? Definitely. You don’t think so? Read the series. Still don’t agree? Agree to disagree. Want to argue about it? Talk to someone else who cares.
If you’re read this far, then you’re interested in knowing more. Let’s lay out what you’ll be dealing with and start describing suggested hardware components to make it work.
Networking:
This is all primarily IPV4 based. You are going to need a static IP for each node, And this model describes using five nodes. You can get away with using three, but I recommend using five. Always use an odd number. You should (obviously) make sure that they are not within your DHCP pool range. Also, you will want some static IP addresses reserved for virtual IPs (VIPs). You’ll learn more about that later.
Example:
Entire subnet: 192.168.0.0/24
Gateway: 192.168.0.1
Reserved addresses for hardware nodes: 192.168.0.11-192.168.0.40 (30 addresses)
Reserved addresses for virtual IPs: 192.168.0.41-192.168.0.100 (60 addresses)
DHCP addresses: 192.168.0.101-192.168.0.250 (150 addresses)
DNS domain:
You will want two DNS domains. One for strictly internal applications (no outside access) and one for applications you want to make accessible from the internet (outside your network). If you don’t want to host any services that will be available externally, then just ignore the examples using the external domain. I recommend using real DNS domains for simplicity. You will only need to purchase one domain name for this. The internal DNS domain will be a subdomain of the external domain.
Example external DNS domain: dann.com
Example internal DNS domain: home.dann.com
Software:
- Debian 13 “Trixie” – base system (as of the last update)
- Kubernetes k3s – This the orchestrator. It’s what manages the containers, decides which cluster nodes run which containers, and ensures that they keep running. Don’t be intimidated by this, this guide will explain it all.
- Docker – An additional containerization platform that will be used for specific use cases.
- GlusterFS – A distributed filesystem that can span multiple cluster nodes. Fast and resilient.
- Ansible – A configuration management tool for deploying software and configuring systems.
These software components will all tie together to create the production grade cluster for your home. They’re also all free.
Hardware
- HP EliteDesk 800 G6 Desktop Mini (5x)
- i5-10500T processor
- 16GB memory
- 256Gb NVMe for base system
- 500Gb NVMe for GlusterFS
- A dummy DisplayPort plug to simulate a monitor connection. Required for vPro functions.
You might look at this hardware requirement and raise an eyebrow. Let’s boil it down:
- The G6 (sixth generation) has several advantages over the G1/G2/G3 models. It has two (2x) NVMe slots (instead of just one). Also, the 10th generation i5 has six (6x) cores instead of four (4x).
- There isn’t a significant enough performance gain by jumping to the newer models (G7/G8/G9) to warrant the cost.
- You can get used G6 models from eBay with the i5-10500T processor, 16GB of memory (a must) and a 256gb NVMe for $200 USD. You don’t care if it has Windows preinstalled or a wireless card. You should make sure it has a power supply.
- Make sure you get the i5-10500T processor, not the i5-10500 (non-T). The T series processor has a lower max TDP. You’re don’t need to build a powerhouse, you’ll do just fine with a cluster of lesser capable systems that distribute the load.
- The 256Gb NVMe is for the base system (operating system). The 500Gb NVMe is for the data disk (glusterfs system). It’s a best practice to separate your system disk from your data disk.
Here are some frequently asked questions:
- Why use HP minis? I prefer Lenovo ThinkCentre Tinys.
Knock yourself out. I don’t really care what you use, this is just what I can recommend from experience. - What about HP ProDesk 600 minis?
The EliteDesk 800’s come with vPro, which provides (among other things) remote management and virtual KVM control of the system over TCP. That’s built into the BIOS, so you can use it to manage the system even if the installed base system fails. The ProDesk 600’s lack this. They’re not worth it. - Five of them!? Isn’t three good enough?
No, it’s not. If you’re really serious about having a cluster that will run all your software and be able to handle critical hardware failure, you need five (5x) nodes. If you have five nodes, you can drop two (2x) nodes and still maintain a quorum of N/2+1. You need to maintain quorum for GlusterFS/Kubernetes. If you have only three nodes, you can lose one… but without an odd number of nodes left you run the risk of a split brain cluster. That’s bad. Very bad. Go with five nodes now, you won’t regret it later.
Summary
So yeah, it feels like a significant financial outlay. But if you do the math, you’re coming in at probably just about or under $1500. How much is your up-time worth?
Trust me, it’s worth it.