So I did it. Writing this post is actually me saying it’s finished. That’s my homelab, the one with the end goal to have my puckpuck.com domain pointed to a WordPress site running inside Kubernetes, setup in a vSphere environment, served up from the closet in my home over SSL for the world to consume.
Just 2 months ago I had a simple-ish home network setup. ISP modem/router -> My WiFi router -> Switch for wired devices. Also included was a NAS device plugged into the switch that we used for home backups and photos.
When we had this home built 10 years ago we dreamed of having it well connected. During construction, with some friends, I pulled over a mile of cabling through the walls. Mostly ethernet, but it also included coax, speaker wires, and even a few hdmi runs. Most of the cabling terminates in a single room I call the “LAN closet”, though the rest of the family calls “the brain”.
So with all the cable pulled through we ended up with a fairly extensive whole home audio setup, a bunch of splitters in daisy chain for antenna TV, and a 2 post rack to kinda hold it all together. Everything terminated directly into components, no real organization, but it worked and it was reliable. That stayed in place for almost 10 years with just a few minor swap-in updates done.
In its last incarnation before the “redo” you can see we have a Bell branded router for fiber termination in our home. This router was the initial reasoning behind doing this. I can’t say enough bad things about this router. Port forwarding didn’t work, Wifi was awful at near range, devices constantly dropped. I even had Bell replace the router, and was still plagued with the same issues.
So finally out of frustration I set out eliminate the Bell supplied piece-of-shit router and terminate the fiber connection on my own device. This was my fist use of an Ubiquiti device. I got an EdgeRouter to terminate the fiber, and plugged my wifi router into it. Just doing this alone was a significant boost in internet speeds, especially for wired devices. Setting the router up was easy, and though the EdgeMax UI isn’t elegant, I didn’t have an issue finding what I needed.
Then it bit me, the homelab bug. Maybe it’s because I started following /r/homelab, or maybe I just got sick and tired of looking at my tired rack, and really wanted to finish it.
So I set out to plan out what I wanted this homelab of mine to do, and how I was going to achieve it. Took an inventory of what I needed, laid out what the rack would look like from top to bottom, figured out each cable, where it would terminate on a patch panel, and ultimately onto the switch or rack device afterwards. I had multiple spreadsheets and diagrams to help me figure this all out.
Nothing was spared in planning, and I can’t stress how important this is. Equally important is to only create a plan that you can actually commit to. This stuff isn’t cheap. Sure you want to expand to your dream setup, but don’t short yourself today for something you will do in 2+ years. Technology evolves quickly, and 2 years is a long time in this space.
I approached the setup in 3 phases: network, storage, and compute. The phases were repeated for both the home and my Kubernetes setup.
The home network is especially important. Though I want to run all the new techy stuff and configure it ad-nauseam to have fun. Ultimately this needs to just work, especially for my wife and kids. Since wifi was sketchy before, especially in our back yard and deck area, I was able to convince my wife on the notion that I was gonna build a better network. She obliged and I set my plan in action buying all the network gear needed.
My network is all redone now, and except for a 24-port switch I carried over from before, it’s all Ubiquiti now too. That experience with the EdgeRouter convinced me that Ubiquiti is enterprise grade networking that you can use in the home. Perfect for the prosumer.
On the storage front I have my trusted 2-bay Synology NAS that I carried over. Though I have dreams of needing more storage and to dynamic flex that storage, this all costs money, and the existing storage setup I had would be at about 50% capacity (though it would need a re-configure). Later in life, I’ll get a bigger (and rack mountable) Synology NAS.
Finally for home compute I found a Dell PowerEdge R420 on ebay with 600GB of SAS 15k storage. There might be some Dell/VMware bias since I used to work for a VMware, but generally speaking this is a well supported and popular homelab setup for compute. I was getting a recent enough system including dual socket compute without breaking the bank.
After putting it all together I’m very pleased with the physical results of my lan closet.
I go over in detail why I did what for each phase (network, storage, compute) for both the home/homelab as well as Kubernetes inside so I could run WordPress and reach my final goal for this setup… well at least my current final goal 😜
- Homelab network with Ubiquiti
- Easy storage for the homelab
- Picking the right Power(Edge)
- Server + rails in a 2-post rack
- Quiet PowerEdge fans
- Quiet fans on restart, impitool for ESXi
- Kubernetes at home. Am I crazy?
- K8s at home – step 1 is network
- Kubernetes DNS for the outside world
- Kubernetes storage at home
- K8s storage classes and access modes
- K8s – I put in the pain and gained … knowledge.
- Ubiquiti EdgeRouter 4 (main router)
- Ubiquiti EdgeSwitch 10XP (devices switch)
- Ubiquiti EdgeSwitch 10X (services switch)
- TrendNet TEG-S240g (home switch)
- Ubiquiti AP-AC-LR (x3 for Wifi mesh)
- Synology DS218+
- Dell PowerEdge R420
- Amcrest NV4108E-HS (NVR for 5 cameras)
- APC Back-UPS 425
- Rockville 9-port Power Strip
- HTD MA-1235 (amplifier – main)
- HTD MC2-86 (controller for main amp)
- HTD MC-66 (controller for amp in back yard)
- Denon AVR-X1000 (a/v receiver for tv and speakers in adjacent room)
- Apple AirPort Express (AirPlay audio sink used as input source)