With my recent post on what’s currently running in my vSphere Home Lab, I received a number of questions about one particular part of the lab; that being my Management Cluster built with Intel NUCs. So here is a quick compilation of those questions (with answers) I’ve had around this topic.
Why did you go with a NUC? There are cheaper options.
My approach for a Home Lab Management Cluster was a bit different than my regular Lab Cluster. I wanted to take a minimalist approach, and provide just enough resources to get my primary VMs off of my other two hosts that I do a majority of my testing against. In other words, less is more. There is a bit of a price premium with a NUC, but there also is a distinct payoff with them that often gets overlooked. If they do not keep up with your needs in the Home Lab, they can be easily repurposed as a workstation or a media PC. The same can’t be said for most Home Lab gear.
Is there anything special you have to do to run ESXi on a NUC?
Nothing terribly difficult. The buildup of ESXi on the NUC is relatively straightforward, and there is a growing number of posts that walk through this nicely. The primary steps are:
- Build a customized ISO by packing up an Intel NIC driver, and a SATA Controller driver, and place it on a bootable USB.
- Temporarily disable AHCI in the BIOS for just the installation process
- Install ESXi
- Re-enable AHCI in the BIOS after the installation of ESXi is complete.
How many cores?
The 3rd generation NUC is built off of the Intel Core i5-4250U (Haswell) chipset. It has two physical cores, and will present 4CPUs with Hyper-Threading. After managing and watching real workloads for years, my position on Hyper-Threading is a bit more conservative than many others. It is certainly better than nothing, but many times the effective performance gain is limited, and varies with workload characteristics. It’s primary benefit with the NUC is that you can run a 4vCPU VM if you need to. Utilization of the CPU from a cluster perspective are often hovering below 10%.
Is working with 16GB of RAM painful?
Having just 16GB of RAM might be a more visible pain if it were serving something other than Management VMs. The biggest issue with a "skinny" two node Management Cluster usually comes in when you have to throw one into maintenance mode. But much like having a single switch in a Home Lab, you just deal with it. Below is what the Memory usage on these NUCs look like.
There are a few options to improve this situation.
1. Trimming up some of your VMs might be a good start. Virtual Appliances like the VCSA are built with a healthy chunk of RAM configured by default (supporting all of that Java goodness). Here is a way to trim up memory resources on the VCSA, although, I have not done this yet because I haven’t needed to. Just don’t use the Active Memory metric as your sole data point to trim up a VM Memory configuration. See Observations with the Active Memory metric in vSphere on how easily that metric can be misinterpreted.
2. Look into a new option for increasing RAM density on the NUC. Yeah, this might blow your budget, but if you really want 32GB of RAM in a NUC, you can do it. At this time, the cost delta for making a modification like this is pretty steep, and it may make more sense to purchase a 3rd NUC for a Management Cluster.
3. Adjust expectations and accept the fact that you might have a little memory ballooning or swapping going on. This is by far the easiest, and most affordable way to go.
How is ESXi on a single NIC?
Well, there is no getting around the fact that the NUC comes with a single 1GbE NIC. This means no redundancy, and limited bandwidth. The good news is that with just one NIC, you can monitor this quite easily in vCenter! Since you are running all services and data across a single uplink, it may be in your best interest to run a Virtual Distributed Switch (VDS) to properly control ingress and egress traffic, and make sure something like a vMotion isn’t going to wreak havoc on your environment. However, transitioning a vCenter VM to a VDS with a single uplink can sometimes be a little adventurous, so you might want to plan ahead.
If you must have a 2nd NIC to the host, take a look here. Nicholas Farmer showed quite a bit of ingenuity in coming up with a second uplink. Also, don’t forget to look at his great mini-rack he made for the NUCs out of Legos. Great stuff.
How do they perform?
Exactly the way I want them to perform. Out of sight, and out of mind. Again, my primary lab work is performed on my Micro-ATX style hosts, so as long as the NUCs can keep the various Management and infrastructure VMs running, then that is good with me. Some VMs are easy to trim up to provide minimal resources (Linux based syslog servers, DNS, etc.) while others are more difficult or not worth the hassle.
Why did you put two SSDs in them?
This was for flexibility. I wanted one (mSATA) drive for the possibility of local storage if I decided to place any VMs locally, as well as another device (2.5" SSD) for other uses. In this case, I decided to apply a little PernixData FVP magic on them and use one of them in each host to accelerate the VMs. The image below shows the latency of the VCSA, which has about 99% writes. Note how the latency dropped after transitioning the VM from Write-Through (read caching) to Write-Back (read and write caching) to a consistently low level. Not bad considering all traffic is riding across a single link, and the flash device is an old SSD.
(click on image to enlarge)
Would you recommend them?
I think the Intel NUCs serve as a great little host for a Management Cluster in a Home Lab. They won’t be replacing my Micro-ATX style boxes any time soon, nor should they ever be part of a real environment, but they allow me the freedom to experiment and test on the primary part of the Lab, which is what a Home Lab is all about.
Thanks for reading.
8 thoughts on “Your Intel NUC Home Lab questions answered”
Would you recommend it for running a single VM? I’m setting up VM’s to learn things like Oracle DBA, LAMP, Ansible , and so on, but I really on need a single VM turned at on a time. My Mac Mini 2014 struggled with just one VM though, so I’m considering getting a dedicated virtual host system.
It really depends on the size of the VM(s), but with the with 16GB installed on the NUC, you should be able to run at least a handful of VMs. Ansible or a LAMP stack should be no problem at all. I have a 3 node Management cluster comprised of NUCs, and have about 8-10 VMs running on them, with a few of them that have been allocated 8GB of RAM (VCSA, etc.)
Thanks. I think I’ll be more than set with a single NUC with 16GB RAM and the tall chassis so that I can put a 2TB 2.5″ HD inside. From what I’ve read, esxi will run on it, and for headless operation, I’ll need to buy a HDMI adapter such as the one by fit-headless to trick it into thinking I have a monitor attached. So the only question left is which CPU ( i3-i5-i7 , though I’m leaning toward i5.
Things will fall over for you with a spinning disk. If it doesn’t choke on its own, it will be painfully slow for you the moment you put any workload on it. Remember you’d get maybe 60 IOPS at most out of that, and your latency would be through the roof. You need to go SSD to even make this workable. (Trust me… it won’t matter what proc you have in there if you have a spinning disk). Remember you have an mSATA slot, along with a 2.5″ slot with the slightly thicker form factor. Can can build up and boot ESXi off of one of the USB slots. The only thing you need a monitor for is to set up ESXi. After that, you should be good. So just buy whatever cable that suites your setup best (mini-DP to DP, or mini-HDMI to HDMI)
I am thinking of getting myself one of the i7 version of NUC (BOXNUC5i7RYHR) with 250 m.2 ssd and 16gb ram.
my plan is to utilize this as a complete server lab running on VMs. (at least 4 VMs simultaneously – DC, AD, SQL and SharePoint). I was wondering, would the NUC be ok for this setup?
If you size the VMs very conservatively, you might be okay. You scale down an AD/DC pretty easily, but there is not getting around the fact that SQL and SharePoint are hogs. CPU will probably be the least of your worries. Disk I/O and the 16GB of RAM will be where you hit your limits first. Be mindful of he extremely low DWPD ratings for some of the m.2 SSDs. Quality of flash will be the difference between it performing consistently or not.
Samsung 850 EVO 250 GB M.2 SSD is the one that I am planning to purchase. I also have 2.5″ SSD that I will be using to store the VMs on.
My main constraint is that the system has to be very portable for me to be able bring places to demo stuff (SharePoint etc). I wish I could get a NUC with 32gb ram.