Three Labs for three reasons

Chalk up 2012 as the year I started paying attention to what everyone in the Virtualization world was doing for their vSphere home/portable labs. Well, it was 2011 to be more precise, but I just didn’t act on it until this year. Since I decided to dive into a lab head first, I thought I’d share with you what I have, and how its been working.

I was reminded of the power of a lab environment last year while I was building out a CoLo site for DR, offsite hosting and VDI for my company. It was more than once that I thought, “gee, this is nice that I’m not playing around with the production site.” I knew that it was time to start thinking about what I wanted for a lab.

If you start digging into the whole home/portable lab movement, and you find out that the ideal home lab is really dependent on what your needs are. Some are perfectly satisfied with a vSphere lab nested inside of VMware Workstation, while others have physical labs that will make the lights dim. Both are viable options, but I’ll tell you what I settled on for mine, and how everything has been working. Regardless of what you end up choosing, some great hardware and software can produce some pretty fantastic lab environments.

But first, time for a change…

So why the need for all of this? Well, in June of 2012, I decided to take my career in a slightly different direction. After 13 years as the Senior Systems Administrator for a software company in Bellevue, WA. I took a position with Mosaic Technology, a Solution Provider and Channel Partner for VMware, Dell, and others. As a Senior Systems Engineer, I now get the opportunity to design and implement virtualization solutions, while applying my practical experience in the trenches with technologies and solutions in every corner of IT. This is extremely exciting for me, as I get to focus more the very software that altered the course of my career just 5 years ago. I get to work with a great team at Mosaic, many of whom I’ve known for quite some time, including my good friend Tim Antonowicz. Over the years I’ve also had the opportunity to establish relationships with many at Dell, including Dell EqualLogic, and Dell TechCenter. I get to continue to work with these folks, but in a different capacity.

The Home Lab

For my home lab, I wanted to simulate some sense of a real world environment. For that you need real hardware; real NICs, and real resources. I also had the desire to eventually run a nested environment in each physical host, so I didn’t want to skimp on physical resources too much.

As much as I wanted real hardware, I also didn’t want my house to sound or feel like a datacenter, so for me, being mindful of power consumption was as much about heat generation as it was about noise, and cost. Another goal of mine was to avoid scenarios where I’d buy something twice where the first time met a certain price point, and the second time to get what you really needed the first time around. For most people, that usually means RAM. Buy it once and be done with it.

Compute:
My physical hosts (qty. 2) most closely reflect the setup that Chris Wahl has described in his home lab. The main differences are: 1.) I dumped 32GB of RAM in each host, and 2.) I threw in two dual port NICs, and a single port NIC. I wanted a minimum of 6 functioning NICs for vSphere. The SuperMicro motherboard comes with two onboard NICs (not including the IPMI port). One is an Intel 82574L which is supported by vSphere 5, but the other is an Intel 82574LM and isn’t easily recognized. You can get it to work, but it was worth $20 for the additional NIC, especially during host rebuilds. A few notes about this setup

If you go with this particular SuperMicro motherboard, keep an eye out for ECC Unbuffered DDR3 DIMMs, as this motherboard requires it. They are down to about $90 a stick as of the time of this writing, so about $360 to populate the host with 32GB of RAM.
IPMI has proven to be extremely valuable. Not only will it allow for some nice remote rebuilds of the hosts, but is a perfect match for DPM in vSphere.

Storage:
For storage, I settled on a Synology DS1512+ NAS unit. I populated the enclosure with two, 128GB Crucial M4 SSDs, and three 2TB 5400 RPM SATA drives. The interesting feature of the DS1512+ is that it has two NICs on the back. This offers up a little flexibility for multipathing iSCSI, or splitting off NFS to a different interface/network.

Synology has been getting a lot of press with the Home Lab crowd, and if you use it, you’ll understand why. The DSM (their OS) provides an easy, flexible way to serve up NFS, CIFS, or iSCSI, including VAAI support. Plenty of other features will keep you entertained as well. The enclosure should fit my needs even as the drives themselves may change.

Switching:
With my desires of so many NIC ports, I knew that they’d get sucked up pretty fast. So a 20 port switch was the minimum. My other requirement was that it had to be fanless. 1U anything with a fan seems to be nothing but a noise maker, I didn’t want that. I settled on a Cisco SG300-20 switch. This is a full layer 3 managed switch that is a real gem. While it doesn’t run IOS, it does have a CLI, and supports just about everything you’d want in a home lab. Inter-VLAN routing. CDP, LLDP, jumbo frames, etc. It’s been fantastic. I feed this switch to a Cisco/Linksys WRT-160NL flashed with DD-WRT so that my lab has internet access.

So, how much power does all this draw? All together, around 160 watts or about 170VA. Yeah, that’s right, under light load, the entire thing is drawing minimal power, with minimal heat, and just a whisper of noise. Considering that a Dell Precision T5500 workstation alone pulls about the same amount of power, I’m extremely happy with the result. Here is how the running load works out.

Device	Running Watts	Running VA
Switch	10	17
NAS	38	34
Host1	57	60
Host2	57	60
Total	162	171

My only complaint is the goof-ball form factors of the NAS, the Lian Li chassis and SG-300-20 switch. No amount of reorganizing them have resulted in an orderly arrangement of systems. I’ll have to build something to accommodate.

The Portable Lab

Due to the job change, I also had the need to have a portable lab (running nested vSphere inside of VMware Workstation); something I could guarantee to spin up if needed. Wifi access wouldn’t necessarily be available where I needed a lab, so a portable lab on a laptop solved this problem. But a portable lab does require some real horsepower, so that is where a Dell Precision M6600 comes into play. This 17” laptop is a beast in every sense. Yeah, it’s a tank to lug around, but it has a 17” screen, quad core i7 processor, and 16GB of RAM (expandable to 32). I have a 256GB Crucial M4 SSD to run the OS and some of the VMs, while a second internal SATA drive carries the bulk of VMs, user data, etc.

I set up the lab in VMware Workstation a few different ways. A few times on my own, from scratch, then a few times using the “AutoLab” Both ways will end up with similar results; a functioning nested vSphere environment. The AutoLab definitely saves time when it comes to the rebuild process. I’ve found that the perfect mix so far has been to install the AutoLab, then tweak where I see fit (typically networking changes) based on personal preferences, or requirements.

With the assortment of powered VMs up required to run the nested lab, I typically use around 10GB of RAM on my system. That doesn’t leave much left over to run additional VMs, but with a little creativity, its workable.

The Verdict

So which one do I like better? Honestly, they both are extremely valuable in their own ways. But here are some generalizations.

If you are testing anything related to networking, nothing seems to beat the physical lab, as it is going to mimic the real deal.
If you are pushing any sizable workload with VM’s (in quantity, or allocation size of VMs), the physical lab shines. With a portable lab, even with 16GB of RAM on a workstation, you really have to trim up the VMs. But then again, it’s a lab, not production.
For professional development, study, documentation, experimenting, and customer demonstrations, the portable lab is second to none when it comes to convenience and accessibility.
The speed at which you can test a setting of one’s ESXi hosts, vCenter, or play with some scripting, is just fantastic. The availability of the lab makes it incredibly valuable.

The comparison is almost similar to the SLR versus point-n-shoot camera debate. One might be technically superior, but if you don’t have it with you because it’s too bulky, etc. then what good is it? This analogy is where the portable nested lab on my laptop proves to be incredibly valuable. You will find those who have run a portable lab and went physical because they were tired of nesting ESXi, and others who have ran physical and moved to a portable setup. It really depends on what your needs are. I continue to use both as the needs dictate.

The third Lab

The third lab might be my most important. This little guy is the most reliable lab I’ve ever had. Here he is on high alert guarding my other lab.