VMware ESXi: Network-In-A-Box
Published: 03/14/2009
I've been playing with VMware since Workstation was at version 3.something. Eventually, VMware Server was available and I implemented that in my work environment since GSX Server wasn't in the budget at the time. Soon after, VMware released something apparently magnificent called ESX. One look at the pricing, however, and I decided this wasn't going to make it on our annual budget. Sounds nice and all, but I'm known to be cheap. I'm the guy at Wendy's asking for a discount on the Super Value Menu. At one point we did stage one ESX server on a Dell PowerEdge 2950 with two Xeons, eight Gigs of memory, and RAID10 over four disks dedicated for VMFS storage (two other disks in RAID1 were dedicated for the OS / hypervisor). It was quite fast and cloning a ten GB disk image took roughly ten minutes.
But when ESXi came out, I decided to take the six older 2850s we had lying around, upgraded them to consistency, and made a dedicated virtualization farm using Openfiler as a iSCSI SAN (running on, believe it or not, a really old PowerVault 200S). It may not be performance-oriented, but it works just fine for the kind of thing we do.
I've been running ESXi at home for my personal network since ESXi was made available for free by VMware. My friend gave me his old dual AMD Opteron 246 and I somehow managed to run over ten guests on the system over a two-disk SATA RAID1 set. However, I needed something a little bit more compact so I decided to build a semi-stylish whitebox that's very quiet in operation but yet can drive almost my entire home network including domain controllers, a certificate authority, RADIUS, WSUS, a couple of Windows clients, a few CentOS hosts for various applications, a few OpenBSD hosts for OpenNTP and syslog-ng, an isolated prototyping network, and even my firewall.
The build list
Here are the parts I recently put together:
(1) Intel Core 2 Quad Q8300 2.5 GHz CPU
(1) Intel DG33BU MicroATX motherboard
(4) Kingston KVR800D2N6/2G memory sticks
(4) Hitachi 320 GB SATA 7200 RPM 2.5" drives
(1) PCMCIS PCM-BPSATA1842B backplane unit for 2.5" SATA drives
(1) HP E200 SAS / SATA RAID controller
(1) Corsair CMPSU-650TX
(1) Intel PWLA8391GTBLK Ethernet interface
(1) Lian Li PC-V350B
I also added a couple of slightly-used 3.5" disks and a spare Intel NIC that I had lying around. When factoring in the sales and mail-in rebates, I built an ESXi whitebox for roughly $1,100 after everything was said and done with six disks inside. While ESXi does not support the particular NIC built into the motherboard, the two PCI Intel NICs works like a charm. I dedicated one for the management VMkernel port and the other for all the other port groups bridged to a common vSwitch. I set up my Cisco switch port connected to this second NIC to be a trunk port and now I can have as many VLANs as I want through a single interface by assigning a VLAN ID for each virtual port group. I even run my firewall (inside and outside interfaces only at this time) with this setup and it works like a charm.
I'm currently running about fourteen virtual machines with this setup, although I figure I could add a few more Server 2008 hosts.
The downside to all this is my choice in a RAID card. The HP E200 is a full-length card which the Lian Li case does not fully accommodate. The motherboard tray has to remain pulled out a couple of inches, but that's a cosmetic flaw that I can't even see from where I'm sitting. However, it's a card that allows me to (if I choose to do so in the future) install another PCM-BPSATA1842B backplane unit into the second 5.25" bay and add four more 2.5" SATA drives. That's quite a bit of drive space for fifteen to thirty VMs, even if you factor in the snapshots.
If you want to know what it looks like, here's a picture of the Lian Li case:
It's a contemporary Borg cube server with a brushed-aluminum finish. Seven of Nine not included.
I didn't replace any of the stock fans, yet this box is still near-silent. I only wish my Cisco 2950 was as quiet.
So if any of you were contemplating on building your own network-in-a-box, here's one way to roll your own ESXi server that doesn't exclusively use components that are on the VMware HCL. Because it's no fun to power your home network with a few measly desktop machines and a boring blue Linksys router hanging at the edge.
Go back to the main articles list.