Update: I came up with a new parts list based on Sandy Bridge parts. You can check it out here.
Currently I dual-boot my home computer with ESXi 4.1, but that's getting old. Yes, you can run ESXi 4.1 inside of VMware Workstation 7.1 (and I do), but you can't run any 64-bit guest operating systems within that ESXi instance. Since Server 2008 R2 is 64-bit only, along with many other applications like Exchange 2010, that really limits what you can do with nested VMs.
So I broke down today and put together a white box micro-ATX computer for home use that should scream. Total cost was about $1,000, which I don't think is too shabby for the hardware specs. I ordered most of the parts through Directron.com, since they don't charge CA tax and have reasonable shipping costs.
Important factors for me were size (small case), noise level (very quiet), 16GB RAM, and dual NICs that were on the ESXi 4.1 HCL. I looked at Intel Clarksdale motherboards with on-die graphics, so I could eliminate the graphics card, but from what I read their memory performance is in the toilet due to an off-die memory controller. So I opted for a separate graphics card and a 45nm Intel Lynnfield processor. Next year when Intel Sandy Bridge processors are released, that should fix the terrible memory performance due to the new ring-bus micro-archtiecture.
You could ceratinly shave off a few dollars by getting a cheaper case, slower hard drive, less full-featured MB, and a single port NIC.
Antec MicroATX Minuet 350 case $98.99
Western Digital 1TB 7200 RPM 6Gb/s SATA $86.99
Asus Maximus III GENE MicroATX MB $136.98 (after rebate)
Intel Core i5-760 2.8GHz Quad-Core $204.99
Qty 2 Mushkin 8GB DDR3 PC3-10666 kit $129 x 2
Sapphire Radeon HD5450, low profile, fanless $29.99 (after rebate)
SuperMicro AOC-SG-i2 dual-port GiGE NIC $82
NetGear GS088T-200NAS Managed 8-port GigE switch $105.98
Total comes to about $1004.00, plus a few dollars shipping. I already had a DVD drive, so I didn't need to get one. The NetGear switch supports VLANs, jumbo packets, and other features that got my interest. While doing some research, I also found a web site that has a pretty long list of whitebox hardware and an unofficial ESX(i) 4.x HCL. You can check it out here. I can't wait to get all of the parts and put the server together.
How did it turn out? What all are you currently running on it now?
ReplyDeleteThe system works great! I'm running ESXi 4.1 enterprise plus edition and using my QNap iSCSI disk array for most of the VM storage, combined with internal storage. Performance is really great, quiet, and rock solid. I'll probably build a second box this year based on Sandybridge so I can drop the discrete graphics card.
ReplyDeleteHow many VMs can you run at the same time on this box?
ReplyDeleteThat depends on several things, like how much memory you allocate to each VM and how disk intensive the VMs are. I routinely run 5-6 VMs, each with ~2GB of RAM, and it doesn't even break a sweat. The bottleneck for me is disk I/O, not mememory, since I'm not trying to pack on a lot of VMs that use a lot of memory.
ReplyDeleteWhat CPU cooler are you using?
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteToday I received the hardware to built my server, however I can't figure out how the CPU cooler will fit in the case. According to the specs, Antec MicroATX Minuet 350 case height is 94mm and the Noctua NH-U12P cooler height is 158mm ?
ReplyDeleteCrap...I was looking at the wrong order..that cooler was for my new tower computer not my ESX server. I opened my case and confirmed I have this:
ReplyDeletehttp://www.scythe-usa.com/product/cpu/048/scbsk1000_detail.html
sorry! It specificallys states good for HTPCs.
I installed ESX 4.0 and it only see the SuperMicro AOC NIC cards but not the onboard NIC card. Did you also have this problem?
ReplyDeleteOne other thing. How can I have one of the guest OS to access a USB device that is on the host?
Yes the onboard NIC is not supported by VMware. That is why I purchased an add-in card which is supported. I've never had the need to allow the guest USB access, so can't help you there. I'd start by trying to add a USB controller to the VM.
ReplyDeleteHi I purchased the hardware you mentioned. Disabled the onboard nic, loaded esxi 4.1 and everything was fine. Even built a vm. Then I turned the esxi host off without going into maintenance mode. ever since vsphere has been unable to connect to this server. I even rebuilt the esxi host twice used two different vsphere clients. My vsphere client can connect to other esxi 4.1 host. I can not connect to the host to download the the vsphere client from it either. But I can ping it all day. They are on a flat network with only a switch between them. It still says vsphere client can not connect to host. I have tried all the troubelshooting steps on could find in Vmware KBs. Have you even seen this issue?
ReplyDeleteNo I have not need that problem. You might want to reset the network configuration on the ESXi host.
ReplyDeleteLooking closer the hostd process is failing to start. It's just strange that it worked then after shutting down the esxi host one time it corrupted the hostd.
ReplyDeleteHave you booted this server from usb? I am unable to do this as well. The only difference in hardware is a SATA cdrom player. I found that hostd does not start in both 4.1.0-260247x86 and 4.1.0 Update 1 - 348481x86. The server does not see the usb drive. The boot options are set for removable hardware first. If I boot from CD with the USB drive in, ESXi reqcognizes two ESXi installations and throws an alert. I have searched all over the web and can't find anyone with similar errors that I am seeing. Any guidance you can give would be greatly appreciated.
ReplyDeletePlease disregard my earlier post. It now boots from usb if I have the usb emulate a hard drive. Thanks.
ReplyDeleteI have looked at this and found the issue. If I use all 4 memory sticks it fails. I can use any 2 memory sticks in either the A or B slots. Did you tweak the bios settings for memory or anything so it would take all 16Gb of memory?
ReplyDeleteI appreciate any help you can provide. Thanks.
I didn't have to do any tweaking..just plugged in the memory and ran. My friend did have a strange memory issue with his server, so he replaced to sticks and then it was stable. Are all four sticks the same kind? Are they on Asus's certified list?
ReplyDeleteThey are the muskin memory sticks from your list and are all 4 the same. It's weird they all worked on first esxi install. This memory issue only started after shutting the esxi server down once. It will accept any 2 of the sticks in any 2 slots just not more than 2 sticks.
ReplyDeleteI'd try to update to the latest BIOS, clear the CMOS and reconfigure the BIOS, and if that still fails, call Muskin. Can you narrow it down to one or two particular sticks?
ReplyDelete