Recently I was about to change my Home lab. I wasn’t changed since 2013. I works stable, everything what were able to virtualize were virtualized. Things like my CCTV, webmail, NAS are running on dedicated separate server. Another dedicated server is my firewall, sorry, but security is vital priority. But my third server is virtualization station for tasks like web servers, remote sandboxes, development environments and various sets of VPN channels. Everything run quite smoothly on Windows Server 2012 R2 under Hyper-V. The 16 GB ram with 8 cores and several SSD and HDD drive are enough to separate all the tasks.

So why I decided to make some changes if everything is up and running ?

• Windows Server 2019 here. Means EOL of Server 2012 R2 is the matter of time.
• Too many adired screaming around KVM, ESXi and XEN
• I had huge experience with ESXi versions 3.3 till 5
• I wish to make hardware upgrade for this virtualization system and this is perfect opportunity to try something new
• KVM, ESXi and XEN are free (the Windows Server 2012 R2 was obtained almost for free £400 with outdated dell server on Ebay). Means after EOL Windows Server 2012 R2 I’ll have to pay to Microsoft crazy price.

Early Saturday in the morning I found that my total night backup was completed and I’ve started my efforts to test and implement following solutions:

• KVM on Debian6. Debian is my favorite distro for servers.
• ESXi7
• XEN 7.6
• Windows Server 2016 (freshly downloaded from Microsoft)

To makes things short I will simply mention what I experienced during the installation and maintanace process, what admired and disgusted me most. And after I will bring some PROS and CONS table about these four products.

I had 4 outdated OCZ SSD VERTEX drives each 64 GB since 2012 when I was making RAID10 drive for my wife’s video editing tasks. She uses now Samsung M.2 EVO 970 512 GB so I was able to use each small SSD per each product listed above.

KVM.

What admired me most that KVM is real open source free solution with no limitations at all. Well documented. I had no troubles with installing. Everything was setup via APT tool. I even install VNC server to make WEB interface oVirt for managing virtual machines. I didn’t like it and I had to install Desktop Environment (my favorite XFCE 4.12) and after install Virtual Machine Manager (virt-manager). I didn’t like it either. I couldn’t create bridge Ethernet connection to newly created virtual machine via any of these interfaces. And as a result everything should be done via long commands in CLI. I could survive this. But there was huge disappointment, and I will notice about it strait away just not to make you read everything till the end. Once I’ve deployed Postgresql 9.6 for my ZoneMinder and simply allowed my physical CCTV server to use it’s database I was frustrated. I lost almost 21 % of performance in comparison with not virtualized postgresql which was installed alongside with KVM on this debian installation. After I’ve googled about this problem and found that this is quite common exactly for this situation postgresql+KVM. And nothing I can do about it.

ESXi6.7

Well, it was a big disappointment since the beginning. I knew that since the time of earlier versions and thought that all bad days has gone. But to my great surprise ESXi still continues its policy for short listing of compatible devices. So my motherboard with realtek NIC was not supported. I’ve spend 2 hours trying to implement driver into ISO file and create bootable USB but finished with changing motherboard and spend another half of the day to resolve this trouble. Next morning I’ve realized that my efforts will not be completed with small blood (time) and I left it until next weekend. Next Saturday once all the steps of exchanging CPU and drives was finished I performed ESXi 6.7 setup and changed all settings. I was already disappointed with this compatibility problem, but as I had experience with ESXi and it was installed and running I’ve decided to continue.

Another disappointment was ESXi Client. I couldn’t find it. I tried to install 5.0 version, but it was not compatible, so I had to use extremely limited WEB client version it. On the top of it couldn’t register it to get the free license. The registration window continuously redirected me to the not existing page on VMWARE website. I also found the core limitations for my virtual machines. But what kill me completely was inability to add separate storage ESXi without formatting it first in its own file system. CLI interface option to perform this task was rejected after an hour and a half hour. Playing with install debian and Windows 10 on virtual drives was normal, but I couldn’t restore my postgresql SQL to perform the load from my CCTV with generates almost 90 Mbit/s from 11 cameras. So nothing to say about performance experience.

Full disappointment and frustration about this product.

XEN 7.6

A whole new animal for me. Installed smoothly, almost same setup looks as ESXi. No compatibility issues with realtek NIC or any others devices. Easy options setup, same as ESXi. Webinterface same limited as ESXi. But no ability to connect XEN client. It’s simply not working and same way limited for sockets and cores. Connectivity of EXT4 drive via CLI required some time. Connectivity of ReFS and NTFS drives failed, or maybe it is my fault. After installing a single core debian virtual machine with postgresql for CCTV test I found it was as fast as on real hardware.

In general opinion, I didn’t like XEN. Better than ESXi but still the same. Or maybe I simply don’t like “Bare Metal” way of virtualization.

Windows Server Hyper-V 2016.

Smoothest installation. 180 Day of free full functional trial. Quickly added all my drives. All virtual machines from separate HDD/SDD drives were quickly imported or recreated via Hyper-V wizard. The only problem I’ve experiences was with on of the WEB server on Debian 9 installed on SSD 256 GB toshiba which was recreated with IDE drive emulation and dropped the disk performance from 460MB/s on physical test to 210MB/s (hdparm –T) in virtual environment. This was only minor disappointment on with Hyper-V. What I also found that there is build in Hyper-V client on Windows 10 which looks exactly the same way as snap-in on Windows Server 2016.

No issue with drivers. Best performance for all my tasks. Easy to mange virtual machines… But I do not wish to pay for this. There is Hyper-V Cli only solution and in future I think I will move to it. But this research consumed 2 weekends and I need a break.

 

My overall opinion is the following. If you wish predictable virtualized environment and you are not IT specialist go to Windows Server Hyper-V. I definitely cannot recommend ESXi or XEN. And if you got vast amount of time and like linux CLI you will definitely be happy with KVM, and maybe in far future I will do the same.

As for me I returned everything back to Windows Server 2012 R2 Hyper-V. Robust, quick and predictable solution which is legal and paid.

Now Pros and Cons of these 4 products: KVM, ESXi 6.7, XEN 7.6 and Windows Server 2016 Hyper-V

	Pros:	Cons:
KVM	·         Free. No limitations ·         Open source ·         Hardware compatibility ·         Easy to install ·         Non bare metal solution (installed upon the Linux OS)	·         No proper free client for managing ·         Limited managing and maintenance via free GUI ·         Poor performance of postgresql 9.6
ESXi 6.7	·         Partly free with limitations ·         Well known oldest product with reputation ·         Bare metal way of hypervisor (not benefit in my opinion)	·         Strict device compatibility restrictions ·         Own File System ·         No more free client. ·         License limitation for CPU cores and sockets. ·         For something real productivity other than testing you have to buy a license
XEN 7.6	·         Partly free with limitations ·         Ex-Citrix child ·         Bare metal way of hypervisor (not benefit in my opinion) ·         Hardware compatibility ·         Perfect performance	·         License limitation for CPU cores and sockets. ·         Troubles with using free license ·         Limited WEB client ·         For something real productivity other than testing you have to buy a license
Windows Server 2016 Hyper-V	·         Hardware compatibility ·         Perfect performance ·         Perfect Desktop Client ·         Perfect “free” Client in Windows 10 ·         Option for completely free Core only Hyper-V in Cli	·         Price ·         Most proprietary software – well it’s Microsoft