Nested KVM Hypervisor Support
Adding Hardware Virtualization Support to a VM in KVM with Nested Hypervisor support is pretty important for folks wanting to do proofs for VMs, IaaS, OpenStack etc. It is fairly well supported now under both KVM and Xen. Not to mention, who wants to lab up bare metal when they can use their laptop for modeling. The major driver for nested virtualization is the ability to run nested Hypervisors in IaaS and/or Public Cloud services. It will be attractive for some to run VM farm as a service instances. This provides added flexibility to the customer to maximize their subscription of leased cloud instances.
Nested hypervisor Support On Laptops →
Add Nested Hypervisor support in the KVM-Intel or KVM-AMD KO Module
|
1 2 3 4 5 6 7 |
#Unload and reload kvm-intel kernel module with nested support. modprobe -r kvm-intel modprobe kvm-intel nested=1 cat /sys/module/kvm_intel/parameters/nested Y (Y for yes) |
Boot the ISO with Nested Hardware Virtualization support. If you get
This is slightly different from the Rackspace guide. The difference is in the form of that loads the VMX HW virt support for the VM.
|
1 2 3 |
-enable-kvm -cpu qemu64,+vmx |
The ability to run nested KVM hypervisors is extremely important for lab environments for test/dev. For some reason KVM pulled support a while back but this is a work around for now written by IBM. More on it in a Sigg here and the code review from the legend himself Torvalds Github.
Booting a KVM Nested Instance
An example booting the RackSpace OpenStack ISO installer.
|
1 2 3 4 5 6 7 8 |
/usr/local/bin/qemu-system-x86_64 \ -enable-kvm -cpu qemu64,+vmx \ -uuid $CONTROLLER_UUID -drive file=rackspacecloud_controller.qcow2,if=virtio \ -cdrom alamo-v1.0.0.iso -boot d -m 8192 -net nic,macaddr=$CONTROLLER_MAC,model=virtio,vlan=1 \ -net tap,vlan=1 -net vde,sock=/tmp/vde_tap0.sock,vlan=1 \ -vnc :1 & |
If you get errors when you try and run the qemu-system-x86_64 command they are likely missing the VMX support.
Booting a with an iso to build an image with an OVS tap:
|
1 2 3 4 5 6 7 |
qemu-system-x86_64 -enable-kvm -cpu qemu64,+vmx \ -m 1024 -hda /media/Storage/imgs/centos.img \ -net nic,macaddr=00:11:22:CC:CC:C5 \ -net tap,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown \ -cdrom /media/Storage/vm-images/CentOS-6.2-x86_64-LiveCD.iso |
Booting a built image with an OVS tap:
|
1 2 3 4 5 6 |
qemu-system-x86_64 -enable-kvm -cpu qemu64,+vmx \ -m 1024 -hda /media/Storage/imgs/centos.img \ -net nic,macaddr=00:11:22:CC:CC:C5 \ -net tap,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown |
If you get errors when you try and run the qemu-system-x86_64 command they are likely missing the VMX support.
Para-Virtualization and Full Virtualization
Some hypervisors support Para-virtualizing and Full Virtualization in hypervisors and . Most hypervisors support paravirtualization through the addition of guest tools. Guest tools are often available to install on the VM host OS to optimize memory I/O such as Guest Ballooning, Hypervisor Swapping. The debate gets into the realm of religion. The gap between these two options is shrinking and likely supported in both manners by future releases. Fundamentally KVM is upstreamed into the Kernel while Xen is run in userland. Both of those have strengths and weaknesses. Here is a nice article describing Swapping and Ballooning. Another nice article from IBM can be read here.
Thanks for stopping by.
Hi Brent,
interesting post 🙂
My question:
Do you know of any (public) cloud providers who allow nested virtualization?
I think the probem is, that they need to expose the underlying hardware (HW virtualization support), right?
Best regards,
Volker 🙂
Hi Volker, I have only tried on the HP Public Cloud recently. I even tried build it with the standalone kernel modules for old kernels with no avail. You are right without HW Virt enabled its a no go, but I bet we start seeing it at some point, hypervisor networking will be important for tunneling hybrid deployments. Thanks for stopping by!
Hi. I’m Joseph P and also your big fan. 🙂
From your youtube videos, I can guess that your testing environment vms are in virtualbox on your macbook.
If you don’t mind, please tell me how can you enable kvm-ok on your vms.
I’ve got double check that I turned on vt-x option on virtualbox
but I can’t get the success message from kvm-ok from my ubuntu vms on virtualbox.
So I can’t test nested virtualization on my mac, neither.
Sorry for my poor English and humble question.
Thanks.
Excellent blog! Do you have any suggestions for aspiring writers?
I’m planning to start my own website soon but I’m a little lost on everything.
Would you propose starting with a free platform like WordPress or
go for a paid option? There are so many options out there that I’m completely overwhelmed .. Any recommendations? Thanks a lot!