Nested KVM Hypervisor Support

Nested KVM Hypervisor Support

Nested KVM Tutorial

Nested KVM Hypervisor Support

Adding Hardware Virtualization Support to a VM in KVM with Nested Hypervisor support is pretty important for folks wanting to do proofs for VMs, IaaS, OpenStack etc. It is fairly well supported now under both KVM and Xen. Not to mention, who wants to lab up bare metal when they can use their laptop for modeling. The major driver for nested virtualization is the ability to run nested Hypervisors in IaaS and/or Public Cloud services. It will be attractive for some to run VM farm as a service instances. This provides added flexibility to the customer to maximize their subscription of leased cloud instances.

Nested hypervisor Support On Laptops →

Add Nested Hypervisor support in the KVM-Intel or KVM-AMD KO Module


Boot the ISO with Nested Hardware Virtualization support. If you get
This is slightly different from the Rackspace guide. The difference is in the form of that loads the VMX HW virt support for the VM.

The ability to run nested KVM hypervisors is extremely important for lab environments for test/dev. For some reason KVM pulled support a while back but this is a work around for now written by IBM. More on it in a Sigg here and the code review from the legend himself Torvalds Github.

Booting a KVM Nested Instance

An example booting the RackSpace OpenStack ISO installer.

If you get errors when you try and run the qemu-system-x86_64 command they are likely missing the VMX support.

Booting a with an iso to build an image with an OVS tap:

Booting a built image with an OVS tap:

If you get errors when you try and run the qemu-system-x86_64 command they are likely missing the VMX support.

Para-Virtualization and Full Virtualization

Some hypervisors support Para-virtualizing and Full Virtualization in hypervisors and . Most hypervisors support paravirtualization through the addition of guest tools. Guest tools are often available to install on the VM host OS to optimize memory I/O such as Guest Ballooning, Hypervisor Swapping. The debate gets into the realm of religion. The gap between these two options is shrinking and likely supported in both manners by future releases. Fundamentally KVM is upstreamed into the Kernel while Xen is run in userland. Both of those have strengths and weaknesses. Here is a nice article describing Swapping and Ballooning. Another nice article from IBM can be read here.

Thanks for stopping by.


About the Author

Brent SalisburyBrent Salisbury works as a Network Architect, CCIE #11972. He blogs at NetworkStatic.net with a focus on disruptive technologies, that have a focus on operational efficiencies. Brent can be reached on Twitter @NetworkStatic.View all posts by Brent Salisbury →

  1. Volker KurthVolker Kurth09-16-2012


    Hi Brent,

    interesting post :-)

    My question:
    Do you know of any (public) cloud providers who allow nested virtualization?
    I think the probem is, that they need to expose the underlying hardware (HW virtualization support), right?

    Best regards,
    Volker :-)

  2. Brent SalisburyBrent Salisbury09-23-2012


    Hi Volker, I have only tried on the HP Public Cloud recently. I even tried build it with the standalone kernel modules for old kernels with no avail. You are right without HW Virt enabled its a no go, but I bet we start seeing it at some point, hypervisor networking will be important for tunneling hybrid deployments. Thanks for stopping by!

  3. JosephJoseph10-26-2012


    Hi. I’m Joseph P and also your big fan. :)

    From your youtube videos, I can guess that your testing environment vms are in virtualbox on your macbook.
    If you don’t mind, please tell me how can you enable kvm-ok on your vms.
    I’ve got double check that I turned on vt-x option on virtualbox
    but I can’t get the success message from kvm-ok from my ubuntu vms on virtualbox.
    So I can’t test nested virtualization on my mac, neither.

    Sorry for my poor English and humble question.
    Thanks.

  4. Louis CunninghamLouis Cunningham08-01-2013


    Excellent blog! Do you have any suggestions for aspiring writers?
    I’m planning to start my own website soon but I’m a little lost on everything.

    Would you propose starting with a free platform like WordPress or
    go for a paid option? There are so many options out there that I’m completely overwhelmed .. Any recommendations? Thanks a lot!