Simply put: Nested virtualization allows a virtual machine (VM) to be a host for further VMs. If you wanted to run VMware inside a VM, you would need this feature. But you would also do well to consider it if you were developing Android apps inside a VM and wanted to use the Android Emulator with hardware acceleration (= virtualization).
Yet, if you do not already know you need this feature, you probably should not be reading this post. Be warned: At the current state nested virtualization in Xen is considered “tech preview” requiring a recent processor and is not a production-ready feature. Citrix does not tell you about the feature and does not even document the parameter at all. I had to go through a bit of their open-sourced code to finally find the line.
Disclaimer done, lets start to play: To enable nested virtualization, go to the XenServer console (e.g. in XenCenter) and find out the UUID of the VM you want to permit being a virtualization host:
# xe vm-list
Go through the list and remember the first characters of your VM´s UUID. After making sure the VM is powered down, turn on nested hardware virtualization. Enter your VM’s UUID instead of <UUID>. After typing the first characters, you can hit TAB to auto-complete:
# xe vm-param-set uuid=<UUID> platform:exp-nested-hvm=true
Boot the VM and check if the virtual processor reports hardware virtualization (“vmx”) capabilities. In Linux this might look like this:
# cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 60 model name : Intel(R) Core(TM) i5-4460 CPU @ 3.20GHz […] flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl pni pclmulqdq vmx ssse3 cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes rdrand hypervisor lahf_lm abm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 smep bmi2 erms invpcid […]
If you see the vmx flag, your VM should be able to be a virtualization host itself – at least in theory: Remember this feature is experimental.
For further reading check out these links:
Thanks very much, Justus !
You saved me a lot of time.
Wow, that’s an excellent find!
I’ve been playing around with my home lab XenServer, but unfortunately I can’t get this to work on one of my Ubuntu VMs. Do you know if there are additional requirements with regards to the processor or VM?
The CPU I’m using is an Intel Core i5 3550 @ 3.30 Ghz. I think it’s relatively new and I hope it supports it. The Ubuntu VM is in para-virtualized. The setting seems to imply hvm?
Thanks for allowing me to pick your brain on this in advance.
Cheers,
Jan
Just converted the VM to HVW and it seems to work now. According to /proc/cpuinfo the CPU in the VM has a VMX flag 🙂
I’m a happy puppy!
Cheers,
Jan
Pingback: How to get Jenkins to build and deploy an OSv Unikernel
virt-host-validate returen following error for kvm
FAIL (Check that the ‘kvm-intel’ or ‘kvm-amd’ modules are loaded & the BIOS has enabled virtualization)
Was that error Message from a XenServer 6.5 or rather a newer version? Back when I played around with this feature, my server used Xen instead of KVM.
Hi,
I’m using virtual box on windows 10, i-5 7th gen.
On virtual-box, I installed xenserver 7.3 vm.
on xenserver vm, I installed an ubuntu appliance.
It is throwing error: “HVM is required for this operation”.
I have enable virtualization on xenserver using above commands but still getting same error.
also i’m not getting VMX flag on running : # cat /proc/cpuinfo
o/p:
flags : fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush mmx fxsr sse sse2 syscall nx lm constant_tsc rep_good nopl nonstop_tsc pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx rdrand hypervisor lahf_lm abm 3dnowprefetch avx2 rdseed clflushopt
Thanks in advance,
Ankit
Hi Ankit,
For the use case you describe, VirtualBox would need to support nested virtualization. You want to run XenServer, which is a virtualization system, within VirtualBox, which is another virtualization system. The use case I had was essentially to run a virtualization within a Xen VM. There, Xen(Server) was the “top-most” / physical-hardware-level hypervisor.
If you do succeed, please leave a note. I’d be highly interested if you managed to get it to run within VirtualBox.
Justus
it does work for enabeling HAXM emulator on a Windows VM for Android Studio, thank you !
Didn’t get to have my virtual device booting yet, but you’re help got me further ^^
Pingback: How to get Jenkins to build and deploy an OSv Unikernel
Pingback: Nested VM in XenServer - Boot Panic