VMware ESX & ESXi Error – Can’t boot system as genuine NUMA

When running a virtualized instance of ESX/ESXi 4.0 Update 1 from within VMware Fusion or VMware Workstation you may notice the following error message on the front ESX/ESXi screen:

“NUMA: 706: Can’t boot system as genuine NUMA. Booting with 1 fake node(s)”

I started experiencing this message after beginning to run VMware ESXi (4.0 Update 1) on my new MacBook Pro with VMware Fusion. As you see from the screenshot below the CPU in my MacBook Pro is an Intel i7 with a pair of vCPUs and 4GB of memory allocated to the ESXi VM instance.

VMware ESX ESXi NUMA Can't boot system

 

The good news is that despite the message you can still run VMware ESX/ESXi under VMware Workstation or Fusion without any issue though you may want to disable this message, which can be easily achieved via the following couple of steps:

1.  Open the vSphere Client

2.  Choose you ESX/ESXi host within vSphere Client, select the ‘Configuration’ tab and then ‘Advanced’ Settings

ESX ESXi NUMA

3.Select ‘VMkernel’ and ‘Boot’, then scroll down to almost the bottom of the ‘Boot’ settings.  Here you will find the ‘VMkernel.Boot.useNUMAInfo’ option.  This is option consists of a check box which you can toggle to enable/disable the NUMA Information.

 

vSphere NUMA Workstation

 

Give the ESX/ESXi host a reboot and then you’re good to go and should receive the NUMA message anymore.

This fix is also outlined in this VMware Knowledge base article (KB1016141).

 

But what is NUMA?  With modern server architecture and vSphere, although you are not likely running VMworkstation on server grade hardware, it is useful to know NUMA stands for and how its shared memory architecture applies to an ESX/ESXi host.  The following is a good description from the “VMware vSphere 4: The CPU Scheduler in VMware ESX 4” white paper:

In a NUMA (Non-Uniform Memory Access) system, there are multiple NUMA nodes that consist of a set of processors and the memory. The access to memory in the same node is local while the access to the other node is remote. The remote access takes longer cycles because it involves a multi-hop operation. Due to this asymmetric access latency, keeping the memory access local or maximizing the memory-locality improves performance. On the other hand, CPU load-balancing across NUMA nodes is also crucial to performance.

The NUMA load-balancer in ESX assigns a home node to a virtual machine. For the virtual machine, the memory is allocated from the home node. Since the virtual machine rarely migrates away from the home node, the memory access from the virtual machine is mostly local. Note that all vCPUs of the virtual machine are scheduled within the home node.

If a virtual machine’s home node is more heavily loaded than others, migrating to a less loaded node generally improves performance, although it suffers from remote memory accesses. The memory migration may also happen to increase the memory-locality. Note that the memory is moved gradually because copying memory has high overhead.

So in a nut-shell,  a NUMA node consists of a physical CPU and an allocation of physical memory to which a VM and its vCPUs and memory are then allocated to.  There is an excellent article from Frank Denneman which covers the topic of sizing CPU and NUMA nodes in which he goes into a good level of detail around NUMA along with a few of the more common gotchas – definitely worth a look.  It is worth pointing out that for ESX/ESXi to apply NUMA nodes the underlying physical(s) CPU and architecture of the ESX/ESXi host must be NUMA compliant and have the associated NUMA architecture to support this mode.  Modern Intel based CPUs such as the Nehalem and Opteron processors from AMD are such processors.

In most instances where VMware workstation or Fusion is used to run VMware ESX/ESXi it is being run on a non-NUMA based system, with the underlying workstation or laptop also only having a single physical CPU installed.

 

About Simon Seagrave 706 Articles
Simon is a UK based Virtualization, Cloud & IT Technology Evangelist working as a Senior Technology Consultant and vSpecialist for EMC. He loves working in the ever changing IT industry & spends most of his time working with Virtualization, Cloud & other Enterprise IT based technologies, in particular VMware, EMC and HP products. As well as on this site, you can find him on Twitter and Google+

Be the first to comment

Leave a Reply

Your email address will not be published.


*


This site uses Akismet to reduce spam. Learn how your comment data is processed.