Skip to content

Workaround for crashing XenServer 7 with Xeon v4 CPUs, Nvidia M60 / M10 (and Dell R730)

8. February 2017

You might have read my last article about a crashing XenServer 7 when it’s used with an Intel Xeon v4 and a NVIDIA M60 in a Dell R730. The same problem happens when you use a M10 from NVIDIA. Furthermore I have heard that the same problem happens with Fujitsu Servers.

Our workaround until now was to replace the Xeon v4 CPUs with v3 ones (with less performance). Luckily Citrix Engineering (and Support) found the issue for the problem. It’s related to the Intel PML (Page Modification Logging) function. You can find some detailed information on this side from Intel.

Thus we disabled this function in XenServer and replaced the v3 CPUs with v4 ones. After that we started stress testing the system. No error occurred. Thus we placed users on the system. The system was still running. Until now we had no more crashes while the function is disabled.

To disable the function you need to modify the grub.cfg – those of you who have more than 512GB Ram in their hosts should already now that process.

Open a console session and either switch to /boot/grub (Bios) or /boot/efi/EFI/xenserver (UEFI) – depending on the boot setting of your server.

BIOS:
/boot/grub

01_no-pml

UEFI:
/boot/efi/EFI/xenserver

02_no-pml

Open grub.cfg with vi:
vi grub.cfg
03_no-pml

Now you need to add the following command to the multiboot2 /boot/xen.gz line:

ept=no-pml

After that the file should look like this (don’t remove iommu… if you have more than 512GB Ram):
04_no-pml

Save and close the file with :wq and reboot the host. That’s it. Now the function is disabled and the problem is gone. A big thanks to Antony Peter (Citrix Support) and Anshul Makkar (Citrix Engineering) for taking so much time to look at our environment and debugging the problem with me.

One Comment
  1. Tobias Knolmar permalink

    Sehr guter Beitrag!!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: