Skip to content

Microsoft Office shows Network Printers multiple times when using Citrix Universal Print Server

We had some users reporting that all Office programs (Word, Excel, Outlook,…) show each Network printer multiple times in a XenDesktop 7 environment. The printers where connected from a central Print Server using the Citrix Universal Print Server. All users affected where using the same VDA Catalog.

When you checked the Devices and Printers area the network printers where only shown once.

Furthermore other applications only showed the printers one time.

After searching for some time I found a (hidden) Citrix Blog (!) post from 2014 which describes the issue for XenApp 6.5. Ok let’s check the mentioned Registry-Key



And look – it was the same problem. In the order key the Universal Printer was listed three times – and guess: The network printers are also shown three times.

So I removed the doubled entries and only kept the first Universal Printer. Moreover I removed empty lines at the end of the list.

After removing the entries and restarting the Print Spooler (with the corresponding Citrix Service) the duplicate printers in Word were gone.

Citrix PVS Reverse Imaging with Windows Backup

During the last weeks I had different problems with the “known” Reverse-Imaging Technics. One of them was the after reverse Imaging a Windows 2012 R2 Server the Server Manager didn’t start any longer.

Faulting application name: ServerManager.exe, version: 6.3.9600.17238, time stamp: 0x53d0b3e7
Faulting module name: ntdll.dll, version: 6.3.9600.18233, time stamp: 0x56bb4ebb
Exception code: 0xc0000374
Fault offset: 0x00000000000f1b70
Faulting process id: 0x1214
Faulting application start time: 0x01d1de6cee400844
Faulting application path: E:\Windows\system32\ServerManager.exe
Faulting module path: E:\Windows\SYSTEM32\ntdll.dll
Report Id: 2c2107c1-4a60-11e6-80f6-0050568817e7
Faulting package full name:
Faulting package-relative application ID:

Another was that the Server Manager wasn’t able to refresh the Roles and Features. So I wasn’t able to see (or add) any installed Roles / Features.

I tried different options to fix the problems – e.g. dism – none of them worked.

Sometimes even the complete reverse imaging failed with the following error.

Failed to load registry key {_BCD_} on \Boot\BCD. The system cannot find the path specified. (0x00000003)
Volume to Volume stopped
Volume to Volume lasted  834,7 seconds
Failed to convert Boot Configuration Data. The system cannot find the path specified. (0x00000003)

Also it didn’t make a difference if I used the P2PVS Tool….

…or the old BNImage.
The errors were different – but the reversed image was not usable.

So I started to look for a different method and found one: The integrated Windows Backup.

Before you can start you need to extend (or add) a separate drive from your Master-VM so that a backup of the whole system drive fits on it. I just extended the Cache Disk. You can’t use the local system drive for the backup.

As you can see my Cache Disk has now 200GB – more than enough for a 100GB System drive.

The next step is to install Windows Server Backup Feature from the Server Manager.

Confirm the installation.

And Close the installation when it’s finished.

Now start the Windows Server Backup and select Local Backup on the left side. On the right side choose Backup Once.

You can now only select Different options.

In the next step it’s important to select Custom – otherwise you can’t exclude e.g. the Cache Drive.

Now you have to Add Items for the backup.

Select System state, System Reserved and Local disk (C:).

Your selection should now look like this:

As the Backup Destination pick the local drive (which you extended or added at the beginning).

A last check if everything is correct before you can start the Backup.

Windows is now creating the backup which will take some time (like reverse imaging…).

After the backup it is important to clean the local system disc (not the PVS one the system is booted from Winking smile)

You can try to just format it…
… but I sometimes had problems with just formatted disks. So I suggest to remove the current disk from the VM and attach a new one with the same size.

The next step is to boot the VM from a Windows-Installation-Iso (with the same OS you plan to reverse image). Select your Language/Time/Keyboard and continue.

Now don’t choose Install now instead click on Repair your computer.

Go to Troubleshoot…

… and select System Image Recovery.

The shown OS Version must match your installed OS.

The wizard now detects the backup on the local disc – only if you created multiple ones you need to Select a system image.

No changes needed here.

Start the Restore with Finish.

You need to confirm that the local disc will be formatted and the system is restored.

The restoring starts…

.. after a successful restore an automatic reboot follows. The VM will still boot from the PVS.

So open the properties of your Master-Target Device and select Boot from Hard Disk.

That’s it. After another reboot your VM can start from the local Hard-Drive.

Hope the article was helpful for some of you Smile

Removal of Citrix XenDesktop / XenApp 7.x VDA fails with InstallFailure 1603

Some of you might have seen the following problem – the removal of the VDA failed:

If you check the details you get some more information’s – e.g. the removal of the Citrix Profile Management failed.

Removal of MSI Product ‘profilemgt_x64.msi’ (‘{9321930C-A53F-452C-80D5-AE1A8D69D9C0}’) failed with code ‘InstallFailure’ (1603).

Or the VdaMonitorPlugin_x64.MSI cannot be removed.

Well – but there is no solution shown how to fix it nor you can find a documentation from Citrix to solve the problem. You can’t just start the removal from the Control Panel – they are not shown under Programs.

Luckily Microsoft has published a really helpful Tool that helps to remove “problematic” Applications. After downloading open the Tool and press Next.

Now select Uninstalling.

You now get a much bigger list (compared to the one under Programs) – here you can see all Programs and their “Sub-Programs”. Now select your faulting part of the VDA – e.g. Citrix Profile Management.

Select Yes, try uninstall to start the removal.

The Tool now tries to remove the selected package….

… and resolves problems found during the removal.

After some time you should see the following dialog:
The problematic VDA-Component is now removed.

You can now restart the VDA removal like before – if another failing component is shown – just repeat the steps above.

Configuring a Nvidia Tesla M60 under Citrix XenServer (and assign a vGPU to a VM)

In this blog post I would like to show you how to configure a Nvidia Tesla M60 under a XenServer and deploy a VM with a vGPU assigned. There are a lot of good documentations from Nvidia for the different steps – but I didn’t find one complete for the whole process after putting the Tesla into the physical Server and installing XenServer.

In this post I will cover the following areas:
Installing the XenServer Nvidia vGPU Supplemental pack
Changing the Tesla M60 from Compute to Graphics Mode
Installing and Configuring the Nvidia Licensing Service
Configure a VM with a vGPU and assign a license

Installing the XenServer Nvidia vGPU Supplemental Pack

After the installation of the XenServer (I am not going to cover that here – there are enough other resources describing this) the necessary Nvidia supplemental pack is not available. You can see the installed packs in the XenCenter under the Server General Properties => Updates.

To download the necessary package you need an Nvidia Portal Account. Interestingly at the moment there is no Register-Button on the Login-Page. You can register on this page. The problem only is that you require either a valid Key or a Demo-Key. It’s not possible to get one here. So you need to contact an Nvidia Sales Guy to get a Demo key. After registering and logging in you have the area Product Download available. Here you can download the necessary extensions for XenServer, ESX and so on. Download the package corresponding to your XenServer version (currently 6.2, 6.5 and 7).
The download is a zip file containing different documentations, packs and applications. First of all you only need the following file:

To install the supplemental pack open the XenCenter and navigate to Tools => Install Update…

Confirm the Starting guide with Next.

Now you have to select the ISO file lower area Select update or supplemental pack from disk.
There are different supplemental packs for the different Nvidia vGPU cards available. One is for K1/K2 – the other for Tesla cards. The name of the package is different:
K1/K2: Contains the word Keplar
Tesla: without Keplar (see screenshot).
At the moment it is not supported to run cards of both types in the same server (thanks @Rachel Berry for the information).
Furthermore it is currently not possible to remove an installed pack – so if you have a Tesla card and accidentally installed the Keplar pack you have to reinstall your XenServer.

After choosing the ISO select on which XenServers you would like to install the pack.

The pack will be uploaded to the XenServer.

After installing the updates some tasks have to be done (e.g. rebooting) – you can either let that happen automatically (from the XenCenter) or do that manual (like you know from every other update).

The installation starts, VMs (if there are any) are migrated off the XenServer and it is rebooted (if you selected automatically before Winking smile).

After the reboot the Maintenance mode is quite and some checks are done.

If you now check the Updates section again you should see the NVIDIA vGPU package.

Moreover you can run the following command on the console to check if the pack is correctly installed:

lsmod | grep nvidia

If everything is correct the output should look like this.

When you now open the GPU area of the XenServer it might be possible that still no vGPU Profiles are shown on the right side – also the Tesla Card is correctly detected. The reason for this is that the Tesla card is often shipped in Compute mode and not in Graphics mode.

Changing the Tesla M60 from Compute to Graphics Mode

To check in which mode the Tesla card is running open the XenCenter Console. Now enter the following command:

lspci -n | grep 10de

The red marked area shows the mode:
0302: Compute
0300: Graphics

To switch the mode, you need an Utility from Nvidia. You can either Download this from the Product Download page (after logging in) or under the Nvidia support download for the Tesla cards.

The downloaded Zip-Files contains the following files. There are different options how to change the GPU mode (e.g. install Linux) – to make it easy we will integrate the utility in the XenServer. Thus we only need the file gpumodeswitch.
Important: Not the file with the extension exe – the file without an extension (the first in the screenshot).

To copy the file to your XenServer you need to start WinSCP. Select SFTP, enter the XenServer IP-Address as the Host name. Username is root with the same password you use in the XenCenter.

Confirm the Warning.

Now copy the gpumodeswitch file to a temporary folder (e.g. tmp) on the XenServer.

Switch to the command line from the XenServer and navigate to the tmp folder. To show the current GPU mode enter the following command:

./gpumodeswitch –listgpumodes

But what’s that – the permission is denied.

To allow the execution of the file you have to modify the file permission. Enter the following command:

chmod 700 gpumodeswitch

Now you should be able to execute the gpumodeswtich utility. After the execution you get another warning that first the NVIDIA kernel driver has to be unloaded.

Before you can remove the driver first stop the xcp-rrdd-gpumon service:

service xcp-rrdd-gpumon stop

Now remove the unload the NVIDIA driver.

rmmod nvidia


You can now change the graphics mode with this command:

./gpumodeswitch –gpumode graphics

The mode change needs to be confirmed with Y.

The Tesla cards now receive the graphics mode firmware.

After the update finished you can again check if the graphics mode is now active.

lspci –n | grep 10de

In the red marked area you now see a 300 for all Tesla cards – the Graphics mode is now active.

If you switch to the GPU area from the XenServer the different vGPUs are shown.

Installing and Configuring the Nvidia Licensing Service

The next step is to install and configure the Nvidia Licensing service. The necessary downloads are also available in the Nvidia portal under Product Download.


The package contains a setup.exe and two documents. Start the installation with the setup.exe

If you haven’t read the requirements you might have missed that the service requires a Java 32Bit installation.

So first install Java….

And then start the setup again.

Accept the license agreement for the Nvidia software.

And also accept the license for Apache – an Apache Tomcat is used for the license service.

You can now change the installation path (if you like).

The default setting is to configure the Windows Firewall to only allow connections to the license service itself – not to the management website. If you want to access the website not only from the license server itself (and the Windows Firewall is active) you need to activate the Management interface as well.

A summary shows the installation path and the required disk space.

The license service (including Apache Tomcat) is now installed.

Finish the installation with Done.

The first check is now if the tomcat is running. Open a browser on the server and open http://localhost:8080 – you should now see the Tomcat welcome page if everything is running correctly.

To open the Nvidia license server enter http://localhost/8080/licserver
The first things shown are the Licensed Clients – of course there is no client shown until now because we didn’t configure a client or install a license.

The licenses are managed under License Management. As you can see you need to upload a license file.

To get the license file browse again to the Nvidia portal and switch do License History. Here you can see all of your licenses. Select the licenses you would like to assign.

For assigning the licenses you need the MAC address of the license server.

To get the MAC open a Command prompt or a Windows PowerShell on the licensing server. Enter the following command:

ipconfig /all

Search for the Ethernet-Connection you plan to use and note the Physical Address.

Enter the Physical Address in the MAC address field (without any signs like “-“ or “:”). Add a free choose able Alias and Site Name.

The server is now created – but no licenses are assigned. Choose Map Add-Ons to add a license.

Press Search to view your current licenses. Enter the Qty to Add of the licenses you want to assign and press Map Add-Ons.

Now the licenses are mapped to the server. Select Download License File to download the license BIN file.

Put the downloaded BIN file on a folder on the license server.

Now open again the license management console and go to License Management.

Select the downloaded BIN file and press Upload.

A sucessful upload is confirmed with Successfully applied license file to license server.

Go to License Feature Usage to show the currently available licenses.

Configure a VM with a vGPU and assign a license

The last configuration step is to assign a vGPU and install the Nvidia Drivers inside the VM.

Open the properties of a VM (I am not going to cover how to create a VM and install Windows on the VM) and switch to GPU.
Now you have to select the vGPU-Profile.
Depending on the profile you need a different license. The profiles are split into the following licenses:
A: Grid Virtual Applications
B: Grid Virtual PC
C: Grid Virtual Workstation

You can find a good comparison of the types in this blog post from Richard Hoffman.

When you now start the VM and check the Device Manager you will still only see a Default VGA Graphics Card. To change that the necessary drivers need to be installed.

Copy the ones for your OS to the VM.

Start the setup and confirm the Windows UAC Message with yes.

Select a folder to extract the driver files.

Now the actual setup starts. Agree to the licenses,

Either choose Express or Custom installation.

There is no difference between the installation except that in a Custom installation you can see which components are installed and it’s possible to Perform a clean installation if you had problems with the currently installed driver.

The installation starts.

After the installation is finished a reboot is necessary.

If you now open the Device Manager again and check the Graphics Card you will see a NVIDIA GRID card – here you can also see which vGPU-Type is assigned.

The last step is to assign a license to the VM. Open the Control Panel and search for Nvidia.

Open the Nvidia Control Panel.

Now select License management. Here you can enter License Server FQDN and the configured Port (default: 7070).

After entering the information’s press Apply to assign a license.

If everything is working correctly the license is now applied.

If you now open the License Server Management Interface again you will find the client under Licensed Clients.

For each client you can also open get detailed information’s.

That’s it. Now you can continue with everything else (e.g. Citrix XenDesktop HDX 3D Pro VDA). Hope the article helped some of you Smile

Update to Exchange 2016 CU1 fails with WMSVC error

During the last days I had to update a few Exchange 2016 servers to CU1. Unfortunately the update failed with the following error:

The following error was generated when "$error.Clear();
          $keyPath = "HKLM:\Software\Microsoft\WebManagement\Server";
          if (!(Get-Item $keyPath -ErrorAction SilentlyContinue))
            New-Item $keyPath -Force
          Set-ItemProperty -path $keyPath -name "EnableRemoteManagement" -value 0x1 -Type DWORD -Force;

          if (Get-Service WMSVC* | ?{$_.Name -eq ‘WMSVC’})
            Set-Service WMSVC -StartupType Automatic
            Stop-SetupService -ServiceName WMSVC;
            Start-SetupService -ServiceName WMSVC
        " was run: "Microsoft.Exchange.Configuration.Tasks.ServiceDidNotReachStatusException: Service ‘WMSVC’ failed to reach status ‘Running’ on this server.
   at Microsoft.Exchange.Configuration.Tasks.Task.ThrowError(Exception exception, ErrorCategory errorCategory, Object target, String helpUrl)
   at Microsoft.Exchange.Configuration.Tasks.Task.WriteError(Exception exception, ErrorCategory category, Object target)
   at Microsoft.Exchange.Management.Tasks.ManageSetupService.WaitForServiceStatus(ServiceController serviceController, ServiceControllerStatus status, Unlimited`1 maximumWaitTime, Boolean ignoreFailures, Boolean sendWatsonReportForHungService)
   at Microsoft.Exchange.Management.Tasks.ManageSetupService.StartService(ServiceController serviceController, Boolean ignoreServiceStartTimeout, Boolean failIfServiceNotInstalled, Unlimited`1 maximumWaitTime, String[] serviceParameters)
   at Microsoft.Exchange.Management.Tasks.ManageSetupService.StartService(String serviceName, Boolean ignoreServiceStartTimeout, Boolean failIfServiceNotInstalled, Unlimited`1 maximumWaitTime, String[] serviceParameters)
   at Microsoft.Exchange.Management.Tasks.StartSetupService.InternalProcessRecord()
   at Microsoft.Exchange.Configuration.Tasks.Task.<ProcessRecord>b__b()
   at Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String funcName, Action func, Boolean terminatePipelineIfFailed)".


One part of the error message was really important:
Service ‘WMSVC’ failed to reach status ‘Running’

So i opened the services console and tried to start the service Web Management Service (WMSVC) manual – which also failed. In the event log the following two errors were logged when I tried to start the service:

Log Name:      Application
Source:        Microsoft-Windows-IIS-IISManager
Date:          23.06.2016 10:17:52
Event ID:      1007
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      SERVER-FQDN

Unable to read the certificate with thumbprint ‘29827623d1eec8d17743d4bca2beb9cbcb2027d8’.  Please make sure the SSL certificate exists and that is correctly configured in the Management Service page.

Log Name:      System
Source:        Service Control Manager
Date:          23.06.2016 10:17:52
Event ID:      7024
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      SERVER-FQDN
The Web Management Service service terminated with the following service-specific error:
Unspecified error


The first error shows quite clear what the problem is:
Unable to read the certificate

To check the certificate for the Web Management Service you have to open the IIS Manager, select the current server and switch to the Features View. Now open Management Service.

As you can see here there is no SSL certificate selected.

Select a (valid) certificate and apply the settings.

After that you should be able to start the WMSVC service again and the Exchange setup should continue. The reason that the problem happened is that the local certificate was renewed a few days before. In the default IIS configuration renewed certificates are not automatically rebound. To fix this open: IIS Manager => Server => Feature View => Server Certificates
Now switch the setting on the right side and enable Automatic Rebind of Renewed Certificate.

That’s it Smile

No vGPU is available with NVIDIA Tesla M60 on Citrix Xenserver

During the last days I had to invest an interesting problem. On a XenServer there were no vGPUs available on a Tesla M60 from NVIDIA. Only GPU Pass-through was available.

Ok – time to check if the necessary supplemental pack is installed:

As you can see this was installed – so maybe the Tesla is running in Compute-Mode instead of Graphics-Mode. To check this, connect to the console of the XenServer and login. Now run the following command:

lspci -n | grep 10de

This will show you all available cards with their current configuration (compute or gpu):
If the marked value is 0300 everything is fine and the card is in graphics mode. If it’s 0302 it’s in compute mode and you have to change it to graphics mode.

So it looks everything correct – but vGPUs are still not available. Possible it’s just a problem with the NVIDIA vGPU pack – let’s reinstall it with the XenCenter.
Tools => Install Update
Choose “Select update or supplement pack from disk” and browse to the vGPU Iso from the Nvidia Driver Download Package.

Continue with the installation – but what’s that:
The installation fails (without a detailed information why).

So time for the manual way– copy the vGPU rpm File to the XenServer with WinSCP.

And start the installation with the following command (after browsing to the correct folder):

rpm -Uv NVIDIA-vGPU-xenserver-7.0-361.45.09.x86_64.rpm

But guess what happened again? The installation failed with the following messages:

Preparing packages…
file /usr/share/doc/NVIDIA_VGX/LICENSE from install of NVIDIA-vGPU-xenserver-7.0-361.45.09.x86_64 conflicts with file from package NVIDIA-vGPU-kepler-xenserver-7.0-361.45.09.x86_64
file /usr/share/hwdata/pci.ids.d/nvidia.ids from install of NVIDIA-vGPU-xenserver-7.0-361.45.09.x86_64 conflicts with file from package NVIDIA-vGPU-kepler-xenserver-7.0-361.45.09.x86_64
file /usr/share/nvidia/monitoring.conf from install of NVIDIA-vGPU-xenserver-7.0-361.45.09.x86_64 conflicts with file from package NVIDIA-vGPU-kepler-xenserver-7.0-361.45.09.x86_64
file /usr/share/nvidia/vgpu/vgpuConfig.xml from install of NVIDIA-vGPU-xenserver-7.0-361.45.09.x86_64 conflicts with file from package NVIDIA-vGPU-kepler-xenserver-7.0-361.45.09.x86_64


Luckily this time the error message is much more helpful – it tells that there is a conflict with the NVIDIA-vGPU-kepler-xenserver-7.0-361.45.09.x86_64 file.

So why can’t I upgrade the vGPU Package? The most important part is the word KEPLER in the conflicting file. A lot of people don’t know (and it’s not well documented until now) that NVIDA deploys two different versions of the vGPU Package. One is valid for GRID K1 and K2 – the other one for the Tesla cards. So at the end you might have two packages:
K1 + K2:

If you now think – ok just remove the K1/K2 package – sorry that’s (still) not supported in XenServer (XenServer 7.0 – Point 7.4):

At present, uninstallation of supplemental packs is not supported. However, to ensure that it is simple to provide such a feature in the future, all RPMs that are included in supplemental packs must be able to be uninstalled cleanly (using rpm -e). This includes reverting any configuration such as firewall rules or log rotation configuration.

The only (supported) solution is to completely reinstall the XenServer. After that install the Tesla vGPU Package and you should be fine and see all the available vGPU Types:

Hope that helps some of you to know that there is a difference between the K1/K2 and Tesla package.

There are only a few questions which are still open for me at the moment:
Why is it necessary to switch from one vGPU Type (K1/K2 to Tesla) to reinstall the XenServer?
What happens if a customer would like to use a K2 and Tesla M60 in one physical Server?

Upgrading Citrix NetScaler 10.5 to 11–SharePoint Websites might fail to load

During the last weeks I was facing a really strange problem. After upgrading a NetScaler from Version 10.5 to 11 one specific SharePoint Website was not accessible any longer. Either it looked like this – so it was not completely loaded:
(As you can see the browser showed “ERR_CONNECTION_ABORTED”.)

Sometimes it even directly stopped when “Is loaded” was shown.

Interestingly other websites – opened through the same NetScaler – worked without any problems. Furthermore there were no specific Actions / Policy bound to the Virtual Server – it was just a load balancer.

We tested different things and even debugged our Backend-Systems. Unfortunately we didn’t find any problems. Luckily one day I discussed the problem with a Citrix SE. He told me that he was facing the same problem when the NetScaler Insight Center (and so AppFlow) was used to monitor a Virtual Server. I checked the Insight Center and guess what – the Insight was enabled for the Website which failed to load after the NetScaler Upgrade.
netscaler insight center

So i disabled the monitoring – upgraded the NetScaler – checked the website – Problem fixed Smile
I still don’t get why AppFlow prevents loading a load balanced WebSite – but if you have problems like that just try to disable AppFlow.

%d bloggers like this: